I've posted this on HN before, but in the world of optimization, I really like the book Convex Functional Analysis by Kurdilla and Zabarankin. It's the only book that I know of that gives a comprehensive treatment of when a solution to an optimization problem exists and can be attained. Essentially, the entire book builds up to what they call the Generalized Weierstrass Theorem, which gives the appropriate conditions in a few different ways.

Anyway, I work on optimization solvers and this is a big deal since there's always a question in whether or not the problem we pose can really be solved. Numerically, the solver can churn forever and maybe it's just hard, or maybe there's no solution. Outside of LPs, it's really hard to tell. Even convexity isn't enough for a solution. For example, min exp(-x) doesn't exist even though exp(-x) is strictly convex. There's an infimum, however.

Mostly, this is a way to say that I agree that the theory matters. I like the book above for establishing these conditions in optimization.

Edit 1: Does anyone know a good, nonnonsense book for establishing similar conditions for either ODEs or PDEs? In case it matters, I prefer looking at things from a general functional analysis/operator point for a view.