But math is a broad field so you're going to have to pick specific courses. For example, partial differential equations are quite common. But if you learn it in a math department it'll have proofs and full rigour just like a course in set theory. While some techniques for solving them will be covered, we mostly study the underlying mathematical structures, why certain techniques work, etc. If you take it in a physics department they'll teach you techniques and numerical methods to solve a certain class of problems like heat equations or fluid dynamics. But learning through direct applications will inevitably limit you to those techniques while learning things in full generality tends to make it easier to pick up specific techniques when needed.