What are Differential Equations and how do they work?

I recommend James Murray mathematical biology book for nice examples of how ODEs can quickly give insights into complex phenomena.

This is called "modeling", and together with analysis and simulation is one of the three main parts of the theory. Where did you study differential equations? From my experience, it is typically taught that way, at least in the european universities that I know. Even in the most hard-core mathematics departments they may focus strongly on the analysis part (existence, uniqueness, regularity) to the detriment of a bit of modeling or simulation; but I have never seen those hidden. The "algebraic" solution of the equations is a fringe subject that is most often omitted, (except when it leads to a simple analysis, e.g. linear equations).

IMO, I really enjoyed learning about differential equations (DEs) and found that solving them analytically, besides being a nice intellectual challenge, also helped me with advanced courses later on. All of AC circuit analysis and the signals theory classes you take as an electrical engineer, for example, are firmly rooted in the language of DEs and just being able to plug some equations into a computer misses out on all of the real understanding you need to have in those fields to use them in practice.

One of the big red flags a junior electrical engineer can show, in fact, is too much reliance on simulation tools without having a good background understanding of what to expect from them.

We can argue about how much time he and I put into the problem from ground zero. The thing I’m arguing is that one can spend a lot of time misusing the building blocks if one doesn’t understand them. Existence and uniqueness seem hoity toity until you try to numerically solve an equation over a domain for which a solution doesn’t exist, or you try to solve it and you get one of many solutions and think you’re done.

It’s hard to measure the value of knowing the fundamentals... especially 10 years out when it seems like you’re reusing the same tiny subset of your toolset over and over again. The fundamentals though exist as a little voice in the back of your head reminding you not to do something that won’t work.

> Because in this way you can obtain solution to any DE with any degree of precision (within reasonable assumptions about complexity of the problem).

IMHO one of the key goals of education is exactly to understand why/when this is not the case, so you don't shoot yourself in the foot. Numerical methods will often generate junk results (too many examples to count, of people running useless simulations to problems whose solutions don't exist, or algorithms don't converge, so that the results of numerics are fundamentally misleading).

Further, probably one of the most valuable topics is an understanding of linear differential equations (and linear shift-invariant systems more generally), including typical solutions and how they compose. A huge chunk of our science & engineering knowledge is built on this edifice. This is hardcore theoretical stuff (ideas & concepts), that can't be derived from being able to numerically solve differential equations. Further, a solid understanding of these concepts allows one to decompose a problems solutions into eigen-modes, and build much better solvers specialized to a situation.

Your assertion doesn't make sense. Runge-Kutta does not solve differential equations. That's not what they are used for. A solution to a differential equation is an equation that satisfies both the differential equations and the boundary conditions. Runge-Kutta, or any other iterative numerical method, does nothing if the sort. The best they do is provide estimates on what values the solution might exhibit at a given point within their domain. These estimates may be of use for some application, but they are not solutions.

If you care about solutions to differential equations but for some reason don't want to learn methods that allow you to determine their solutions then the next best thing is getting up to speed with approximate methods such as the Galerkin method, which for some classes of problems and choices of trial functions can actually output exact analytical solutions.

The typical progression is to start with systems you can solve by hand and then move to ones you can’t usually solve, but can still usefully describe, teaching how to analyze them.

It should be the other way around! Start with generally useful tools that apply broadly and then teach the special cases you happen to be able to solve analytically. General tools like basins of attraction and nondimensionalization, should be considered more rudimentary than perfect closed form solutions.

> Because in this way you can obtain solution to any DE with any degree of precision (within reasonable assumptions about complexity of the problem).

This is actually an entirely different class of thing, and it's important not to confuse them.

That's kind of like saying, "why would I ever need a function that can compute square roots at run-time? Just tell me whatever number it is you want the square root of, and then using my pocket calculator, we can hard-code its square root to any number of digits of accuracy at compile-time!"

A numerical solution is a specific answer only to a specific input; depending on what you're trying to do, that might be useful or it might be absolutely useless. An analytical solution is like a short "run-time" program that computes solutions for given parameters, and which can often be reversed and inverted.

One example where it's especially painful is when you're trying to do the inverse: find the correct differential equation that results in a desired solution. (AKA a design problem: you need to design a filter, oscillator, linkage, control law, etc, that results in a desired behaviour). This isn't just about ivory tower mathematicians! Engineers need to do this! An analytic solution makes this easy.

Imagine if you, an electrical engineer, only know how to be handed a given schematic, break it down into the DE's that govern it, and then numerically solve them. You wouldn't be able to design a new circuit, just predict the behaviour of existing ones.

Imagine needing to run sweeps of SPICE simulations over wide ranges of resistor and capacitor values to design a low-pass filter just because you never learned how an RC filter works. It's not that you can't do that, and in some cases where you're dealing with nonlinear systems it's the most pragmatic approach. But that's also a very simple example; as the system gets more complex, the curse of dimensionality will make that more and more challenging, while a system of linear differential equations could continue to have exact solutions no matter how many dimensions you have, and furthermore, make it easy to pick out exactly which variables are the most critical.

Or maybe you want to find an optimum point. You can do numeric optimization, but again, you'll get hit with the same curse of dimensionality if the system gets complicated. But if you have an algebraic solution, you might be able to pick out a formula for it. Then not only will you get the optimum (either its coordinates, or its value), you also get an equation that lets you see, in functional form, how that optimum point moves around as you change the other variables in the problem. Then maybe you can maximize that if you want to, and so on.

In other cases, being able to remove even one dimension from the analysis can be used to speed up a numerical solution over all the other parameters. Like if you're designing a plane and you have to search over various wingspans, pitches, taper, and so on... if you can get an analytic equation that gives you an optimum wingspan for a given pitch and taper, then you can reduce your search space by an entire dimension, which has a huge impact.

The statement "you can obtain solution to any DE with any degree of precision" is practically not true for many systems. Despite massive increases in computational power, I don't know anyone who (e.g.) believes that turbulence will be solved any time in the next few decades solely by throwing more computing power at the problem. Through a combination of experimentation, analysis, and computation, however, we have made a lot of progress.

One major advantage of analytical methods is that they more easily allow you to see larger scale trends in the problem. If a computation is expensive, calculating sensitivities can be difficult, and in my experience sensitivities are found in CFD less often than they should be. Computation in some sense is more like experimentation: it gives you a local value. You need to do multiple runs to look at trends.

I think most people also underestimate how far analytical methods can get. In my PhD, I produced an accurate approximate solution to a nonlinear series of ODEs relevant to the problem I was studying. This gave a lot of insight into the problem that researchers using numerical methods for decades apparently couldn't figure out.

By the way, I approximated the unsolvable nonlinear system of ODEs there as a different nonlinear system of ODEs, albeit an exactly solvable system. Most people linearize, but that would have lead to a far less accurate solution in this case. Many nonlinear differential equations can be solved exactly; you can see a catalog of these at this website and in the associated books: http://eqworld.ipmnet.ru/

It's really quite similar to integral tables. You look up the form of the equation you have and they tell you the solution (often with a proof or citation for the proof) or at the very least give you a head start if they don't have the solution.

I'm working on another theory project right now and I intend to use the same "approximate an unsolveable nonlinear system with a solveable nonlinear system" trick here.

In my engineering program, they did something like this, though it was weak on the differential reasoning part. And heavy in hand-coding numerical solvers in FORTRAN.

1) simply solving an equation is not enough, you have to understand the dynamics, how changes in parameters change the solution

2) some models simply don’t have solutions that are straightforward to calculate

But 50 years later, I still have those gonna-flunk dreams.

I think, I only passed because my discrete math prof told my calculus prof I'm not an idiot.

I strongly encourage you to relearn these concept with OCW MIT with Strang and Moler [1].

[1] - https://www.youtube.com/watch?v=ZvL88xqYSak&list=PLUl4u3cNGP...

Don't trust your teacher, read 4 different textbooks if that's what it takes to make it stick - it doesn't matter whether is dry as a desert or visual, just whatever works. The amount of intuition you have grows exponentially.

The rule of thumb that I live by is that learning mathematics is difficult, and teaching mathematics well is almost impossible - prepare before hand. I don't think I've learnt anything in a lesson or lecture post-algebra, that may not be the best thing but I have a very good feel for what I need to learn because of it.