How to Turn Physics into an Optimization Problem

A couple points:

> In Lagrangian Mechanics you minimize the total action of a system to find its motion

Strictly, realizable paths are an extremum (maximum, minimum, or inflection point) of the action.

> it’s representation invariant

This form of representation invariance is true in Newtonian and Hamiltonian mechanics as well. It's a statement that physics is invariant under changes of coordinates. I think a better way of stating what you're after is that you work with a function of the whole state of the system rather than having to do the double entry bookkeeping for all the interactions among subsystems of what you're modeling.

> Studying the “shapes” of systems in this manner is part of larger field called Topology.

The geometry of configuration space (for Lagrangian mechanics) or phase space (for Hamiltonian mechanics) isn't really part of topology. For phase space, it's properly called symplectic geometry. For configuration spaces, it's just high dimensional Euclidean geometry. It is sometimes interesting to look at the topology of these spaces, but it's much more than just topology.

I'm glad to see a mention of SICM. That really is a great book. It might be worth pointing out that the field of mathematics that leads to the Euler-Lagrange equations goes under the name of "calculus of variations." I learned it from Weinstock's excellent old book by the same name.

Maybe also worth noting that you can't add non-conservative forces to Lagrangian mechanics in any generally accepted way, so if you have friction in your system, you're stuck in Newtonian.

The article claims "In Lagrangian Mechanics you minimize the total energy of a system to find its motion". But if you actually look into it, you derive Lagrangian Mechanics by minimizing (or maximizing) the _action_. Action in this context is defined as the integral over the Lagrangian, or in other words the _difference_ between kinetic and potential energies (not their sums). The author knows that because later he states "All physical processes take the path that minimizes total action." and gives the corresponding mathematical expressions.

Also the statement about the Lagrangian being representation independent is misleading. The formalism stays the same and this is great for picking "suitable" coordinates that makes solving the system easier, but how the Lagrangian looks and how the equations of motion for the coordinates look can be quite different. This is actually what makes Lagrangian mechanics so powerful. You can transform coordinates (possibly several times) to get the Lagrangian into a simple shape and get equations of motions you can actually solve. You have transformed some of the difficulty of solving differential equations into the difficulty of finding natural coordinates, something humans tend to be better at.

The point that the multipliers \lambda_i that appear when modeling constraints should be names "Lagrange multipliers" has already been raised.

I'm not sure what the author's target audience is, but:

> But we know from high school physics that a = v' = p''

It would be exceedingly rare to see "p" used as the position variable in high school physics - it's almost exclusively reserved for momentum.

Most high school students do 1D physics with position as "x". Those who go on to study more physics usually use "s" as their displacement function/vector and maybe "r" as a position vector.

(Also, the overwhelming majority of high school physics students never touch calculus-based physics -- only about 50-60 thousand students take the AP Physics C exam each year. 5 times that take AP Physics 1, and even more take non-AP Physics.)

In case you want to know more about the constraints, the search term is Lagrange multiplier.

Names shouldn't be omitted because they are essential to searching for details. A name might make sense once you know it but you can not really derive it. If you do not know it, you are lost.

Nice article by the way!

Nice write up, truly after my own heart! Part of my research had to do with using Lagrange's method of multipliers to design B-spline geometry via solution of constrained nonlinear optimization problems, where we had these bending energy measures included to smooth the resulting curves and surfaces. I realized in the course of things that the functional I was minimizing had this parallel with Physics. The answer to so much is at the minimum! Nice to see somebody with the same train of thought put it to print!

Annd... From the perspective a freshly minted PhD, who was always in a terrible fright before giving a talk, I look at the critiques in the following way: It's hard to put yourself out there about such a topic, knowing at the same time such awe at the beautiful, towering edifice of mathematical theory, experiment, and geometry that makes up physics. But the critiques help refine one's thinking, and are always to be welcomed. Good on them for pointing out the places where things need to be sharpened a bit. At the same time, I love the big picture. That's what keeps me coming back. That's the viewpoint where there are such similar structures and relationships between the way we solve problems in far flung fields. One can leverage what one has learned in one field to get a head start, (or at least a toe-hold!) learning another. Cheers.

I don't know if this is a good analogy, but as an Electrical Engineer, the process of simplifying the analysis by going from Newtonian to Lagrangian gave me flashbacks from the first time we used Laplace transforms to solve (previously very, very tedious) circuit analysis problems. Almost felt like cheating.

Maybe someone finds this interesting. This paper seems to be related: https://arxiv.org/abs/1907.04490

Fantastic article but... “ The implementation here is in LISP which is a great language to learn but in my experience a tough language to use on larger projects.”

OBJECTION YOUR HONOR!!

Regarding SICM, Sussman et al. point out that the traditional notation for differential calculus contains "type errors". In particular they point this out regarding the Euler-Lagrange equations. And they then make the point that their "functional" notation (based on Spivak Calculus on Manifolds) is free of these type errors.

Obviously Sussman has a special relationship to Scheme, but I am curious whether it would have been beneficial for the code implementation (scmutils) to have used a statically typed language.

Do you (formalsystem or anyone else) know whether the Julia implementation makes much use of typing? I don't know Julia but from a few seconds googling it sounds like it's also not statically typed. I do wonder whether something like Haskell wouldn't make most sense for reimplementing scmutils -- you'd be able to make a lot of the pedagogical issues clear at compile time, in the type language alone (disregarding the actual implementation of the functions in the term language).

I see the article for a split second and then get an Error 500 page. This happening to anyone else?

pleasantly surprised to see julia used in this