I have had the same thoughts too. What also had me a bit puzzled in the ML community is the lack of discussion on local optimality, and constraint/Lagrangian loss functions. But there is probably a subtlety that I'm missing.