A long long time ago I tried every gradient optimization method available at the time on fitting a model to noisy / non-smooth data with many local minima..... and found all of them lacking in robustness.

In the end the only truly robust algorithm I could rely on was a combination of Nelder and Meads downhill simplex method with simulated annealing. I'm curious to see in the overview he mentions neither....

Have they been supplanted by something better? Or is he only considering "nice" functions (ie. "Smooth" even for higher derivatives with no local minima)?