It's with the utmost humility that I confess to falling back on "just use Nelder-Mead" in *scipy.optimize* when something is ill behaved. I consider it to be a sign that I'm doing something wrong, but I certainly respect its use.

Nelder–Mead has often not worked well for me in moderate to high dimensions. I'd recommend trying Powell's method if you want to quickly converge to a local optimum. If you're using scipy's wrappers, it's easy to swap between the two:

https://docs.scipy.org/doc/scipy/reference/optimize.html#loc...

For nastier optimization problems there are lots of other options, including evolutionary algorithms and Bayesian optimization:

https://facebookresearch.github.io/nevergrad/

https://github.com/facebook/Ax

SAMBO does a good job of finding the global optimum in a black-box manner even compared to Nelder-Mead, according to its own benchmark ...

https://sambo-optimization.github.io

ChatGPT also advised me to use NM a couple of times, which was neat.

Look into zeroth-order optimizers and CMA-ES.