Another derivation-free optimization algorithm is the Cross-Entropy Method [1].

The main idea is to use a random generator to sample 100 points for example. Then take the 20 points with maximal evaluations. Finally adapt your random generator to maximize the chances of sampling those 20 points, and start again.

When used for continuous functions, you can use Gaussian random generator, and this becomes similar to the CMA-ES algorithm [2].

[1]: https://en.wikipedia.org/wiki/Cross-entropy_method [2]: https://en.wikipedia.org/wiki/CMA-ES