"Years of works of the genetic algorithms community came to the conclusion that if you can compute a gradient then you should use it in a way or another.

If you go for toy experiments you can brute force the optimization. Is it efficient, hell no."