This is awesome. I was working on something similar (but got distracted) except my strategy was evolutionary. The idea was that the LLM had to figure out what it’s optimizing for itself.

I find that as soon as an LLM has an explicit definition of what it’s doing, the exploration/exploitation ratio swerves into exploitation and it gets stuck

This is very nicely done and much more clear to follow than the paper. One question: Does LLM get to evaluate the code using tool or guess the return value of f(x)?