>How about axioms of logic as legal moves, and asking it to go from a set of axioms to open mathematical problems?

>Or chemical procedures and components as moves, and asking it to tackle a disease with a known molecular structure?

>It is not as straightforward as I make it sound, but these are complete information problems.

However they are not adversarial games. Self-play only works for adversarial games. Also each mathematical problem is different and needs to be solved only once, and likewise for molecular structures. Additionally, they only need to be solved once, so there is no "gradient of skill" that we know to climb.