With LLMs, having questions is more important than having answers. When you run out of questions, the exploration stops.
This causes the LLM to orbit around some point in the latent space, but doesn't cause it to explore. You have to tell it to pick a direction and there is no better direction than a question.