Some things are easier to verify than to generate.

LLMs can often make a distribution of generated outputs that is much closer to a good answer than other methods.