Because language models don’t think before doing, they think by doing.

Maybe a more idealized training set could improve things, but at least for today’s SOTA, you have to get the shitty first draft out and then improve it.

Harnessing makes a difference, but it’s only shuffling around when and where the tokens get generated. It can trade being slower by doing a hidden first draft and only showing the output after doing a self review. But the models still need to generate it all explicitly.

Why would it? It doesn't do anything with intention without being prompted. When you ask it to do something it's going to give you what seems like the most likely result, it isn't striving to give you the most correct result, those things just have some overlap.

I assume it would involve wasting a lot more tokens reasoning about this. It is known that GPT uses less tokens than Claude, but Claude uses them to reason about problems more, which is part of its "secret sauce" and why so many swear by Claude Code.