That chess analogy deeply confused me. Chess engines don’t compute win probabilities and choose the highest move.

I don’t think a chess engine is an apt analogy at all. In a chess engine, there is a concrete search tree and although it emits one move at a time, it’s actually picking the entire branch (of course, with iterative deepening as the game progresses).

There is no obvious place in transformer models where the entire trace was already computed prior to a single token being chosen. It’s possible, maybe even likely, that the whole trace exists internally as activations. Multi token prediction and diffusion adapters point to that being the case. But to my knowledge no explanation has been given for where in the model the future plan is stored.