It's not planning ahead, it is looking at the probabilities of the tokens altogether rather than one by one.