The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on.
Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful assistant’ or ‘sets of changes to a codebase’.
And that isn’t really ‘prediction’, so much as ‘generation’.
It’s not been tuned to ‘guess the next token right’. It’s been tuned to generate the token that leads to it ultimately scoring highest on its reward function.
It’s not predicting the token, it’s predicting the reward.
If you can’t grasp that logic gap then there’s no point discussing further.
No logic required, you can just build an LLM yourself, including post training. You'll see that predicting the next token isn't something the model does or is optimized for in RLHF or RLVR. You can hand wave all you like, but you have never done it.
Carry on good soldier.
That you tie yourself up in knots of fancy acronyms instead of plain words and that your argument boils down to semantics of the word prediction, it's pretty clear what is up brother.
Boy, oh boy, do you keep digging your logic hole that much deeper.
As mentioned earlier, Sam thanks you for your obfuscation efforts while his equity keeps going up. The swindle continues.
If you want to understand how this stuff works, there are totally decent books about building them from scratch. It's not that hard, and you'll likely find it interesting. Sebastian Raschka and Nathan Lambert have good books out, and the Allen Institute has available all the code and data they have used for several projects.
Keep digging that hole, I’m sure you’ll surface somewhere with some sunshine.
1. At inference time, LLMs emit one token at a time given the prior tokens. This looks like prediction and I concede that.
2. During pre-training, LLMs predict the next token and compare to the actual next token in the training data. This is the classic setting for ML predictions. And I think its meaningful, the model really is predicting what the ground truth next token will be in the data.
3. During post-training, in the case of RLVR, there is no ground truth next token. In pretraining, the question is "what token actually came next?". In RLVR, the question is "what sequence of actions gets a high reward?"
And the whole point is that thinking about the RLVR is important. A mental model that stops at 1 or 2 is incomplete and doesn't capture what drives LLM tokens.
And it's not just RLVR. RLHF has been going on for years and years. LLMs have not been "next token predictors" for probably 5-6 years.
I’m still not sure what doesn’t qualify any of that as a prediction, and I’ll be more blunt: a guess.
But that’s the point: so is an LLM. Putting one token in front of another, hoping it’s doing the right thing to bring about the rewards it’s trained to… trying its best.
So yeah, not ‘next token predictors’. ‘Next token tryers’ maybe.
For example ask it for a recipe for rock pizza or glue pizza or whatever and if it had not been specifically trained on it or had guardrails introduced, but has some nonsense in its dataset, it will reproduce the nonsense.
It's both. Generation in a predictive way. That's what G and P in GPT stand for.