At the same time, it ... is literally a next token predictor. Like that's what it is. The input is a sequence of tokens. The output is a probability distribution of next tokens.
With an LLM, the tokens are the valuable part. That's what I want from it. That's why it exists. The tokens are the point, and it produces those tokens one by one for me.
But there is no trillion dollar industry around cheap top Markov models. So there must be something about LLM tokens that makes them more valuable than those generated from a simple Markov chain. And that substance, that makes one valuable and the other not, is exactly what reduction to "next-token predictors" masks.
There's nothing more valuable about them. Their advantage is they alone are affordable.
But when we talk about humans, we're not talking about the chemicals involved in those humans.
When we talk about LLMs, the tokens are the valuable thing they produce for us. We want LLMs because they give us sequences of tokens.
To be clear, all life is a next state of the local world predictor. What makes humans somewhat unique among life is we're much better at predicting states of the world neither we nor any of our ancestors have ever experienced, for various reasons such as having the ability to legibly communicate very complicated information strings to each other, being able to build and use tools to record states of the world we can't directly sense.
Similarly, what makes LLMs and multimodal versions of the same architectures "better" than previous generations of electronic predictive models is factors like being able to read and understand roughly the same corpus of data humans have been recording all these millennia, being able to read and remember much more of it than any individual human, and being better at generalizing than other electronic predictive models, but not better than humans. And, of course, they can produce far more predictions in far less time. Frankly, that is probably the key advantage that makes the Hacker News crowd love them so much. They're not any better at predicting byte strings that can be compiled or interpreted into executable code than humans are if you gave both infinite time to do it, but they're a lot faster.
1. Useful work that has been done (the previously generated token sequence :: the mechanical work already accomplished)
2. The role of structure in relation to the application (post-training :: other components like crankshaft etc)
The output of the LLM is literally a probability distribution of what the most likely next token is.
It kind of does, though. In a gasoline engine you need to spark the combustion in advance of the piston reaching top dead-center to ignite the fuel early enough that it is able to provide downward pressure on the piston as it rolls over top dead-center. The amount of advance required changes with RPM, fuel octane, etc.
Start of delivery timing in a diesel is similar. You have to do it sufficiently far in advance to account for compressibility of the injection lines, fuel burn rate, etc as a function of RPM. A mechanical governor on an injection pump has a timing advance device built in. Electronically governed injection pumps, or modern common rail systems, do that in software.
So mechanically, engines kind of "predict" the next combustion event. Even moreso when you consider a modern ECU, which may be working at nanosecond resolution to time multiple injection events per cycle. To do this at such a resolution it will have to send signals to components based on a predictive model derived from "past" sensor data. E.g. it needs to act ahead of time to account for electrical and mechanical delays in the system.
Both examples involve the same "aha" moment: even though it's true that you are literally 'just' doing XYZ, unbelievably complex patterns and sub-goals can emerge.
I don't think you intended this, but the word choice here gave me a chortle.