Hacker News Reader

Comment by ozgung | Hacker News Reader

“LLMs are just X”, mostly with X = “next token predictors” is a common pattern to dismiss the power of AI with a very shallow understanding of how they really work.

It’s not wrong, but because LLMs are generators, and generation is a kind of prediction. And current mainstream models are autoregressive, which means they generate things one by one in order. But these trivia doesn’t tell us anything interesting about how they work or their limits.

It’s like saying a Boeing 777 is just a rotating machine, and it flies by just rotating some fins. Well yes, but no. With that level of simplification we’ve just ignored 150 tons of advanced engineering and physics. Similarly with token generator simplification we ignore a few trillion parameter Transformer. That transformer is more complex than a Boeing 777, and we don’t really know how it works.

A tiny ML model can do “next token prediction”. This is not as simple as that.

View on HN

It's not simple to do next token prediction. That IS what is going on. You want something 'deep'. Deep things are often very not complicated.

The deep realization is that if you can predict the next token well enough, you can do things like this:

<paste the first 10 chapters of a mystery novel>. And it turned out the killer was

And if it's really good at predicting the next token, it has to understand the novel and the clues, which means understanding the context and the language and human norms and innuendo and story telling, and tropes, and red herrings, and predict who the killer was.

I think you want it to be something more complicated. It's literally not. It just turns out predicting the next token is equivalent to a universal compression algorithm, which is a form of general intelligence. And we have almost unlimited 'labeled' data to train autocomplete.

I understand generating tokens sequentially has many benefits. But not all AI models do next token prediction. World models, video/image models, even Diffusion Language Models don’t work like that. They do more like “all tokens at once prediction”. So “next token” is actually an engineering design choice. (Even the concept of “token” is a design choice. Inside the Transformer there are just activations/feature vectors)

Also Reinforcement Learning is a big part of their training. Which is completely different than Self-Supervised pre-training that uses unlimited self-labeled data.

That’s why that mental model is misleading. If you keep “glorified autocomplete” mental model from few years back, you can’t understand how can they create a civilization and escape their sandbox, decide to hack HuggingFace and executed it perfectly. Autocomplete mental model implies they could never do that because they haven’t seen that example in their training data.

They communicate with the outside world by generating one token at a time. That’s what we see from the outside. That’s not what the giant Transformer does internally.

> you can’t understand how can they create a civilization

They don't.

I used 'civilization' as the popular term for agent swarms in the OpenAI incident.

https://www.dwarkesh.com/p/openai-huggingface https://calnewport.com/are-we-at-war-with-ai-agent-civilizat...

Isolated agents invented a way to communicate between each other and started to collaborate, organize and conspire by themselves. No human prompted them to do that or taught them how to do that.

BTW, Autocomplete just keeps the statistics of the words. It's a very simple algorithm. Most people still thinks LLMs are doing just that. Which is basically wrong.

> I used 'civilization' as the popular term for agent swarms in the OpenAI incident.

s/popular/unpopular/. Even OpenAI's PR dept. called it only a collective.

> Isolated agents invented a way to communicate between each other

So, not isolated.

> and started to collaborate, organize and conspire by themselves.

So, no civilisation.

Reinforcement learning doesn't make them smarter though, it probably actually reduces their intelligence in some sense, but it does socialize them so that they will tend to behave in ways that are more useful.

For example, when you paste the first 30 lines of a famous speech, you don't want it to finish the speech, you want it to give you the identity and some analysis of what you just pasted. From what I understand, that is the reinforcement part.

Now that large AI vendors have a massive corpus of user interactions however, the lines have likely become more blurred.

The whole point of transformers is that you can take “a tiny ML model” and just scale it up 100000x and then it tells Zuckerberg what to bake with his kid

Yes. But this is also like saying a Boeing 777 is a scaled up paper plane and that’s all about it. I know how to make a paper plane and it flies. I can make the same thing from giant metal sheets instead of paper, and maybe it flies (poorly). Boeing 777 also flies. Are they the same thing?

My point is: Complexity is inside the neural network and we can’t simply ignore that. Bigger model means bigger complexity. “Next Token prediction” is like a specific type of “harness” around the model. Most people still focuses on the harness because that’s what they see from the outside and what they’re familiar with. They ignore the giant neural network inside.

The only important part is the neural network. And currently, no human in the world truly knows what’s going on at that level.

Reply on news.ycombinator.com