Hacker News Reader

> LLMs predict next words.

That's an oversimplification. Deep learning LLMs such as ChatGPT are designed to solve natural language processing (NLP) tasks, not just next-word prediction. In order to "predict next words" they create a model and "reason" about it. They are not simple Markov processes. There's a lot happening in that ~100 layer neural network.

View on HN

Technically they are designed to solve next-word prediction.

It just turns out that good next-word prediction requires understanding the world.

(And you can equally view humans as "next word predictors" anyway.)

Yes that's exactly right. NLP problem solving abilities are a consequence of the fact that in order to predict the next token it actually needs to "understand" all of the previous tokens and how they relate to the next predicted token, and determine that the new output is "logically" coherent in context.

Okay, it seems I should dig more on the topic. Thanks!

Well as the other commenter here clarified, you're right about transformer LLMs doing next word prediction, but my point is that's only the tip of the iceberg. The complexity of the model doing the prediction is incredible. Not basic stats.

Reply on news.ycombinator.com