Hacker News Reader

Comment by neilellis | Hacker News Reader

It can only predict words, is like saying all neuron's are just switches, all matter is just protons, electrons and neutrons.

It's just reductive. What we're seeing from LLMs is simply amazing considering what they are supposed to just do.

Whatever LLMs are capable of, big or small it exceeds such reductive thinking as 'it predicts words'.

View on HN

And "predicting words" given a huge corpus of interesting text by itself seems like it could be amazing. Predicting words effectively can mean that it has internalized all the logic used to generate the original texts. At some extrema simply predicting words could mean more than just "knowing facts", but actually having the accumulated smarts of those that made large contributions to the corpus.

Allow me to participate in some hyperbole here, but "predicting words" could result in a model that is "smarter" in most capacities than most humans.

It's basically creating an accurate world view

Exactly. These kinds of reductive arguments always bother me because they ignore emergent effects. Chess engines are “just” running minimax? Well guess what, they’re evaluating millions of positions per second.

If this whole intelligence thing can be reduced to “just” predicting the next word, maybe we’re not as special as we thought. In fact anyone who reads the Wikipedia article on cetacean intelligence will quickly realize that we aren’t the hot shit we once thought we were.

One expects the academic community to step in and provide this kind of analysis, namely that an LLM combined with a great deal of computing capacity exhibits capabilities greater than the sum of its parts.

I'm a long time LLM enjoyer and by far the best analysis I've seen for that is https://generative.ink/posts/simulators/ but it's way "too much" for normal people who are learning about stochastic parrots and blurry jpegs instead.

Reply on news.ycombinator.com