Hacker News Reader

Comment by ttul | Hacker News Reader

I'd respectfully disagree with this characterization of LLMs. While they certainly excel at pattern recognition, calling them mere "autocomplete machines" vastly undersells their capabilities. LLMs demonstrate complex reasoning, multi-modal understanding, and emergent behaviors that go well beyond simple pattern continuation.

They've succeeded in areas like mathematical problem-solving, creative tasks, and various real-world applications outside of GitHub Copilot. Sure, LLMs have limitations, but reducing them to autocomplete overlooks their nuanced language understanding and generation abilities. I also think the "lossy translation" argument doesn't give enough credit to the sophisticated ways problems can be reformulated for LLMs.

View on HN

I don't understand how anybody can still claim LLMs show "complex reasoning".

It's been shown time and time again that they'll produce a correct chain of reasoning when given a problem (e.g. wolf, goat, cabbage crossing a river; 3 guards and a door; etc.) that is roughly similar to what's in the training data but will fail when given a sufficiently novel modification _while still producing output that is confidently incorrect_.

My own recent experience was asking ChatGPT 3.5 to encode an x86 instruction into binary. It produced the correct result and a page of reasoning which was mostly correct, except 2 errors which if made by a human would be described as canceling each other out.

But GPT didn't make 2 errors, that's anthropomorphizing it. A human would start from the input and use other information plus logical steps to produce the output. An LLM produces a stream of text that is statistically similar to what a human would produce. In this particular case, it's statistics just weren't able to cover the middle of the text well enough but happened to cover the end. There was no "complex reasoning" linking the statements of the text to each other through logical inferences, there was simply text that is statistically likely to be arranged in that way.

Perhaps it’s because I know human beings that have the exact same operation and failure mode as the LLM here and I’m probably not the only one. Failing at something you’ve never seen and faking through it is a very human endeavor.

Regarding errors: I don't know the exact mechanism in the brain that causes humans to make them but i believe it's a combination of imperfect memory, attention span and general lack of determinism. None of these affect logical reasoning as performed by a machine.

Regarding faking it till making it: This is a more general point that there's a difference between simulating human behavior and logical reasoning.

If you teach your kids to read, they behave almost exactly like an LLM in so many cases it's eery.

>there was simply text that is statistically likely to be arranged in that way

This is wrong. An LLM can produce text that has never been arranged that way in training data.

Your statement doesn't contradict their statement. It produces text that is statistically likely to be arranged that way, but because we use non-deterministic sampling we get a wide variety of results that weren't necessarily in the training data.

No, I don't think they are talking about the decoding process there. If they were, it's a non statement. They are almost certainly talking about training data.

I agree that LLMs don't really reason, but I'm starting to think that the name "Large Language Model" is just wrong enough to create this confusion. What they seem to be doing is modeling human reasoning as encoded in language, and then enable some mixing and matching of elements of that encoded reasoning.

This somewhat means that these models are trapped within the universe of "human capable reasoning" with some possibility of escaping it through the stochastic generative processes they're built on. But they simply can't think through novel problems and arrive at new conclusions.

Furthermore, they're limited by the fact they're built on human knowledge as encoded in text, which terribly imprecise and fluid. Any hope to have a path to AGI, where reasoning might actually happen, will have to have something far more rigorous for the internal reasoning, with language just being a clever interface rather than the mechanism by which the thinking is done in.

They're really "Large Analogy Engines" or maybe "Large Captured Reasoning Engines" and are an incredible technology.

GPT-3.5 is not the same thing. When people talk about LLMs having capabilities like complex reasoning, they're talking about current-gen models (eg Claude 3.5 Sonnet, GPT-4, Llama-405B).

Either this whole comment is from a GPT or the author is a major contributor to the GPT training dataset.

Anyway, LLMs are nothing more than very expansive error reduction algorithms. They don't reason. They store and manipulate meaning in an automated way, but they don't reason.

I wouldn't say they're mere autocomplete machines, but their tendency to go into a loop to complete a pattern does break the magic, since it's a rather machine-like thing to do.

> LLMs demonstrate complex reasoning, multi-modal understanding, and emergent behaviors that go well beyond simple pattern continuation.

Can they win Wordle yet?

I never said that we couldn't use them for other things—it turns out that a lot of problems can be translated to autocomplete! And yes, in many cases the loss can be made minimal by being clever with the translation.

What I'm saying is that nearly every failure of LLM applications boils down to the engineers not understanding that they're just autocomplete engines. Most new LLM products are put together by someone who claims to be an "AI Engineer" but who approaches an LLM with magical thinking and only a vague understanding of the way they work internally.

They're autocomplete machines. That's not a bad thing or an argument that they're incapable of solving complex problems, it's just the reality of what they're doing, and a good AI engineer understands that and works with it.

In that vein, posting comments to hackers news is just an auto-complete, or “what should the next token be” problem, right? :-)

Yet here we all are, reading the comments, looking for insights, knowledge, and entertainment.

Reply on news.ycombinator.com