Hacker News Reader

Comment by samtho | Hacker News Reader

This is only a common mistake to the subset of people who labor under the delusion that we have actually stumbled upon Generalized Artificial Intelligence.

We know how LLMs work fundamentally and what their limits are. LLMs are only able to make “correct sounding” statements which have the side effect of being correct a certain percentage of the time. They do not have the ability to reason nor engage in high level thought.

View on HN

How can you say they don't reason if reasoning is required to produce high rates of success that previously have only been successfully answered using reasoning? Its not like they are guessing randomly but just cooindidentally having high rates of accuracy. They clearly have an emergent property of reasoning. Perhaps reasoning is too ill-defined of a word. They are deducing or calculating the answer from their being. What we call the thing that occurs when they produce an insightful answer is less telling than the fact that they produce answers previously only possible by thinking and reasoning humans.

> We know how LLMs work fundamentally

We know how they work only at the lowest level (the arithmetic operations) and the highest level (the optimization criterion and the representation of various layers, like the input/output layer and for things we can easily probe like embedding matrices).

We do not know "what they are doing" on the inner layers. This is an area of active research.

> They do not have the ability to reason nor engage in high level thought.

You are speculating (and probably incorrectly), or you're really holding back some valuable research from the field of AI interpretability.

You talk about about as if a human-created neural network is at the same level as quantum physics where there are limits as to our understanding. We know very well how large language models work even if the capabilities of this technology are actively being explored.

You along with others here are far overstating the unknowns we have within the context of AI, whether this is the result of a misinformation campaign targeted at trying to boost the value of this tech or if the pop-sci takes have really gotten too prevalent, it is unclear to me.

For the definition of "understand" that most people use, humans don't understand things which are highly complex. We don't really understand the weather, we can't predict it, it's too complex. But, you can break it down into matter and forces and energy and simulate it and get pretty darn good predictions. We can now throw it in a deep learning model and get good predictions. But, to suggest we "understand it" doesn't gel with most people's definition of "understand".

wyager is correct. There are very serious limits to our understanding of what's going on inside these networks. Even how an LLM answers simple factual questions like "The capital of France is ..." is only now just coming into view. And the moment it gets more complex than that, interpretability is lost again.

> This is only a common mistake to the subset of people who labor under the delusion that we have actually stumbled upon Generalized Artificial Intelligence

That is technically true but deceptive: that subset is enormous! Skim any HN thread and you'll see many people talking about reasoning ability or how we just need a little more magic sauce and the "hallucination problem" will be solved. And a lot of what people say in this vein is not even wrong.

And that's just HN. Non-technical users are of course going to assume that if it looks like a duck and is dressed up as a duck by its creators, then it's a duck. Why wouldn't they? So I would claim that the subset is a majority.

Reply on news.ycombinator.com