Comment by eevilspock | Hacker News Reader

Comment by eevilspock | Hacker News Reader

Calling it a mind or "A.I." is confusing power with intelligence.

Imagine a person taking a test. This person has the power to look at everyone else's answers, and the power to quickly choose or calculate the answer that is the most common. This person will probably score well above average on the test (though probably not as high as the top score). This person is not smart, but powerful.

A CPU that can calculate a zillion additions in a second does not understand those calculations, much less be able to come up with mathematics in the first place. It is not intelligent. It is powerful.

ChatGPT is supremely powerful in its ability to generate "answers" that are the amalgam of hundreds or thousands or millions of answers or statements made by humans, but dumber than a bag of rocks.

LLMs demonstrably model their training data, which has a correspondence to the structure in the world captured in the data[1][2]. This goes far beyond just searching a database for the closest relevant response. The question is what does that mean regarding understanding? I go into that in some detail here[3].

[1] https://arxiv.org/abs/2210.13382

[2] https://twitter.com/leopoldasch/status/1638848881558704129

[3] https://www.reddit.com/r/naturalism/comments/1236vzf/

If you give that person the ability to represent unknown questions as combinations or functions of ones for which they have the class' answer distribution, then - things get blurrier between being "powerful" and "smart".

To take us back to the age-old Chinese Room story: What does it mean, to _understand_ something? If an entity has internal processes which allow it to converse with humans consistently and reasonably about something (and I'm not saying GPT/LLMs are that) - can you not claim that entity understands it?

> Imagine a person taking a test. This person has the power to look at everyone else's answers

No, that analogy fails. ChatGPT can pass new tests that it has not seen before.

Reply on news.ycombinator.com