Hacker News Reader

Comment by dingnuts | Hacker News Reader

This is a demonstrative example of why an LLM is NOT intelligent at all, and we need to stop calling these models AI.

A real intelligence would be able to either put together the information needed and fill in gaps with research or it would be able to tell you that it does not know.

"Hallucination" is yet another anthropomorphism of the model. A mind hallucinates; a model just produces incorrect results.

LLMs are very cool but I'm really sick of the "intelligence" branding. If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent.

View on HN

I think this train has passed, AI has been accepted as the name and has now changed ownership to the public. This is how live languages work and we older people are supposed to be crying "But this is not what it means!"

Well, now it does I'm affraid :-)

I know we have had this conversation with our daughter several times and we have three languages to work with...

Your correct. Its ML/LLM at heart but now accepted as to the word "cloud".

Where all the "cloud" is, is an server in a datacentre with a fancy gui.

I guess we've needed a new buzz world to keep the population excited.

Some younger guy was trying to explain that I used the word cluster wrong when talking about clustering pins on a map. Cluster was a collection of servers in a server room. He had never heard about k-means :-)

I like to think of them as Artificial Idiots.

Exactly. Humans are intelligent, and we never misremember anything, and we definitely never are sure we remember a thing correctly when, in fact, we don't.

And I agree with you, I know people who make shit up about something they don't know about, and it goes "actual humans > LLMs > these people".

> If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent.

Then you have a highly flawed understanding of humanity... to the point you should check out of tech for awhile and go read about psychology instead.

Extremely intelligent people confidently make shit up when they don't know the answer all the time.

I’ve just tries GPT (again, after months) on high level pseudomathematics and it blabbers on and on about nonsense…

I dont know about the newer models but you used to be able to get authoritative sounding answers asking about "the fallacy of affirming the antecedent". So you don't need to be so high level.

Well, that is really good to know. I was just reassuring myself about its nonsense generation.

Terrance Tao, who is a pretty good mathematician, made a post [1] not too long ago about how he found ChatGPT to be at least slightly useful for solving a non-trivial math problem. His basic insight is to not ask for an answer directly, but to ask auxiliary questions and for suggestions of how to approach the problem, which eventually lead him down the right path. Basically how you would discuss the problem with a colleague how didn't know the correct answer of the top of their head either.

[1] https://mathstodon.xyz/@tao/110601051375142142

Frankly this sounds very close to rubber-ducking. I'm sure LLMs throw out some creative seeds, but they're much better at coming up with creative ideas than truth. I think people are too used to getting creative ideas from other people, where the ideas are at least superficially useful, to understand that they're dealing with the world's best improvising rubber duck.

I don't have a clear view on whether LLMs are intelligent or not, but I don't understand your argument at all. Why couldn't an intelligent agent exist that refuses to admit they don't know something, and just make stuff up? I've known people who come pretty close to this behavior.

> If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent.

It seems to be a good recipe for success in many fields (business, politics, art, apparently somewhat in social sciences if you make up the evidence for it too) so I'm not too sure about that!

The poster said “intelligent” not “successful”. I can name several “successful” politicians I would never, ever call intelligent.

I think you want "intelligent" to mean "ethical". Those politicians are definitely intelligent, they've devised a successful strategy to gain power. You could argue a successful sprinter doesn't need high intelligence, but politics is all about it.

No, seriously, one of our PMs was clearly as dumb as a metaphor for something really dumb. Rhodes scholar too, which shows you what that's worth these days.

What about "Fake it till you make it" then?

Reply on news.ycombinator.com