Hacker News Reader

Comment by dkjaudyeqooe | Hacker News Reader

As much as people want to believe, LLMs are not intelligent by any stretch. They are very limited in what they can do and they mostly are regurgitating human intelligence supplied in text form.

If you ask an LLM about something that isn't very written about, it'll make up nonsense, substituting things that are nearby (in some sense).

We're not even touching on the question of what intelligence is. There is no concrete definition so there is no concrete way of deciding if something is intelligent. IQ tests are not it.

View on HN

IQ tests are the single most empirically validated and meaningful proxy we have for intelligence.

Remember that it is an even stronger predictor of lifetime income and life expectancy than your assigned sex at birth, your gender identity, your race, your personality, or even the socioeconomic status of your parents.

It is also a powerful predictor of your educational trajectory and outcomes, your job performance, and so much more.

Hell, it's the single best predictor we have for your lifetime chances of being involved in a car accident. That's more inclusive than just causing one - having a high IQ literally means you're less likely to get hit in a car accident.

This actually covers death from all sources. If you divide IQ into 9 "buckets", those among the lowest bucket of IQ have a threefold increase in all-cause chance of death relative to those among the highest bucket.

IQ is also not many things. It's not a perfect proxy for intelligence. It's not very changeable (largely genetic). It's not possible to test for IQ in a way that cannot be studied/gamed. It's not fair. It's not equitably distributed.

But it's not nothing. Whether or not you like it, recognize it as legitimate, love it, or hate it, the fact stands that it's the single strongest predictor of broad life outcomes humanity has ever discovered. To ignore that simply because the implications make you deeply uncomfortable is no different from the renaissance church adamantly refusing to accept that the earth is not at the center of the universe.

Burning Galileo at the stake didn't make heliocentrism false or irrelevant, it made the church look willfully ignorant and intolerant of reconciling their reality-divorced views with what the natural world was screaming at us.

>LLMs are not intelligent by any stretch.

>There is no concrete definition so there is no concrete way of deciding if something is intelligent.

You say there is no concrete way of deciding if something is intelligent, yet you yourself have decided that LLMs are not intelligent.

There's no concrete way to prove my kid's imaginary friend isn't real, but that doesn't mean both sides have an equivalent likelihood and burden of proof.

If that were the only known fact about said imaginary friend on either side, then it would mean exactly that. The reason that there are different expectations of veracity are exactly because there are a bunch of priors being held about imaginary friends, by definition, not being real.

The output of the LLMs would suggest a different metaphor. The supposed divine inspiration of the Bible, perhaps?

Compare "Intelligent Design" vs. the use of genetic algorithms in AI. Simple forms of intelligence can get you a long way and can seem very impressive, especially if they have a lot of subjective experiences, which DNA gets from deep time and which AI gets from transistors outpacing synapses by the ratio to which a pack of wolves outpace continental drift.

But why are his toys floating around in the air and moving on their own? Why is there a guitar playing itself? Burden of proof just shifted.

You see the strings and wires and notice the bad ventriloquism.

We may be unable to provide a concrete definition of what intelligence is, but we can certainly provide definitions for what it isn't. E.g. we don't need a concrete definition of intelligence to say that a rock isn't intelligent. A pencil isn't intelligent. A calculator isn't intelligence

I don't disagree for the items you listed, but for something that exhibits what in many aspects can AT LEAST be mistaken for 'intelligence' (among signs of the complete opposite, of course), I would just say that there is no way for anyone to know.

I'd argue that a pencil has some intelligence.

You seem to have a misunderstanding of what the condition "concrete" means.

He's not saying there's no way of judging intelligence, he's just pointing out there's no universal agreement on what intelligence even is.

Edit: To add, this discussion becomes pure semantics. On one side is a strict definition of AGI, on the other side are the most generalized definitions of artificial intelligence. It gets kind of silly because technically, every "if" statement is a type of "AI" by the loosest definitions.

>he's just pointing out there's no universal agreement on what intelligence even is.

Which is why I find it strange that he takes it upon himself to proclaim in a definitive manner that LLMs are not intelligent, and not "by any stretch."

Why would you find that strange? He's using his own definition of intelligence while acknowledging there's no universal agreement.

Expressing your own point of view while acknowledging that other points of view exist shouldn't be strange. Strange is not being able to see things from different perspectives, those people are abnormal even when they happen to be in the majority.

The only valid criticism I see is that "by any stretch" is hyperbolic, but that's easily forgivable.

Except that he doesn't acknowledge other points of view:

>As much as people want to believe, LLMs are not intelligent by any stretch.

Acknowledging other points of view would have sounded more like "People are free to believe what they want, but LLMs don't strike me as intelligent."

>Except that he doesn't acknowledge other points of view

False, he explicitly acknowledges other points of view on what defines intelligence:

"There is no concrete definition so there is no concrete way of deciding if something is intelligent."

Acknowledging that your way of thinking isn't the only way of thinking doesn't make someone a hypocrite, it's actually a sign of intelligence (in my opinion).

> As much as people want to believe, LLMs are not intelligent by any stretch

They said "As much as people want to believe" which means that it shouldn't be counted as intelligent by other people's definition. Even by most liberal interpretation, the comment(which is top rated) doesn't say what you are trying to imply

I'm not trying to imply something, you're ignoring what's explicitly stated:

"There is no concrete definition so there is no concrete way of deciding if something is intelligent."

The fact that it contradicts previous statements makes me believe there's some hyperbole going on.

It is actually quite well defined: https://en.wikipedia.org/wiki/Fluid_and_crystallized_intelli...

LLMs are lacking in fluid intelligence and there is even a good benchmark for it called the abstract reasoning corpus.

I have a pet rock. By your standard I can't say it's not intelligent.

Sure you can. Just don't shut down counter-arguments from, say, believers in certain form of animism, by implying that their side is invalid because there is no universally accepted definition of intelligence without giving your own assertion that same treatment.

> If you ask an LLM about something that isn't very written about, it'll make up nonsense, substituting things that are nearby (in some sense).

So what a human might do if they wanted to sound smart but had no idea about the something.

This is the most disturbing thing about LLMs, imo. Not that "LLMs aren't intelligent, they just say something plausible and if they don't know what they're talking about they'll fluently bs about it" but that "when forced to respond to a question out of their depth humans sound exactly like LLMs... oh no humans aren't intelligent" and in fact we really don't have a solid definition of intelligence beyond "yeah but did it work though?"

The only way to distinguish genius and madness, the old saying goes, is in the results.

When ChatGPT was new, some people did describe it as "Mansplaining as a service". much as I like AI, I can see why.

It’s great if you understand what it is doing, it is really just search 2.0, but it’s a huge leap from search 1.0. It can’t do original research. But most humans aren’t doing original research either.

I'm old enough to remember actual (web) search 1.0, e.g. AltaVista.

LLMs today are a bit like Wikipedia circa 2004: simultaneously fantastic and yet also flawed and used for disinformation campaigns… and surprisingly bad at advanced mathematics.

I remember visiting DEC/Compaq WRL (which originated Alta vista) as a new grad student, and seeing my internship mentor at DEC SRC quit after my internship and go to a new search engine startup in Palo Alto that wasn’t Alta visa (it worked out really well for him).

But ya, wiki was magic when it came out, and it got better over time, but still not perfect. This is similar, it is going to have an impact even if it has lots of flaws.

That's a human being actually quite intelligent. The LLM on the other hand has no higher order intelligence and the ability to create nonsense with intent, knowing it's nonsense, it merely does what it's programmed to do, ie produce nonsense sometimes.

But more generally, listing any human behavior and its similarity doesn't prove LLM intelligence.

It doesn’t. But it might be impossible for us to tell in general that a human wrote something rather than an LLM. The Turing test is toast.

The good thing is, we don't really have to argue about it. We can just wait for GPT-5 and see what happens.

Latest models can solve some relatively challenging math problems, such as those in the MATH dataset. Assuming not all of the capability stems from memorization, that exhibits a certain level of fluid intelligence.

Although recent research shows much of LLM’s reasoning capability are indeed based on memorization, some models can actually reason a bit.

https://arxiv.org/pdf/2402.19450.pdf

https://twitter.com/_saurabh/status/1763626711407816930

The way I see it, LLMs have taken one mode of thinking, which is educated guessing, and ramped it up to superhuman levels. Same way Deep Blue took rote memorization and ramped it up to superhuman levels. I think the true AI would come from digitizing all the different modes of thinking, then somehow bringing them all together into one superintelligence

This discussion of if LLMs are truly intelligent or not (they are not) seems like the "What is really DevOps" or "it's not serverless - it's just other people's servers" or "it's not web 2.0 - it's just ajax" discussions. In the end to me it misses the main point - is it useful to you or not? Can you rely on it for a feature or not? Is hallucination going to be a problem or is a bit of fake creativity actually useful for what your users need?

If it's truly intelligent or not to me seems like a "café conversation", it's entertaining as basic chit-chat to spend time but I don't see the point and people seem to make it more deep than what it needs to be.

And what do most humans do when asked about things they haven't read about? Hallucinating ChatGPT sounds very much like when the teacher calls on someone who didn't do the reading to me.

Right. And the industry has slowly realized this. The "AGI" hype has died down since reaching a fever pitch when GPT-4 was released. Result after result has shown that LLMs have extreme difficulty generalizing beyond what they've been trained on. Which really isn't much of a surprise, if you think about how deep learning works.

Of course, the major labs and AI influencers are still publicly shouting from the rooftops, because that is what sells. But it's getting harder to deny the obvious.

Ultimately, the problem is that there is really no consequence for being wrong, but tremendous upside for this kind of magical thinking. Someone should start tracking predictions so we can ignore those who continually get things wrong.

Don't feed the LLM?

Maybe we'll start seeing programming languages which licenses exempt being trained upon.

In other words; at this point in time, AI is more religion than science.

Reply on news.ycombinator.com