Comment by orbital-decay | Hacker News Reader

Comment by orbital-decay | Hacker News Reader

Parent Full thread

>LLMs <snip> do not meaningfully share any of the mechanisms that animals or people use to reason or think.

This seems to be a hard assumption the entire post, and many other similar ones, rely upon. But how do you know how people think or reason? How do you know human intelligence is not an illusion? Decades of research were unable to answer this. Now when LLMs are everywhere, suddenly everybody is an expert in human thinking with extremely strong opinions. To my vague intuition (based on understanding of how LLMs work) it's absolutely obvious they do share at least some fundamental mechanisms, regardless of vast low-level architecture/training differences. The entire discussion on whether it's real intelligence or not is based on ill-defined terms like "intelligence", so we can keep going in circles with it.

By the way, OpenAI does nothing of this, see [1]:

>artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work

Neither do others. So the author describes "tech industry" unknown to me.

[1] https://openai.com/charter/

> Decades of research were unable to answer this.

From the article:

> The field of AI research has a reputation for disregarding the value of other fields [...] It’s likely that, being unaware of much of the research in psychology on cognitive biases or how a psychic’s con works, they stumbled into a mechanism and made chatbots that fooled many of the chatbot makers themselves.

Just because cognitive scientists don't know everything about how intelligence works (or on what it is) doesn't mean that they know nothing. There has been a lot of progress in cognitive science, in the last decade in particular on reasoning.

> based on ill-defined terms like "intelligence".

The whole discussion is about "artificial intelligence". Arguably AI researchers ought to have a fairly well defined stance of what "intelligence" means and can't use a trick like "nobody knows what intelligence is" to escape criticism.

>Just because cognitive scientists don't know everything about how intelligence works (or on what it is) doesn't mean that they know nothing.

Author's claim is pretty strong: that human intelligence and what is called GenAI have no common ground at all. This is untrue at least intuitively for the entire field of ML. "Intuitively" because it cannot be proven or disproven until you know exactly what is human intelligence, or whatever the author means by thinking or reasoning.

>Arguably AI researchers ought to have a fairly well defined stance of what "intelligence" means and can't use a trick like "nobody knows what intelligence is" to escape criticism.

If you don't formalize your definition, the discussion can be easily escaped by either side by simply moving along the abstraction tree. The tiresome stochastic parrot/token predictor argument is about trees while the human intelligence is discussed in terms of a forest. And if you do formalize it, it's possible to discover that human intelligence is not what it seems either. I'm not even starting on the difference between individual intelligence, collective intelligence, and biological evolution, it's not easy to define where one ends and another begins.

AI researchers mainly focus on usefulness (see the definition above). The proof is in the pudding. Philosophical discussions are fine to have, but pretty meaningless at best and designed to support a narrative at worst.

>Author's claim is pretty strong: that human intelligence and what is called GenAI have no common ground at all.

I feel that you're being unfair to the author here; the quote you responded to in your GP post alluded to "reason or think", and their argument is that LLMs don't. This is more specific than the sweeping statement you attribute to them.

> AI researchers mainly focus on usefulness (see the definition above).

And usefulness is something the article doesn't touch on, I think? The point of the article is that some users attribute capabilities to LLMs that can be explained with the same mechanisms as the capabilities they attribute to psychics (which are well understood to be nonexistent).

> Philosophical discussions are fine to have, but pretty meaningless at best and designed to support a narrative at worst.

Why this is relevant to AI research is (in my interpretation) that it is known to be hard for humans collectively to evaluate how intelligent an entity really is. We are easily fooled into seeing "intelligence" where it is not. This is something that cognitive scientists have spent a lot of time thinking about, and may perhaps be able to comment on.

For what it's worth I've seen cognitive scientists use AI to do cools stuff. I remember seeing someone who was using AI to show that it is possible to build inference without language (I am speaking from memory here, it was a while ago and I lost the reference sadly so I hope I'm not deforming it too much). She was certainly not claiming that her experiments showed how intelligence worked, but only that they pointed to the fact that language does not have to be a prerequisite for building inferences. Interesting stuff though without the sensationalism that sometimes accompanies the advocacy of LLMs.

> The field of AI research has a reputation for disregarding the value of other fields [...] It’s likely that, being unaware of much of the research in psychology on cognitive biases or how a psychic’s con works

Is the reputation warranted? Just a US thing? Or maybe the question is "since when did this change"? because in the mid 2000s in france at least, llm research was led by cognitive psychology professors who dabble in programming or had partnerships with a nearby technical university.

I am not based in the US, nor in France, so I can't say what the situation is there. I was only pointing out that quote to GP.

My experience isn't much, since I am neither doing AI nor cognitive science but I have seen cognitive scientists do cool stuff with AI as a means to study cognitive science, and I have seen CS researchers involving themselves into the world of AI with varying success.

I would not be as emphatic as the author of the article but I would say that a good portion of AI research lost the focus on what is intelligence and instead just aimed at getting computers to perform various tasks. Which is completely fine until these researchers start claiming that they have produced intelligent machines (a manifestation of the Dunning Kruger effect).

As far as I know the best definition of intelligence is "ability to solve problems".

The best we've got is "produce some speech that may contain the answer, BUT craft the question in a way that is more likely to generate an answer".

best definition according to whom?

There is a lot of work to define intelligence, both in the context of cognitive science and in the context of AI [0].

I haven't spent enough time looking for a good review article, but for example, this article [1] says this: "Intelligence in the strict sense is the ability to know with conscience. Knowing with conscience implies awareness." and contrasts it with this: "Intelligence, in a broad sense, is the ability to process information. This can be applied to plants, machines, cells, etc. It does not imply knowledge." If you are interested in the topic the whole article is interesting and worth reading.

[0] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&as_ylo...

[1] https://www.cell.com/heliyon/fulltext/S2405-8440(21)00373-X

> best definition according to whom?

It's what makes most sense to me.

> Intelligence in the strict sense is the ability to know with conscience. Knowing with conscience implies awareness.

This one I would disagree with.

> Intelligence, in a broad sense, is the ability to process information. This can be applied to plants, machines, cells, etc. It does not imply knowledge.

This one starts with one part of what I consider intelligence, but besides processing it also needs to be able to use the information to solve problems. Because you could process information, which everything in the World actually does technically, but you would not be using that information to do anything.

So ultimately, maybe it would be best to define it as it's the ability to take in and use information to solve some sort of problem.

Reply on news.ycombinator.com