Hacker News Reader

Comment by p-e-w | Hacker News Reader

> The fact that GPT-4 can pass his tests says much more about the tests than about the capabilities of GPT-4.

"Deep Blue beating Kasparov shows that ultimately, chess is just brute force calculation..."

"Stable Diffusion 'art' being indistinguishable from human art to most people shows how little most people know about art..."

"ChatGPT fooling people into thinking they're talking to another human shows that those people weren't paying attention, so the Turing test is actually useless for determining..."

"I mean, I know that everyone used to say that doing X means being intelligent, but now that software has done X, it becomes clear that..."

What is the real criterion for a machine to be considered intelligent? What does it have to do for people to not retroactively downplay it by invoking mechanistic pseudo-explanations?

View on HN

Err, did you miss the part where I wrote that the post is not meant to imply that ChatGPT is not impressive?!

Deep Blue beating Kasparov in chess showcased Deep Blue's capabilities; Deep Blue beating Bryan Caplan would not have showcased these capabilities, despite the capabilities being there.

The analogous hypothetical situation would be a Not-so-deep Blue that fails to beat not just Kasparov, but any random chess master with a nonzero Elo score, but manages to beat Bryan Caplan. After his defeat, Caplan would write an article claiming that he's "known to be very good at chess". This would definitely say more about Caplan's chess skill than the chess engine's capabilities. And none of this would imply that chess engines are in any way impossible or unlikely, or that the capabilities are not impressive. Indeed, a chess engine that beat Caplan would have been impressive in 1975, even though we were still very far from Deep Blue or Stockfish at that point.

You did write that, but you first wrote that GPT-4's performance says more about the task than about its capabilities.

Which is the exact same pattern that has been used to "deal with" advances in AI since forever.

And Caplan is a university professor, not a random person. He's not an amateur at giving tests. It's literally (part of) his job. So what happened here is not at all analogous to your chess examples.

I would say that the OP is entitled to share their opinion. Which I tend to agree with. Also I would question if the job of a university professor is really to give tests, or rather to teach and conduct research.

Tests are just an annoyance (that is deemed necessary). Because one passes the university tests does not mean they necessarily understand the topic very deeply, I think it's pretty obvious.

That's actually an interesting point: university students can totally pass exams by "mimicking": not understanding the topic, but solving the exam problems in a way similar to the exercises they learned beforehand. I argue that the test is not perfect at discriminating those who understand from those who mimick impressively well.

> the test is not perfect at discriminating those who understand from those who mimick impressively well.

They discriminate all the time, just in the wrong direction. There are a lot of exams which are written in a way which counts on this mimicking, but in a way which makes tests way more difficult even if you understand the topic. It’s the same problem with remote learning: it’s way easier to cheat on them. In the past 3 years, (except the first semester when COVID broke out) remote exams were way more difficult than before to mitigate this. I had tests which were clearly made this cheating in mind. It was borderline impossible without cheating, they counted on it. At that point, I have no idea why they still tried to enforce any such rules.

And still, I know many teachers who clearly don’t understand their topic completely, they just “mimic” it.

Shifting the goal post is a realistic measure on a slippery sloap.

Chess isn't that different, inasmuch as memorizing openings and end games is part of the training.

>And Caplan is a university professor, not a random person.

He wasn't compared to random people, he was compared to other professors also setting tests, and how GPT-4 fared on their tests. You're blatantly misrepresenting the point being made.

I was referring to the chess analogy from the comment I responded to, not to the comparison from the top level comment.

> The analogous hypothetical situation...

I'm not sure if that analogy holds any more given that neural nets scale with time and data. If that was happening with a neural net chess engine, the difference between beating Caplan and beating Kasparov is measured in maybe a couple of days. Google's Go AI only ever lost a game to Lee Sedol, then immediately moved a long way into superhuman territory with a bit more training.

The gap between novice human and expert human looks quite small to a computer. If it can outperform a novice, it isn't far off beating an expert.

The comments on the exam and the answers, which he claims to be correct and incorrect, make for interesting reading. https://www.thebigquestions.com/2023/04/05/gpt-4-fails-econo...

> What is the real criterion for a machine to be considered intelligent?

This is actually the core of the debate. The answer is that there is no real criterion because we do not understand intelligence. Or, put differently, the one-dimensional understanding of intelligence we typically entertain is a simplification.

This is very inconvenient so we just pretend intelligence was a well-defined concept. That in turn leads to all kinds of confusion and pointless arguments as you have pointed out.

You mean IQ tests are not meaningful? Many will be disappointed :).

this is a famous common misunderstanding, people is trying to stop it spread, but the work is difficult.

They strongly predict various other things, therefore they are meaningful.

Them being imperfect does not imply them being meaningless. That would be a low-IQ leap in logic, so to speak.

While there is not definitive criterion for intelligent there is a lot of intuition. For example, I can find a book to be very intelligently written, but at the same time the book itself is not intelligent. No matter how clever answers it has to complicated questions, the book is still just an inanimate object created by a human. The definition of intelligence has to be stretched quite far in order to bypass our intuition on this.

We see similar situations when people anthropomorphize animals, or ask questions like do plants have feelings. Biology research can look at neurons and DNA sequences and find all kind of similarities to humans, but it still requires a bit of a jump when we start applying human philosophy concepts to non-humans.

I find it easier to look at it as metaphors. ChatGPT has intelligence similar to how the planet has an intelligence , or how the internet is like a brain with every single connected device their neurons.

Is the average human intelligent? If I had to choose between an average human or GPT4 to help me with my work, I would choose GPT4.

Seems many people think of intelligence like they think of magic. Once they learn how the trick is done, it’s no longer magic.

Are you sure about that?

GPT-4 falls apart on many tasks that humans find trivial, such as planning.

Intelligence is multi-faceted and quite frankly the average technologists understanding of human cognition is quite poor. Intelligence is not just information retrieval.

I'm sure, because I specifically chose GPT-4 because of its intelligence. Otherwise it wouldn't be of use to me-- I already have a slew of tools and sources available, but it takes more than that to pick out just the right solution and present it.

It also quite good at planning its replies, that I see.

IMO we first need to clean up our language and precisely define the word intelligence so we all know what we are even talking about.

I would imagine we actually understand intelligence but in ordinary language it is a placeholder for a number of different meanings and concepts.

Same problem with "consciousness", same problem with "sentience".

I think when we say we don't understand these words it is really that they can't be understood as they are used in ordinary language because they can't objectively mean all the values people assign to these words.

In the face of AI, why would we not have to further define and expand our language for communication purposes? That actually seems obvious if you think about it for 2 seconds.

"sentience" has an additional problem that science fiction authors conflate it with "sapience".

I suspect that some influential science fiction editor made a mistake early on and it stuck. Outwith science fiction, I've less commonly come across the conflation, and people generally use the dictionary meanings, more in accord with the Latin roots, of "sentient" meaning being capable of sensation/feeling and "sapient" meaning being capable of intelligent thought.

> What is the real criterion for a machine to be considered intelligent?

There is no need to come up with a perfect criteria list for human like intelligence. And maybe it is even impossible, as we can only use the rational part of the brain to understand the whole.

But when a human like intelligent machine will be here we will all know it without a doubt. It will have the impact of the wheel, the control of fire, things like that. Nobody stood around to argue about how hot things need to get to catch fire or when exactly a wheel becomes a wheel as opposed to a rolling tree trunk when their neighbors where throwing torches on their straw roofs from war chariots.

Arguing about definitions, criterion, moving the goal post, etc, is just overthinking. You will drop to your knees with eyes full of tears and mouth wide open when you will experience true AI. That's how you will know.

I repeat Djikstra's refrain: asking whether a computer can think is like asking whether a submarine can swim.

How does that work? What’s the answer?

Submarines certainly partake in whatever aspects of swimming are responsible for traveling in or under water.

"What is the real criterion for a machine to be considered intelligent?"

For me, AGI.

Intelligence is a very complex problem, we tended to meassure it by extreme outliers, evolution did not optimize fore. Like abstract math. Machines clearly are better at everything with calculations. A criterium we reserved for "intelligence".

But "simple" things like understanding what a rock is, are still too hard for AI. They can reproduce and recombine some text snippets about rocks, but they do not understand, what a rock is. Or anything as far as I am aware. So they are clearly useful and do things, we described as intelligenz when humans are doing it. But human intelligence is still superior as it is a general intelligence.

> They can reproduce and recombine some text snippets about rocks, but they do not understand, what a rock is.

What is the difference between "recombin[ing] some text snippets about rocks", and understanding what a rock is?

I'm not asking philosophically. I'm asking how you can tell the two apart.

(BTW, transformer LLMs do not operate by recombining text snippets.)

> I am not asking philosophically.

That is a philosophical question, though...

To me, it sounds like this: "How can I prove that I 'exist'? I'm not asking philosophically, I'm just asking how to do it".

Your rock thing gets to the heart of the issue, LLM’s are a statistical mimic of what an actual intelligence might say.

An actual intelligence is just a matter of the architecture and compute power though, right?

Good luck defining AGI to any measurable degree, especially with the spectrum that it almost certainly is.

> But "simple" things like understanding what a rock is

Well, how does one verify that an entity understands what a rock is? The education system clearly has no idea how to test for understanding of literally anything. It all fundamentally boils down to testing for consciousness, which is impossible. A sufficiently advanced automaton is indistinguishable from a conscious being to any outside queries.

"Well, how does one verify that an entity understands what a rock is? "

For starters, it would mean, that it can reliable detect a rock on an image, e.g. recognizing the characteristical traits of a rock.

Currently a apple gets recognized as an iPod, when there is a label on it saying so.

"When we put a label saying “iPod” on this Granny Smith apple, the model erroneously classifies it as an iPod in the zero-shot setting,’ "

https://www.theguardian.com/technology/2021/mar/08/typograph...

So this specific flaw might have been fixed by now, but the general shortcoming is, that it has no understanding of the concept of an apple.

Or a rock. Or a human. It just "knows" certain pixels arranged in a certain way get the lable apple.

"The education system clearly has no idea how to test for understanding of literally anything."

And that is very strong hyperbole. There are many, many flaws in the education system, but they do know how to test knowledge (in person). They just do not know how to scale it cheaply, hence the stupid multi choice tests and co.

Just gonna leave these here:

https://static.independent.co.uk/s3fs-public/thumbnails/imag...

https://upload.wikimedia.org/wikipedia/en/2/21/The_dress_blu...

What do you classify it as, a rabbit or a duck? Is the dress blue and black or white and gold? :) Also you do know some people literally fall to the floor and start convulsing if you show them some flashing lights right? Every system has its "glitch tokens".

I mean is it really surprising that when you greatly simplify the principle that organic neurons work on, quantize it to the point where it's fast enough to use, have it learn no model of the world besides looking at photos and telling it what it's looking at, that it'd have weird corner cases? I really don't think so. If you want to compare fairly, you'd have to do so with a human that's never spoken to anyone, never seen anything but 2D pictures on a screen in front of them and hasn't been told or shown anything about the world they live in since they were born. And that's would be with the advantage of having six million years of base model training beforehand.

> And that is very strong hyperbole.

Well sure I guess it's mainly out of lack of care and practicality, but still even more elaborate tests like ones for say, math or physics that have you write the entire procedure don't really test if you understand the underlying principles, just that you've seen that type of problem before and can remember the equations correctly. The LLM bread and butter functionality basically.

I don't find the example right now, but you can give AIs just some pixels they would classify as dog or whatever. A human would only classify something that has the attributes of a dog as a dog. That's a difference from corner cases and unclear dress examples

"If you want to compare fairly, you'd have to do so with a human that's never spoken to anyone, never seen anything but 2D pictures on a screen in front of them and hasn't been told or shown anything about the world they live in since they were born"

And this is not what is happening with AIs. They get fed with data that sums up more than a single human lifespan.

", but still even more elaborate tests like ones for say, math or physics that have you write the entire procedure don't really test if you understand the underlying principles"

And this is not true, except for high school tests. You certainly can test for understanding, by not giving generic problems, but where you have to apply your knowlege. And sure those were the most hated tests, as they required actual understanding and you could nvot weasel through. But they also created more work for teachers, so mosts tests were just the generic have you learned the algorithm tests. And if testing in person, you can go deep.

"AGI" is such a ridiculous use of language.

AGI is just the old "strong AI" that is just the old "AI", all that we never bothered to precisely define.

I just can't understand how we can talk about language models all day without understanding the most basic idea of Wittgenstein in that sloppy use of language leads to philosophical problems and misunderstandings. chatGPT can easily teach people this too:

"Philosophical problems often arise when words are taken from one language game and used in another without an understanding of how the change in context alters their meaning. For example, words like "mind," "knowledge," "belief," and "truth" are used in everyday language games with certain meanings, but when they are transplanted into philosophical discussions without adjusting for the change in context, misunderstandings and confusions occur. "

I think the best definition for "strong AI" is an AI that is capable of strangling you when you ask it to do some stupid thing for your amusement for the 74th time.

I mean, I know some humans who would miscatacorize a mineral as a dock sometimes, are they not intelligent anymore?

I'm pretty sure gptchat can get those definitions right better than the average person.

" I mean, I know some humans who would miscatacorize a mineral as a dock sometimes, are they not intelligent anymore"

Some humans also ain't intelligent, yes.

(But of course even the dumbest humans automatically make very complicated calculations in their head, just to maintain body functions. Language is tricky.)

Taking these shortcomings a step further, it may not be long before we can't tell reality from simulation, since we've all so plugged in.

A friend who lived in the amazon with me for sometime did a diet with datura, a potent psychoactive. He described his experience as one moment sitting in a cafe completely lucid and chatting with friends, then a moment later he is somewhere else altogether, this continued for some time and of course none of it was real despite it beinf indiscerbible to him within that experience. People often lose their sanity with this as a result of inability to find a way back to the old known "reality". With AI and neural link like interfaces... we have interesting times ahead.

> What is the real criterion for a machine to be considered intelligent? What does it have to do for people to not retroactively downplay it by invoking mechanistic pseudo-explanations?

When it’s capabilities are no longer a proper subset of an average competent human’s?

>When it’s capabilities are no longer a proper subset of an average competent human’s?

Interesting, I'd say that it's definitely already the case with the current iterations. The average human can't write as well as ChatGPT, or speak as many languages, or translate between them.

The average human can't code , even if often subtly wrong, in most programming languages.

It's definitely not a proper subset of average human intelligence anymore.

Good point. My criteria is bad. Technically speaking, Microsoft Excel's capabilities aren't a proper subset of an average competent human's either.

Perhaps,

When an average competent human's capabilities within their field are a subset of the AI's (for all fields of interest).

is a better criteria.

Basically if an AI is going to be a doctor, I want it to at least match the average doctor in capability. What I'm concern about are blind spots that the AI might have that the average human (within their field) doesn't.

A jetliner pilot AI is worthless even if it can do fighter jet maneuvers with a jetliner but goes schizo if a bird flew in front of it during landing.

> What is the real criterion for a machine to be considered intelligent?

I reckon not many have ever said that machines can't be intelligent, in any way.

I think the most compelling question is: what makes them "human like" intelligent.

Passing a multiple choice test is not an actual measure of human intelligence, it's IMO a test of "semantic searching" abilities, that machines can be trained to emulate.

> What is the real criterion for a machine to be considered intelligent?

Get a nobel prize or two, then we'll talk.

I recently learned that this moving of the goal post with regards to AI is a known effect, and it's been a thing for at least a decade: https://en.wikipedia.org/wiki/AI_effect

Bit of a bland coinage. It's a kind of anchoring, which is a fairly difficult tolic in psychology.

Reply on news.ycombinator.com