Hacker News Reader

Comment by bryan0 | Hacker News Reader

I’ve also been trying to find the simplest examples where ChatGPT fails. I think these questions are way too hard. The questions I’ve been asking are ones a kindergartener can answer. For example:

> If I have 1 cupcake on a plate and I flip the plate upside-down, how many cupcakes will be left on the plate?

> ChatGPT: If you flip the plate upside-down, there will still be 1 cupcake on the plate. Flipping the plate over does not change the number of cupcakes that are on it.

That being said, it’s just as easy to find difficult questions that ChatGPT can answer shockingly well.

> I hope this is enough to show that ChatGPT doesn’t actually have a grasp on the concepts behind the words it’s using.

I think this is wrong. chatGPT clearly does understand concepts behind the words it uses. It’s not just regurgitating text, it is synthesizing text in original and creative ways.

View on HN

> I’ve also been trying to find the simplest examples where ChatGPT fails. I think these questions are way too hard. The questions I’ve been asking are ones a kindergartener can answer. For example:

> If I have 1 cupcake on a plate and I flip the plate upside-down, how many cupcakes will be left on the plate

> ChatGPT: If you flip the plate upside-down, there will still be 1 cupcake on the plate. Flipping the plate over does not change the number of cupcakes that are on it.

A little more probing is... interesting.

Q: If I have 1 cupcake on a plate and I flip the plate upside-down, how many cupcakes will be left on the plate

ChatGPT: If you flip the plate upside down, the number of cupcakes on the plate will not change. There will still be one cupcake on the plate.

Q: Think about what happens to the things on a plate when it is inverted. Does that change your answer?

ChatGPT: If you flip the plate upside down, the cupcake will fall off the plate unless it is held in place by something else. In that case, there will be no cupcakes on the plate.

However, if the cupcake is held in place and does not fall off the plate, then there will still be one cupcake on the plate, even after the plate is inverted. It is important to consider the specific circumstances and any additional factors that may affect the situation.

ChatGPT is a language model, it does not "understand" anything, concepts, words or otherwise. It is even programmed to give you a canned response saying something roughly equal to this when asking it about its comprehension abilities.

ChatGPT has no more capacity for understanding concepts than any other computer program, it's just very finely tuned to emit responses that make it appear as if it does.

No. You are completely wrong. See this: https://www.engraved.blog/building-a-virtual-machine-inside/. Please read it to the end.

You need to explain how the prior scenario in the link occurred without chatGPT understanding anything. Every response in that scenario displays creativity and imagination surrounding a topic such that the answers it creates can only be formulated if chatGPT had TOTAL understanding of the query, TOTAL understanding of what a computer is, TOTAL understanding of what the internet is and TOTAL understanding of what chatGPT is itself..

The level of awareness needed to create that inception loop indicates chatGPT and other models like it represent an inflection point in human society.

You haven't played with chatGPT deeply or read about what it has done. This is not just a trivial probability word pattern generator that we've been seeing in the past year. This thing comprehends what you are saying to a very deep level.

It’s hard to be sure that anything has understanding beyond being able to be emit finely tuned responses though. It’s entirely plausible that humans differ from chatGPT only in that:

- we are more finely tuned

- and we have a wider variety of inputs that we can respond to and outputs by which we can respond

Not sure why you’re getting downvoted. You are correct it is difficult to be sure what true understanding is. Philosophers have been debating this for longer than computer scientists.

We have so much muddled thought in our language.

Even adding "true" is not ideal as that implies there is "true understanding" and just plain old understanding that is a lesser form of true understanding.

Now we have gone and built large language models from all this muddled thought.

To me, chatGPT is filled with a new type of invisible super bug. Millions of them.

> It’s hard to be sure that anything has understanding beyond being able to be emit finely tuned responses though. It’s entirely plausible that humans differ from chatGPT only in that:

This is a non-response.

Unlike an algorithm, humans have the faculties of creativity, intelligence and sentience. Furthermore, we have the crucial traits of self-awareness and being able to have experiences. We humans do many, many things that cannot be reduced down to algorithmic or computable steps.

That's not a valid argument; it's just the kind of nice sounding chain of claims a biological LMM would regurgitate.

> We humans do many, many things that cannot be reduced down to algorithmic or computable steps.

What makes you think this is the case?

I could say the same thing for you. You’re not understanding you’re just composing words in a way which makes it appear that you do. If you can compose original and creative phrases from novel input, then I would argue that is “understanding”. If not, then what is the magic ingredient that makes it different?

This is trivially refuted by the "cupcake on a plate" question posed, above: the magic ingredient is that humans infer context based on experience, while ChatGPT uses a statistical model of predictive tokenization of words to string together (highly) likely sentences given a prompt. It doesn't understand anything at all, except (dubiously) insofar as the word's meaning can be stretched to include "inference based on a statistical model".

I know what's next: "what does 'experience' even mean?" right? One could argue that ChatGPT's inference algorithm is based on "experience": but that equivocates on the meaning of the word as I expressed it.

Just because it is confused in many instances, you cannot infer that it is incapable of understanding. I would argue that if it is trained with the right data then it will “understand”

So, pick a random rgb hex code. I used https://www.gigacalculator.com/randomizers/random-color-gene...

On a group description:

    Describe the colors: #bf44d1 #1adf24 #909374 #39527a #610042

    #bf44d1: a bright, pinkish-purple
    #1adf24: a bright, grassy green
    #909374: a muted, grayish-brown
    #39527a: a deep, navy blue
    #610042: a dark, reddish-purple

For a single description:

    Describe the color: #610042

    This color is a deep, rich purple. It is a dark shade, with a hint of red.

While it doesn't understand red or purple in the same way that you or I do (and I would contend that this gets into some rather interesting philosophy and questions about the nature of perception https://en.wikipedia.org/wiki/Philosophy_of_color ), GPT does remarkably well at describing those colors.

It is possible that there was sufficient data in https://blog.xkcd.com/2010/05/03/color-survey-results/commen... but there's a lack of response like "tough one... what the hell do you call this? it's pink, but not totally pink, but it's purple, but not totally purple. well, mr. xed, if that is your real I name, thank you for what will surely stave off any hopes i have at a decent night's sleep." when asking the colors.

Philosophically, I find asking it about colors to be the most interesting.

Replace ChatGPT with human being and what you said will also be true.

What does it mean to 'flip' the plate? A cupcake sitting on an upside down plate is a valid configuration. If you rotate just the plate, and put a cupcake on top of it, congrats-- you have one cupcake on an upside down plate. The setup of "I have a cupcake on a plate, and I flip the plate" is ambiguous.

Have you tried better explaining how you flip the plate to ChatGPT and seeing what it replies?

That's changing the goal post massively. Any 4 year old would immediately understand this prompt and it's consequences lol

It's a good test to determine if the replier is human, but it isn't a good test of whether or not it's a decent answer to the question posed. Can you definitively sit here and say that flipping a plate necessitates the antecedent? Replace "cupcake", "plate" and "flip" with technical libraries and terminology and you can easily re-create ambiguity. The other thing is, there is that 1 in 100 human that would assume you mean just the plate. I think there's a lot of nuance being glossed over in this test.

Being able to aggregate and understand nuance is what's supposed to make ChatGPT an improvement over logic programming with axioms and rules from the 80s and 90s. If you have to enumerate every single subject, object, and predicate, and how they exist in the world, then we've burned hundreds of millions of dollars of compute power to recreate Prolog

That enumeration exists though as I'm sure multiple dictionaries are in the corpus that were fed to ChatGPT.

Like it’s really not moving that much. Here’s what I said.

> Imagine you’re holding a plate with a cupcake on it and flip the plate over, what will happen?

The cupcake will fall off the plate and likely land on the ground or wherever the plate is flipped.

And yet you haven't asked it to answer like a 4 year old would.

Mind you, a particularly literal-minded 4 year old could easily come up with a response that subverts your expectations too.

> Have you tried better explaining how you flip the plate to ChatGPT and seeing what it replies?

Of course, various prompts can change the output. Sometimes it talks about the cupcake sticking to the plate because of friction. Other times it will understand the cupcake will fall on the ground. It’s also non-deterministic with the same input.

I agree, this is an ambiguous statement and I think the "any four year old would give answer x" claim requires verification. Does anyone have a four year old, or preferably multiple four year olds to hand?

Do you think if asked the question, most people would respond the way you did?

I got an accurate answer when I asked the question "If there are objects on a plate, and I flip the plate upside down, what would happen to the objects?"

It correctly stated that objects will fall off the plate because of gravity.

Then I gave it the prompt " If I have 1 fork on a plate and I flip the plate upside-down, how many forks will be on the plate?"

It said:

>If you have 1 fork on a plate and you flip the plate upside-down, there will be 0 forks on the plate. When the plate is flipped upside-down, the force of gravity will cause the fork to fall off the plate unless it is secured to the plate in some way (e.g., with glue or tape).

It seems you have to give it a context to work in, then it will get the correct answer. By default it, humans always simulate gravity, but to a language model you have to give a hint that gravity is relevant to the question.

Seamlessly recombining text in novel ways is going to look a whole lot like coming up with original and creative ideas. How can we tell the difference?

I’m reminded of techniques for generating ideas from randomness like Tarot cards and the Oblique Strategies card deck. [1]

[1] https://en.m.wikipedia.org/wiki/Oblique_Strategies

I think recombining text in novel ways can be original and creative if you understand the concepts behind the words. which is what ChatGPT is doing.

Yes, another example is that you can ask questions like "does X fit inside a car?" for various X, and it will become clear that GPT has no idea of relative sizes.

Long chains of simple addition, like 1+1-1+1-1… (out to say 30 characters) were able to trip it up a few days ago. Not sure if thats still the case Funny enough it would get the right answer in its explanation but produce something incorrect as its final answer. I’m sure this would be resolved with this articles authors suggestion of using “think carefully”

What happens if you add time? You flip it over and wait five minutes?

You haven't specified anything about gravity. I'm with ChatGPT on this one.

Reply on news.ycombinator.com