Hacker News Reader

Comment by thereddaikon | Hacker News Reader

>I still don't understand the purported difference between "sounds smart" and "is smart".

Its easy to see in practice with AI. Look at AI generated images and how they just cant get human hands right. Everyone knows a human should have 4 fingers and a thumb on each hand. But the AI does not understand that it is drawing a hand with those parameters. It is making something that sort of looks like other things it has been trained with.

The same is true for text generation. A common description for Chat GPT I've seen catch on is calling it "confidently wrong". It will string together words that sound good but it doesn't actually have any understanding behind them. Its logic is in stringing words together. This is more akin to a good con man faking knowledge than an expert explaining a topic.

View on HN

It's funny to watch the bar for what counts as quality AI continually and automatically rise so that the threshold for "good" is always "just a bit better than what we have today." DALL-E, Stable Diffusion, and ChatGPT are absolutely astonishing. Ten years ago their outputs would be disbelieved by the entire field of Computer Science. 20 years ago they would have been called sorcery. Upon release they were way better than state-of-the-art, and blew away the previous quality bar.

No matter how amazing software gets, there's always someone out there reminding us that it's inadequate because it is not yet perfect.

No, there’s someone out there reminding you it’s not intelligence, because it’s not. Astonishing results are great, let’s celebrate them without claiming they’re something they are not.

I think the reason the bar keeps shifting is that we're in the middle of the uncanny valley with regards to AI. Diffusion models and LLMs are just convincing enough that their mistakes are starting to evoke a more visceral reaction than they did with simpler models.

I think drawing hands with 5 fingers is a reasonable ask for anything you want to label "intelligent". Back in the days of Lisp AI hype, you'd have expert systems told that human hands have 5 fingers.

If you want to avoid overselling, call it ML. The machine has learned to reproduce and combine patterns in images.

Humans often draw bad human hands, as well. Don’t even try to ask a human to draw a bicycle.

For those who haven't seen this little gem before: https://www.booooooom.com/2016/05/09/bicycles-built-based-on...

It's astonishing how recently I saw a thread on Mastodon prompting artists to draw a bicycle and it was not much better than that. What about building a visual library?

Now there's a lot of AI results coming up and it's not performing much better in that regard either though.

https://mstdn.social/tags/CrabCycle

That's because hands and bycicles are complex 3D shapes and it's hard to picture them in your mind from arbitrary angles. It's not because the artist doesn't understand what a hand is or how many fingers it's supposed to have.

Someone else posted this article about how people draw bicycles: https://www.booooooom.com/2016/05/09/bicycles-built-based-on...

You can see that some people have a fundamental misunderstanding about how bicycles work, and this misunderstanding does not stem from the complex 3D shape of bicycles.

Don't you think drawing a hand is a bit of an unfair "gotcha" for an AI that can accurately draw thousands of other things? Humans have hands and they're one of the most important parts of the body; humans use hands for everything. And yet, all but the most artistic humans cannot draw human hands accurately. An AI doesn't have hands. Hands make up a small part of its training data. Why should we consider the hand the standard upon which to judge the quality of the AI?

Given that there are accurate representations of hands in it's data set, and also given that the AI would have no problem copying or averaging only these accurate representations, it seems the AI doesn't have the evaluative ability to recognize accurate versus inaccurate hand representations.

This seems like a huge leap. Sure, there are hands in the dataset, just like a human artist encounters hands every day of his or her life. Yet there are many artists that cannot draw hands but can easily draw other objects they encounter. Or, going back to bikes, some humans cannot accurately draw a bike despite encountering them frequently. Perhaps hands are simply a complex thing to draw accurately. Perhaps the AI's poor drawing of hands is itself evidence of understanding. Rather than produce an incorrectly oriented hand from its dataset, it "chooses" to draw something less specific as a reflection of its lack of ability to draw hands.

And this seems like a huge leap to me. You're imputing a very specific human fallibility to a software that isn't human (or even derived from the same neurological underpinnings).

I guess it would take someone who can look at and interpret the code to figure it out.

I'm not imputing anything on the AI. I'm asking a question and then offering conjecture as to why the current answers the question are unsatisfying. The question is: Why do we accept that a human understands the concept of hand despite drawing hands poorly while simultaneously saying the AI draws hands poorly because it doesn't understand hands?

> "Why do we accept that a human understands the concept of hand despite drawing hands poorly"

We use hands, recognize hands, some of us can draw or sculpt accurate representations of hands, we can talk about aspects of hands such as grip and fingerlength. All of these things encompass the idea and physical instance of "hand" in much more than just the artistic sense.

Also, we certainly know when we draw a hand poorly. Unless we are children, or have a brain condition. And if we are children or have a brain condition other adults wouldn't say that we have a good understanding of what a hand is.

Perhaps the AI knows it's drawing hands poorly. Perhaps it can't communicate this to us. Even so this should be in the code run somewhere.

> "while simultaneously saying the AI draws hands poorly because it doesn't understand hands?"

For two reasons:

1) The AI is just drawing. Even humans draw many things that they don't understand. If we draw something that we completely don't understand (such as through random scribbling) we don't even call it a representation. It's a fluke. I used to scribble and then trace images in my scribbling (if possible). Often I ended up tracing things that looked like a child's bad drawing of Donald Duck, but once, without having to trace particular lines at all, my scribbling was a perfect seeming of a rose flower (with some minor additional flourishes). I recognized the rose flower, but I certainly didn't set out to draw it.

2) I'm assuming the AI wasn't trained on medical and other information pertaining to a hand, the way humans are (even if the training isn't formal). It is trained on images. At best it is trained on images of discrete parts of the body, but this only allows it to understand the shape and relative position of each body part, at best. Not to understand the body part.

Ultimately it would take something looking at the code run to determine whether or not the AI brings an understanding of the concept of "hand" into the literal picture. If all it's bringing is #1 then it's not an understanding of "hand".

> We use hands, recognize hands, some of us can draw or sculpt accurate representations of hands, we can talk about aspects of hands such as grip and fingerlength. All of these things encompass the idea and physical instance of "hand" in much more than just the artistic sense.

Sure, but the AI doesn't use hands so why would hands be of particular importance to the algorithm? Let's say the algorithm did draw perfect hands every time. Would you accept that it has understanding then? I doubt it. Your argument is essentially a slippery slope: no matter what it can do, you'd find something it didn't excel at and say "See, it has no understanding."

> Also, we certainly know when we draw a hand poorly. Unless we are children, or have a brain condition. And if we are children or have a brain condition other adults wouldn't say that we have a good understanding of what a hand is.

I've seen plenty of people claim they produce great art, when in reality it's terrible.

My argument is not a slippery slope, though I see how it could easily go that way. I outline the kind of multi-factor understanding the AI would have to have to be considered to have understood "hand".

This "hand understanding" could probably be simplified for an artistic concept of "hand".

I believe a computer "understands" basic arithmetic on a non-conscious level. I haven't been convinced it understands the human hand, even in an artistic sense.

> "I've seen plenty of people claim they produce great art, when in reality it's terrible."

Yes. But other people typically don't go around saying that those people understand great art. Understand what makes it great.

Art is a skill that takes time to master. There are college level classes on mastering the human form. Just because most people are bad at drawing hands doesn't mean they don't understand what a hand is. That's a fundamentally different problem than what AI systems have.

As for the bikes, I'm not surprised most people cant recall the exact shape or design of a mechanical object they don't use or work on every day. I'm sure a hobby cyclist could accurately lay out one.

> That's a fundamentally different problem than what AI systems have.

Is it? Humans understand that humans have hands, yet most are incapable of drawing a hand well. The AI also clearly understands that humans have hands. It does put something hand-like where a hand typically goes. But, like most humans, the AI is not good at accurately reproducing a hand.

A human's drawing of a hand will show a clear understanding of what a hand is on an intellectual level. The AI's drawing of a hand will not show such an understanding. "They are both inaccurate at reproducing a hand" flattens out this important difference - in fact, this difference is explicitly what the poster is getting at.

> A human's drawing of a hand will show a clear understanding of what a hand is on an intellectual level.

What is the definition of "understanding on an intellectual level?" I have a friend who is an artist. She is doing a series of paintings with figures. She cannot paint hands to save her life. The hands she paints look very similar to the eerie vignettes produced by Stable Diffusion. I fail to see what makes the hands my friend paints somehow superior on an "intellectual level."

Really? Your friend's attempts to draw hands wind up having way too many or too few fingers, or phantom hands disconnected from wrists and arms, or fingers jutting out at odd angles? Maybe this is an artistic choice your friend is making, but I would wager she could paint something more recognizable as a human hand than those, if she wanted to.

Edit, to add some context. How many of the hands your friend paints will look like the ones in this link?

https://huggingface.co/spaces/stabilityai/stable-diffusion/d...

No, those are better than what my friend draws, which are more like ghastly appendages. The examples you provided are at least recognizable as hands. But again, I don't understand this focus on hands, specifically. You're looking at this picture and saying: "See, it doesn't understand hands? Look how many fingers there are!" I'm looking at this picture and thinking: "Wow, it perfectly understands fingers. It understands how light subtly diffracts when hitting skin. It perfectly understands how veins peek out beneath the skin and how wrinkles look."

I think you're getting way too hung up on the number of fingers. It's clear that the algorithm could do a better job with hands given more training data. There is nothing special about hands. So, if you're right that it doesn't understand hands, then this lack of understanding stems purely from a lack of training data. In which case, I think it's fair to say that it understands the other things it has sufficient training data to draw well.

No, the number of fingers is absolutely critical, you simply fail to understand what understanding a concept means. ChatGPT is, in this sense (and I know it’s probably a gross simplification, sorry OpenAI), a fancy markov chain. It can sample data that, by sheer correlation in the training data, reflects many features a typical drawing that matches the prompt would have. But it doesn’t understand what a human hand is, it lacks the ability to abstract, so it cannot deduce that there are (almost) always exactly 5 fingers. The probabilities reflected in the model kinda caught the aspect of a human hand being a collecion of these geometric shapes with those shades and whatnot, but they didn’t catch the hand as a coherent concept with a rather strict definition.

This is like the joke about training a neural net on arithmetic where you get the wrong answer repeatedly until it remembers to answer 5+5=10. but then, until there are more data, 10+5 is also 10, because it didn’t actually understand arithmetic (to be fair, a human wouldn’t understand it by a single example either).

And you can see this in action by making ChatGPT believe 5+3=7.

ChatGPT manipulates symbols, and it captured the rules to do that very well. That is one of the abilities of general intelligence, but it’s not the only criterion for intelligence. You can do more than just manipulating symbols, you can also abstract over them, form your own thoughts and questions about them, be curious, reflect on your reasoning and explain it, deduct patterns from very few data because you have all the context from your previous knowledge and the abstractions built on top of it. Besides of the abstractions and functions programmed into it ChatGPT only has probabilities of symbols being related to other symbols (and the rules implied by that), but it cannot reason about these symbols and cannot form creative thought. Its „intelligence“ is limited to a finite order/level of abstraction (the features and parameters that define the model and allow it, for example, to capture shading and geometry, but not the concept of a human hand) while yours is basically limitless. You can always put another abstraction on top of what you just thought or experienced. The magic of deep learning was basically increasing the order of abstraction a neural net can capture, but it’s still limited.

On the other hand, I have my pet theory that the weirdness of dreams or psychedelics arises from the brain basically sampling the connections in the brain / piping random noise through its neural net (as a side effect of all the reorganization it’s doing).

> No, the number of fingers is absolutely critical, you simply fail to understand what understanding a concept means. ChatGPT is, in this sense (and I know it’s probably a gross simplification, sorry OpenAI), a fancy markov chain. It can sample data that, by sheer correlation in the training data, reflects many features a typical drawing that matches the prompt would have. But it doesn’t understand what a human hand is, it lacks the ability to abstract, so it cannot deduce that there are (almost) always exactly 5 fingers.

Yet it correctly understands that faces typically have two eyes, one mouth, one nose, etc. So clearly this "lack of understanding that hands have five fingers" is unlikely to be inherent to the model.

Let's say I ask you to draw a lady bug. You'll draw a red shell with some black dots haphazardly strewn about. However, the most common lady bug in Europe always has 7 spots. It's unlikely that your drawing will reflect that. Why? Because you lack understanding of Coccinella septempunctata. But does that you mean you lack understanding in general? Of course not. Lady bugs simply aren't important to you.

So again, why are we elevating hands to be the litmus test of understanding? Yes, hands are important to humans. But this algorithm is not a human, so hands are no more important to it than anything else it can do. Like let's say if could draw perfect hands 100% of the time. Does that mean you would concede that it has understanding? I doubt it. You'd pick some other thing it didn't do well and say "See, it can't accurately draw eggs stacked in a pyramid, therefore it lacks understanding." The issue with your argument is that is a slippery slope without a specific reason why the correct rendering of hands is important.

And I'm not arguing that GPT-3 or Stable Diffusion are omnipotent. Clearly they're not. But that doesn't mean that can't understand things in their domain. As others have mentioned in adjacent comments, the only test we have for understanding, in humans or ML models, is measuring the correctness of an output for a given input. Essentially, your argument is that "It's an algorithm, it can't understand like a human," which is begging the question.

I'm not claiming that ChatGPT or any other ML algorithm is "generally intelligent." Just that it has an understanding of certain concepts.

>Yet it correctly understands that faces typically have two eyes, one mouth, one nose, etc. So clearly this "lack of understanding that hands have five fingers" is unlikely to be inherent to the model.

It is, we just don’t know it’s exact features. It might very well be optimized for recognizing faces (and therefore to identify the features that make up a face). A general AI doesn’t have to be retrained on specifics. Sure, you „can“ (in a very generous hypothetical sense of the word) train a model like this on all pictures and movies in existence and then some, so that it has seen everything and never fails to give the wrong answer for any prompt that only involves things that were depicted at some point in time. You don’t have to show a child all hands on the planet for it to recognize hands have 5 fingers. You don’t even have to show children pictures of every body part once for every skin color. They only need to see 1-2 different skin colors once to make the deduction that every body part can come in different skin colors. That‘s understanding, a general intelligence. Try this with a model like ChatGPT and you get a racist model.

>And I'm not arguing that GPT-3 or Stable Diffusion are omnipotent. Clearly they're not. But that doesn't mean that can't understand things in their domain. As others have mentioned in adjacent comments, the only test we have for understanding, in humans or ML models, is measuring the correctness of an output for a given input. Essentially, your argument is that "It's an algorithm, it can't understand like a human," which is begging the question. I'm not claiming that ChatGPT or any other ML algorithm is "generally intelligent." Just that it has an understanding of certain concepts.

We can also inspect the model. And even the ouputs are obviously different from what a human would be able to output, so it fails even that test.

If you argue that’s still intelligence, just on a lower level, you can absolutely do that. But at that point you’re basically saying everything is intelligent/conscious just on varying levels. In the sense that consciousness is what consciousness does. Which is a stance I generally agree with, but it’s also unfalsifiable and therefore meaningless in a scientific discussion.

We actually use tests like that to verify the development of children. I still remember that I had to draw a human and they checked whether I thought of drawing and neck and whatnot when it was time to decide if I’m ready for school.

But were are talking about an AI whose purpose it art, so it only make sense to compare to humans who are art professionals. A human whose profession is art can draw a hand, or at least get an approx. close number of correct fingers. And attach said fingers to actual hands.

And many profession artists cannot draw hands either. We're talking the distinction between "sounds smart" and "is smart." People here are arguing that the AI not "smart"; it only "sounds smart." The evidence provided is that AI cannot draw hands well. But we don't apply this same standard to people, even professional artists.

Do humans artist routinely fail to connect hands/fingers to the rest of the body? That draw fingers with finger nails on both ends? Hands that have 12 fingers when doing realism? We are not talking about 'not very good at draws the details of a hand' but not able to understand fundamental details of what a hand is.

> Do humans artist routinely fail to connect hands/fingers to the rest of the body?

Children do all the time, and I'm pretty sure children have a rudimentary understanding of hands.

But they lack the abstraction necessary to call them self-aware (by the current definition). That’s actually a test to check the development of a child. A child is not very intelligent yet. But it is able to become it on its own just by observing. ChatGPT isn’t.

ChatGPT gained all of its abilities through observation, i.e. training. Changing the discussion from "understanding" to "self awareness" is really moving the goal posts.

I think the question has accidentally shifted here.

ChatGPT knows that hands have five fingers because it's trained on text and text will almost always say that.

DALL-E doesn't know that hands have five fingers, it just knows what hands generally look like, and the number of fingers on a hand is just one of the elements it tries to match. At a glance, there are far more important elements of what a hand looks like, such as the shading.

Neither of these mean either AI is stupid. DALL-E doesn't have a concept of what a hand is, it just has an idea of what is looks like, and it's decent at recreating that.

Reply on news.ycombinator.com