Hacker News Reader

Comment by jonas21 | Hacker News Reader

It's worth noting that modern multimodal models are not confused by the cat image. For example, Claude 3.5 Sonnet says:

> This image shows two cats cuddling or sleeping together on what appears to be a blue fabric surface, possibly a blanket or bedspread. One cat appears to be black while the other is white with pink ears. They're lying close together, suggesting they're comfortable with each other. The composition is quite sweet and peaceful, capturing a tender moment between these feline companions.

View on HN

Also Claude, when given the entire picture:

"This is a humorous post showcasing an AI image recognition system making an amusing mistake. The neural network (named "neural net guesses memes") attempted to classify an image with 99.52% confidence that it shows a skunk. However, the image actually shows two cats lying together - one black and one white - whose coloring and positioning resembles the distinctive black and white pattern of a skunk.

The humor comes from the fact that while the AI was very confident (99.52%) in its prediction, it was completely wrong..."

The progress we made in barely ten years is astounding.

It's easy to make something work when the example goes from being out of the training data to into the training data.

Definitely. But I also tried with a picture of an absurdist cartoon drawn by a family member, complete with (carefully) handwritten text, and the analysis was absolutely perfect.

A simple test - take one of your own photos, something interesting, and put in into a LLM, let it describe it in words. Then use a image generator to create the image back. It works like back-translation image->text->image. It proves how much the models really understand images and text.

I wouldn't blame a machine to fail something that a first glance looks like an optical illusion...

And yet both these astounding explanations (yours and the one in the OP) are mistaking two cute kittens sleeping cuddled in an adorable manner for generic "cats lying together".

Reply on news.ycombinator.com