The Lone Banana Problem in AI
digital-science.com
digital-science.com
AI might have problems with single bananas, but it can do that very well:
> In an experiment with the AI program Midjourney, the author found a peculiar issue: the program rendered images of monkeys holding bananas, but it consistently depicted two or more bananas even when asked to render a single banana.
The author suggests that the AI’s predilection for rendering multiple bananas might be due to biases in its training data or the lack of precise labeling. He points out that AI systems like Midjourney don’t understand objects in the human sense but rather recognize common patterns. These systems are only as good as the data fed to them and can inadvertently contain biases or incorrect representations.
The article further delves into existential questions about the nature of human intelligence, creativity, and morality compared to AI’s pattern recognition. It questions whether human cognition and morality are just advanced pattern matching with a better understanding of the physical world. The author also touches on the distinction between programming and prompt engineering, highlighting that the latter requires a more nuanced understanding of language and how AI models interpret it.
The author was trying to produce a single banana on a grey background.
The monkey was a later attempt at prompt engineering.
That AI summary is completely misleading.
Can't do better then brute force for verification.
Say it plain and simple. No one wants to read your train of thought.
"I believe that we are all the same person. How could anybody come to think that? In this paper I shall try to explain how it happened to me." [https://www.researchgate.net/publication/233329805_One_Self_...]
The biggest communication problem is that you always need to take into account that your reader might be braindead trash. That has two consequences:
1. Unless you expose your full train of thought, expect screeches like "I dun unrurrstand, SPOONFEED ME BASIC REASONING, REEE".
2. Unless you explicitly say something, expect some assumer to claim that you said the opposite. Bonus points if this is due to failure to take context into account, or even notice that the context is missing.
Both consequences have been training writers to idiot-proof their texts with big walls of unnecessary words. And that's the case here.
My preferred format would probably be:
- answer to the title / synopsis
- the article, without ads, using the site’s style
They told me what the problem was in about 2 seconds, and frankly, the title and the first sentence alone were.
All right, I concede, a blind reader would not know, but most people are not blind.
The page is not long, the text is not complex. The message is obvious and it is contained in the title as shown here on HN.
The first pic and its caption make it plain.
The next group of 4 pics and their SINGLE SENTENCE caption spell it out clearly.
As for "almost the whole article" -- it is short! It took about 90sec to read the whole page, top to bottom.
"AI" chat bots can't count. This is well known. It is the stuff of memes.
Cool. The article has 2348 words, so that's around 26 words per second, or ~1560 words per minute.
According to some speed reading pages I randomly found via Google the consensus seems to be that 1000+ words per minute is quite exceptional. https://irisreading.com/what-is-the-average-reading-speed/ puts average adult reading speed at around 300 WPM, which you casually exceed by 400%.
Far be it from me to suggest that your estimate is inaccurate, but at least know that not everybody reads at that speed :-)
Points of comparison: I originally read Hal Clement's classic Mission of Gravity in about 25min, and Joseph Conrad's Victory in about 3 hours.
If the can't skim, for example if they are a foreign language speaker of low-intermediate ability (as I am in 5 or 6 languages, and I am also a professional language teacher among other things), then I expect them to know that they can't skim, and make appropriate allowances.
Yes, there were pictures that communicated the same thing, but people naively thought that those 2348 words may actually hold some additional information, as otherwise ca 2300 of them would be completely superfluous. But they were wrong; again, OP's point. Not sure what's so mystifying by this.
As I said, the title of the article and a couple of image captions are enough to get the gist, and as such, I find it baffling that so many people seem to have totally failed to understand it.
The point is, LLMs look smart but they are not, and an easily-verifiable data point is that they can't count.
You can get them standing side by side wearing trenchcoats. You can get a cat pyramid or a cat totem or a stack or tower of cats (though often the ability to count to three is then lost). You can't get them to share the trenchcoat. Nothing like that occurred in any training set, and the AIs do not understand spatial relationships between objects ("X is on top of Y, inside Z") so you cannot describe how to arrange the things it does know about in the scene. Dall-E won't do it. Midjourney won't do it. Stable Diffusion won't do it.
Eventually, enough images will be seeded into the training sets for this to stop being a useful test. But right now, it gives one a fascinating window on what happens when you try to extrapolate outside the cloud of thingspace described by the training data, rather than just interpolating within it.
> As an AI, I can't actually see the painting, but based on the title and the dimensions you've provided, I can imagine a possible description.
> The painting, titled "Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style," is a large piece, standing 6 feet tall and 4 feet wide.
> At the bottom of the painting, the trenchcoat's hem brushes the ground, obscuring the cats' feet. The trenchcoat is a deep, rich color, perhaps a classic trenchcoat beige or a more dramatic black or navy. The coat is painted with careful detail, showing the texture of the fabric and the gleam of buttons.
> Moving up the painting, the middle section reveals the outline of three cats stacked on top of each other within the coat. The artist might have chosen to show this through a slightly open coat, or perhaps through the distinctive shapes visible under the fabric. The cats could be of different breeds and colors to add visual interest and humor.
> The top third of the painting, the upper two feet, features the head of the topmost cat peeking out from the collar of the trenchcoat. The cat might be looking straight ahead, trying to maintain its human disguise, or it might be looking out at the viewer with a humorous or mischievous expression. If the artist chose to include a hat, it would be perched on the cat's head, adding to the disguise.
> The background of the painting could vary greatly depending on the artist's style and intentions. It could be a simple, solid color to keep the focus on the cats, or it could be a detailed scene that adds to the story - perhaps a city street to tie in with the Vincent Adultman reference, or a domestic setting to contrast with the cats' unusual behavior.
> Again, this is a hypothetical description and the actual painting could be very different.
Until larger models are released, there are "hacks" like this available: https://bair.berkeley.edu/blog/2023/05/23/lmd/ - GPT4-generated bounding boxes guiding stable diffusion
Don't feed the beast.
https://github.com/libreddit/libreddit
You can self-host it as well.
These models have a tendency to move towards the average, especially if unprompted. As we see here, sometimes even if prompted otherwise.
They just have to get better or need a more precise interface like the —no :). We also couldn't have "a man crawling" before and now we can: https://i.imgur.com/ycVpk3i.jpeg
Adjusted and with help of the other commenter°: https://i.imgur.com/e2aE1bQ.jpeg
And you always have some crap generations :)
I didn't think it was that long, it might be that some are cherry picking the points they find interesting and think the rest could have been edited out.
I could have stood even more discussion on how this 'blind spot' in the AI model is very much akin to our own blind spots.
How is this at all whatsoever needed
Not to mention complaining about wealth disparity from investment as if it is a sin of the technology is asinine. If money can produce better results you will see wealth disparity from the simple feedback loop of "more money -> better results -> more money". Your neighbor taking up jogging while you sit on the couch all day is increasing a health disparity, should your neighbor stop jogging then?
Undermining of democratic institutions is just plain a dumb meme from vested interests who cast what were previously abuses or necessary evils of the system at best as integral to democracy. We would all recognize blaming the media for undermining the Catholic Church by documenting their child sexual abuse as batshit insane. Yet somehow it is different when it comes to newspapers moaning about their loss of monopolies and influence of cliques.
For "dumbbell", for the AI, no dumbbell was complete without a muscular arm holding it. That's because in most images in the training dataset have the dumbbells being held, so the system integrated the arm in the pattern. I guess it is the same idea here, most pictures of bananas show several of them, so for the AI, bananas are things that don't go alone.
This is the classic lamb vs. mutton issue. Wealthy land owners who use one set of idioms vs. servants who use a different one. Happens to neural nets on human chassis as well.
Someone should have told Jean-Paul Sartre this. He didn’t have to be miserable wondering what it’s all about.
Maybe the popular sci-fi image of "artificial brains" with humanoid bodies actually turns out to be more accurate than the technical view that robotics and artificial intelligence are mostly separate fields!
What if, instead of machine learning being a source of features we can maybe later build into robots, it's the other way round? What if direct interaction with the real world through robotic limbs and sensors turns out to be what it takes to get AI to stop hallucinating?
What if, furthermore, having a body turns out to be an essential requirement for AI to develop a sense of self? Not just because of sensory feedback, but simply because it provides an anchor point for identity?
We don't even understand ourselves and we hope to model AI alignment in some image of humanity with the goal that it will be just as benevolent as our fractured war eager society.
Yes it is a paradox indeed. I submit there is likely a limitation to how much theoretically AI could ever improve beyond ourselves in the regards to bias. I've described this as the AI Bias Paradox - https://www.mindprison.cc/p/ai-the-bias-paradox
That prompt works fine in DALL·E 2.
But the point isn’t about specific prompts where the error is obvious and relatively easy to fix. This is just a n extremely easy example to understand.
The point is about the subtle biases that are much harder to detect, and which will therefore go unfixed.
https://www.bing.com/images/create/an-invisible-monkey-with-...
(Adobe Firefly also fails the invisible monkey task, despite usually having better image quality than Bing/Dall-E.)
But also Midjourney in particular seems trained to be more evocative / stylistic rather than photorealistic or precise.
Here's my excerpt with GPT-4:
As an AI, I can't actually see the painting, but based on the title and the dimensions you've provided, I can imagine a possible description.
The painting, titled "Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style," is a large piece, standing 6 feet tall and 4 feet wide.
At the bottom of the painting, the trenchcoat's hem brushes the ground, obscuring the cats' feet. The trenchcoat is a deep, rich color, perhaps a classic trenchcoat beige or a more dramatic black or navy. The coat is painted with careful detail, showing the texture of the fabric and the gleam of buttons.
Moving up the painting, the middle section reveals the outline of three cats stacked on top of each other within the coat. The artist might have chosen to show this through a slightly open coat, or perhaps through the distinctive shapes visible under the fabric. The cats could be of different breeds and colors to add visual interest and humor.
The top third of the painting, the upper two feet, features the head of the topmost cat peeking out from the collar of the trenchcoat. The cat might be looking straight ahead, trying to maintain its human disguise, or it might be looking out at the viewer with a humorous or mischievous expression. If the artist chose to include a hat, it would be perched on the cat's head, adding to the disguise.
The background of the painting could vary greatly depending on the artist's style and intentions. It could be a simple, solid color to keep the focus on the cats, or it could be a detailed scene that adds to the story - perhaps a city street to tie in with the Vincent Adultman reference, or a domestic setting to contrast with the cats' unusual behavior.
Again, this is a hypothetical description and the actual painting could be very different.
Models do NOT extrapolate unless they are given access to resources/data that is not somehow in their training dataset.
We have done so much encoding of knowledge that a majority of serious use-cases for LLMs are Interpolation, but the fact that we even need retrieval augmented generation shows that they do not extrapolate.
You see this claim everywhere - that AI operates on statistics and patterns and not actual understanding. But human understanding is entirely about statistics and patterns. When a human sees a collection of particles and recognizes it as, say, a car, all they are doing is recognizing the car-like patterns in how the particles are organized that have a strong statistical correlation with prior observations of things classified as a car. Am I missing something?
The day where these models can show shoes under a fence and a reflection of the person behind the fence in an opposing shop window? That day will come. But not on the current crop.
I've been told that in the real world monkeys and chimps don't peel bananas when they eat them, they eat the peel and all, but i don't know if it's true. Whatever happens in the real world, drawings always show monkeys peeling the bananas as they eat them and I would expect a prompt "show a monkey eating a banana" to show it eating a single peeled banana.
Humans have shitty models for stuff, but we have them. We lack the massively deep numerical associations.
It can't be that long before someone makes an AI that says knows a few common models like "stuff made of steel is rigid, stuff made of cloth is soft, etc". Along with "If someone keeps emphasizing a number, that's the number of things he wants".
Chatgpt: Based on the description provided, it is likely that "Misiblur," the art piece made of cloth resembling a butterfly, would be more comfortable to rest your head on compared to "The gattoya," which is made of steel bars welded into a spiral shape.
The reason for this is that cloth generally has a softer and more cushioned surface compared to steel bars. When resting your head on Misiblur, the fabric would provide a certain level of support and padding, which could make it more comfortable for resting your head.
On the other hand, resting your head on steel bars, as in the case of The gattoya, would likely be less comfortable due to the hardness and lack of cushioning provided by the material. Steel bars are not designed to be soft or ergonomic for resting purposes, so it may not provide the desired comfort.
It's important to note that the comfort level can vary depending on individual preferences and the specific design and construction of the art pieces. However, based on the given materials and descriptions, Misiblur made of cloth would generally be the more comfortable option for resting your head.
Does it make pictures of people who are comfortable in one but not the other?
Most of the time when I see something funny in LLMs it's in the images. People with 8 fingers, elbows bent backwards, that kind of thing. Spatial models.
(edited to remove incorrect info)
I'm saying that we shouldn't expect the models to come up with things we didn't train them to come up with.
The “Lone Banana Problem” describes subtle biases of Large Language Models (LLMs) in AI: LLMs reproduce the statistical average of the inputs that they have consumed in the context of the question they have been asked. It's called that problem because the model used to generate images has never seen an individual banana, so when prompted always generates two bananas.
It's impossible to do something like, "A sphere on the left of the picture." and have it understand it.
After having iterative conversations in ChatGPT, Midjourney felt like it has an extremely poor grasp of language.