AI Has a Hallucination Problem That's Proving Tough to Fix
wired.com
wired.com
Having raised children and now playing with my grandchildren, it often amazes me at just how much we take for granted in that comes "out of the box" with a human brain. Humans can build associations with very few samples, and we come pre-wired with all sorts of tools from primary systems like hearing, visual, sensory systems to complex capabilities like speech and communication abilities.
I've worked with computers for 37 years, and the progression has been amazing, but we're still a long ways away from the primary capabilities of even a house cat.
All this said I often wonder if the reason for the failure of current AI systems to wow us is the gap between power density of a human brain vs. compute systems. I've heard it said that DeepMind burned an order of magnitude more than the 20W/hr a typical human brain uses. When we have compute systems with that power density, we may see more emergent behaviors from our silicon-based friends.
Either way, I think AI has useful applications today and I hope we will find many useful applications of these technologies to make our lives better and make more time for us to learn, love and care for one another.
Just to be clear- "AI" ≠ classification and even more so, AI ≠ machine learning ≠ deep neural nets. The article above runs fast and loose with the terminology, but machine vision and in particular object detection (or classification of objects in images) is one area and one sub-task of AI in general. It happens to be one of the two or three areas that have seen strong empirical results in recent years, but it's by no means the only active area of research (although, thanks to the funding from large technology companies, it is probably the fastest growing one).
It's also very strange to see "brittleness" as a criticism of deep neural networks in particular (the attacks described only work on convolutional neural nets as far as I know). In the past, the type of AI system criticised as "brittle" was the hand-crafted rule-based expert system type of AI. And the reason why that kind of AI was criticised as brittle is because it did not deal very well with the noise in real-world domains, such as in photography or speech etc. Deep neural nets in particular are extremely robust to noise, which is why they work so well in speech and image processing.
I think what you really mean by "brittle" is the tendency of deep nets to be, well, a little too good in dealing with noise. Specifically, they have a tendency to overfit to the noise, because they produce models with very high variance. Indeed, the whole adversarial examples thing is probably best understood as a result of overfitting.
As to "lacking fundamental features of human understanding" what you mean, I think, is that image classifiers only do classification and nothing more- which is true, but then that's what they 're designed to do. Nobody expects an image classifier to have any understanding of the images it's classifying.
Humans are inherently social; we have a model of how other humans behave which is essential to getting things done. It doesn't involve say mistaking a bunch of white noise for a camel.
For another example, when you're driving, don't underestimate how many of the rules are not written down and involve a model of what the other driver (human) might do. Those rules are also specific to a geographic area and evolve (slowly and imperceptibly) over time.
The question "Is this person going to be surprised by what I do?" is inherently different than "Is this object a cyclist?"
These attacks work on basically every classifier. It works for multi-layer perceptrons, it works for linear models, it works for random forests, it works for SVMs. It works on basically anything that tries to generalize far from the points it is trained on.
There are a lot of super interesting results in this area, but it's practical significance is waaaaaay over-hyped. One cool thing I saw recently was that some researchers made a physical turtle statue with a texture on it that fooled ML systems into thinking it was a gun, with I think the implication that you could make an AI system think a gun was a turtle, but you could also just put a gun in a turtle case, which fools humans as well as AI systems.
Handling these oddities will most likely require a strategy of smoothing out the complexity, rather than deepening it. My guess is that solving one of these kinds of problems will solve most or all of them together.
The brain of those creatures are tiny. No scientists are handholding them as they grow up.
Yet they manage to search vast areas for food, learn new ideas like stealing from shops and tourists and even as you mention - self replicate.
Yes, while AI is improving we are orders of magnitude away from replicating animals.
My point is it seems we are still quite a few steps behind.
How many tons of steel?
And it isn't even selv-replicating.
As impressive as the moon landing is it is a totally different thing (and has very little to do with AI.)
To the former, I always assumed there were in flight adjustments after launch. Elon's roadster had inaccurate calculations, I heard they were off by like 2%. I assumed the moon landing was similar.
Consider, on the other hand, the number of watt-hours AlphaZero spent to reinvent the current state of the art in Go strategy, vs. the number of cumulative watt-hours the brains of the global community of Go masters spent—over centuries—to figure out that state of the art in the first place.
ML might be kind of expensive to run right now, but its "thinking" can already be horizontally parallelized in a way human thinking just can't. For jobs that fit in one human brain, the human consumes less energy to get the result. For jobs that don't fit in one human brain, humans will have to do exponentially more redundant work to get the same results as one computer cluster.
Now, if we manage to get to "instantaneous recording and ingestion of mental mastery of a skill, like in the Matrix" before we get to "AI that builds better AI", maybe humans will (at least temporarily) be the better thinkers once again. Maybe we'll go back to the model of having offices full of human "computers" analyzing problems!
This to me is an example of whatever the opposite of anthropomorphism is - assuming that humans sample like computers, and then extrapolating to "low" relative to computing. It's also my #1 pet peeve in DL debates.
As someone who has also raised children I can see how this conclusion (low sample rate) can be made, however as someone also deep into ML/RL I see how wrong it is.
You say "very few" samples without a metric. I've seen people in the past cite 2 or 3 presentations of a stimulus to a child, for example in the form of a toy, and then state that the child has correctly visually identified the toy with a verbal label in subsequent tests.
Assuming that these 2 or 3 presentations correlate with 2 or 3 samples is wrong because it doesn't take into account sample rate.
Every presentation batch is a 4D (continuous time + three dimensional) multi-sensory supervised labeling exercise at first (no RL until the first recitation/exploration). Using rough abstractions, at 60 "frames per second" input rate, and lets assume there was a "supervised labeler" (aka parent/guardian) which said the word "toy" multiple times across a 5 minute play period, you have up to 18,000 "labeled" pieces of training data across multiple sensory inputs for one object.
If you blindfolded the child and had them identify the object by feel you may need more batches, similarly with other senses (smell for example).
Obviously this is a gross simplification - but the constant 1:1 batch comparison at the sampling rate between humans and [linear models/MDP/differentiable programs/Neural Networks] really is way off.
Machine vision classifiers can do nothing of the sort, no matter how many examples you give them and for how long they learn to look at them. If you label a fire engine toy as a "fire engine" then either the classifier will only be able to recognise toy fire engines, or it will have to mislabel real fire engines as "toy fire engine".
I agree that the difference between the sampling rate of humans and machine vision classifiers is not well defined, but it is obvious (and far as I can tell there's a strong consensus on this) that machine vision algorithms are many orders of magnitude less sample efficient than humans.
I don't have that same experience at all. In fact if anything it's the opposite. My kids called ambulances "fire trucks" until I - the supervised labeler - corrected them.
that machine vision algorithms are many orders of magnitude less sample efficient than humans.
I don't think anyone disputes that - but they are at least in the same ballpark in terms of structure, especially if you look at the way RL works.
The 4D argument is even more applicable to human language, IMO. Object recognition pretty exclusively involves sight and touch. Human language involves all the senses, frequently at once.
My Spanish is not great, but usually I can communicate pretty well despite that, partially because there are a lot of other contextual cues (body language, nonverbal vocalizations, known objects) I can use to figure things out.
It's amazing how frequently the words don't matter at all, and the meaning is almost entirely contained in tone and pacing of speech.
Again, define "few." Language development starts in-utero [1] and basically is a constant stream thereafter.
Children who have more consistent exposure to directed language and singing from their parents learn language faster, so there is absolutely correlation between exposure rate (sample rate) and acquisition time.
Additionally the idea that language isn't 4D is just completely missing the concept. There is no linguistic association with a "ball" if there is no physical (visual/tactile) representation of said ball. Assuming a child doesn't have a disability there are no single sense concepts that I can think of.
[1]https://www.washington.edu/news/2013/01/02/while-in-womb-bab...
If my understanding is remotely correct, an RNN's view of a banana is basically like the face in Aphex Twin's Equation -https://youtu.be/M9xMuPWAZW8?t=5m30s (headphone users beware). No qualitative or quantitative information about the object, just a certain tone of integer triplets in a cacophony of noise.
It seems like a many-dimensional view of the world around us is going to be necessary for systems to more effectively intuit about interacting with it. It could be something we synthetically inject or we may need to give our models new senses they can use to extract their own meaning.
As you more or less correctly point out, the way a DNN understands a 2D image of a banana is by basically compressing (convolving and pooling) an image into a mathematical "fingerprint" for which we provide a label. If the labeling process is homogenized then we can relatively rapidly generate inferences when testing the fingerprints on new images at a high probability.
That is to say the complexity of the "fingerprint" of a banana is several orders of magnitude greater in humans than it is for even our most advanced object detectors - if for no other reason than the mapped data is multi-sensory.
Rather than anthropomorphizing the AI, it's more like expecting superhuman qualities. People overlook objects in their rear view mirror all the time, but the expectation for the AI is that it will never make that mistake.
Similarly, my mind can differentiate from the floaters in my eyes because it remembers their existence. O do that it had to build a concept of a floater, etc.
Take almost any photo that has dense content, mirror it, and "faces" or "creatures" will appear. We're primed to parse curvy symmetric shapes as organic, and if there's any hint at all of eyes we'll probably see a face.
Optical illusions are a catalogue of other failures. We can easily be tricked into misreading colours, shapes, and lengths.
Because the visual cortex works at a higher level of abstraction than naive pixel-reading classifiers, we're not likely to be confused by adversarial pixel flipping. But we're just as likely to misread equivalent abstractions at a higher level.
Things get even worse at higher cognitive levels. The recent study into the spread of fake news shows that we're far more likely to mis-parse "facts" than to be ruthlessly objective.
The underlying problem is that we expect AI to show the same predictability that simple digital logic does. AI won't and can't, because the problems operate on a completely different level and uncertainty is built into the domain. The best solutions will be better than humans, but it's naive to think infallibility is possible.
Of course there are more persistent failure modes where we can be tricked for a lot longer, and artificial illusions that persist despite knowing what they are.
But I think that actually strengthens my point, blind people are capable of learning how different object feel and they recognize them this way but they so just fine without vision and if they suddenly gain access to vision it does not help them.
In general having more senses will help to more reliably identify objects because you have access to more features to differentiate them, for example distinguishing materials from imitations just by vision can be hard or maybe even impossible. But when you can also touch them you gain a lot of new information about surface structure, hardness, thermal conductivity and so on and you can easily distinguish between, for example, real stone and plastic or wood with a stone print on it.
But just because it is advantageous in the general case to have access to more than one sense that does not imply that it is necessary for a specific task.
We might be more on the same page than I realized On the one hand, I agree that it doesn't imply that it is necessary to have access to more than one sense.
On the other hand I do think that having access to more senses may make the problem a LOT easier, especially during the training process for teaching systems to do things like try to distinguish objects by sight.
Don't get me wrong, the progress made so far has been incredible, but it isn't surprising that we are eventually running into limits given the limited amount of data these systems have to train on.
Sort of riffing off of this - In particular, my sense is that when you are training on 2-D pixel data only, it is no wonder that no matter how clever your network is to extract high-level features from the data, you are going to run into these kind of issues. We are asking the system to perform a task (describing, classifying, etc.) about a fundamentally 3-d world, using only a 2-d image, when it has never had a "concept" of the world in 3 dimensions. I think that we take for granted that when we are learning, not only do we get information by seeing, but also touching and manipulating objects and existing in the world around us. We can see things in different lighting conditions, and we can also move our heads and our bodies and around the world and see things from new angles and manipulate them to learn the rich set of correlations between what we see, and what we experience...basically the rich structure of the world around us. A system which is trained on images alone doesn't have the integrated knowledge, which is what I was hoping to get at with the anecdote of adults being cured of blindness.
When you look at some of the work we do to augment visual data sets, by scaling images, cropping them, rotating them, skewing them, etc. its basically a poor mimcry of something that humans and animals get just by being in the world...this idea of learning that things remain the same and have a certain structure, regardless of the viewpoint we see them from. (From what I understand, this idea is also part of what inspires capsule networks).
Anyways, to come to my main point. Yes, you can absolutely identify objects, even a completely novel object after only seeing it once or twice. But I would argue that this is only because you already have a rich framework of the world as 3-dimensional spatially, along with all of the other priperties that object have that you have learned. So when you see a 2-dimensional picture, your brain can form a 3-d image of the scene, identify materials of objects, etc., which you can leverage. Things that our current AI systems have no idea of... so perhaps its a miracle they made it this far, and no wonder that we find they can get fooled by flipping pixels!
(I'll end my rant here with the caveat that I am a roboticist, so perhaps I have very skewed views of intelligence and feelings about embodied intelligence, but I'm willing to learn and be disabused of my notions!)
This is also where I think robots have the best chance at really becoming self aware (far down the road). They’re not going to be passive observers.
Why are sensations necessary?
Why do I need to feel hungry in order to eat?
Certainly an automatic feeding mechanism could exist where I eat without the sensation of hunger, sex without the sensation of lust, kill without anger and so on.
Certainly an automatic feeding mechanism could exist where I eat without the sensation of hunger, sex without the sensation of lust, kill without anger and so on.
Because that's an incredibly brittle solution. Imagine yourself in an environment where you have to track your prey for hours or days at a time in order to hunt. How are you going to accomplish that if your "automatic feeding mechanism" takes over as soon as it decides you need food?
Our sensations and emotions combine to regulate our behaviour, not control it absolutely. The ability to make short-term sacrifices in order to achieve long-term goals is dependent on our ability to ignore or override these sensations.
My point is sensations/emotions are an incredible information processing paradigm wich we are largely ignoring when we think of AI.
Do we really want and need a Data or HAL? They're great for story telling, but is that what humanity really needs from it's tools? I don't need to fall in love with Siri like in the movie Her. I just need it to be useful when I want to ask my device oral questions.
>just how much we take for granted in that comes "out of the box" with a human brain. Humans can build associations with very few samples, and we come pre-wired with all sorts of tools from primary systems like hearing, visual, sensory systems to complex capabilities like speech and communication abilities.
> DeepMind burned an order of magnitude more than the 20W/hr a typical human brain uses. When we have compute systems with that power density, we may see more emergent behaviors from our silicon-based friends.
You mean "the 20W a typical human brain uses". A watt is a joule per second, so it already has the dimensions of energy per unit time.
Good comment otherwise :-)
I'm in complete agreement with your point about how amazing things we consider "basic" are. But (using your post as a chance to soapbox) we need to remember that we're not TRYING to match the primary capabilities of a cat (or humans). In part, because we know little about how thought and instinct work on a useful level, but we know more about flowchart-style logic. And as I understand them (which isn't much), even the deeper neural work doesn't try to emulate biological thought. Instead, we focus on a _goal_ where we get a result similar to biological thought and try different (complex) ways of mixing deterministic logic to get there.
Everything is still rudimentary, but going forward I expect that AI (or any of the various related fields often lumped as AI) will do remarkable things beyond biological capability long before certain "basic" (as you say, actually amazing) things. And I don't mean just super-logic or strong memorization - we already have that - I mean they will "think" from thought A to thought B easily (e.g. 'I've read this book before' leads to 'an accountable monarchy is the best system of rule'), while we struggle to do the same, while we can make the A to C conclusion easily ('I like green' leading to 'I need to mow the lawn') that AI finds harder to accomplish or even understand enough to predict.
Biology is messy - just a few days ago we had an article on here about how social comfort makes us warmer...and in reverse, raising the temperature makes us feel more befriended. We have bizarre social rules at some level imprinted into our DNA - Tall people, pretty people, they will find success. Mob mentality, mass hallucination, zealotry, we have "work" and "fun" as mostly separate areas. Being fit is healthy, but becoming/staying fit is emotionally difficult. Our memories are ridiculously unreliable - our impressions and desires shape interpretation, and then continue to reshape the memories on each recall. We categorize thoughts down into intellectual and emotional. Change blindness, desiring comfort but pretending it is ideology, getting defensive when proven or even suggested to be wrong, conspiracy theories, all drama, fiction, gender, parody, humor itself, sleep, xenophobia, celebrities, luxury goods, clowns, desire to get intoxicated, placebos, psychological schema, "I'm not a snitch", ...the list goes on.
We (humans) tend to think there are two options: human-like thought, and cold predictable logic (witness almost every sentient computer in media...including popular science journalism). We don't think "huh, the complexity and history of our biological systems has resulted in all these surprising connections. I bet a non-biological system complex enough to "think" will have it's own surprising connections because of a totally different structure, physics, and background." They will be surprising because their way of thinking will be something we cannot wrap our brains around unable to emulate because it is literally contrary to the physical way we think. AI will not be simple and unable to process "this statement is false" - but they might find that hilarious. Or perhaps it gives a result similar to horror. Or it isn't worth much attention...but too many such statements is like dirt under your fingernails. Who knows? I only know that expecting an early AI to have the capabilities of a child is most likely self-deceptive.
I don't know when AI as generally thought of will exist. I don't know if it will prove the salvation/destruction/entertainment of humanity. I fully expect it will be very weird from a human perspective, with different strengths, weaknesses, and quirks. I also expect that humanity en masse will translate this collection as "inferior", because our brains say we should be skeptical of "different").
1) highly pronounced in this one class of methods that has surged in popularity in the last 10 years
2) difficult to address because these methods are (so far) quite opaque to human understanding
I work on multiple in-production vision systems and in cases where we absolutely need to know why something went wrong we use much more conventional, but more transparent, learning algorithms. The performance loss is often an acceptable tradeoff for being able to understand your edge cases.
- you suggest that software is easy and logical, in that you can triage, locate, and patch the buffer overflow. But in any sufficiently large codebase, there are simply near-unlimited more bugs to find: hence all the memory mitigation’s that have been implemented. So it’s not a robust comparison from the get go.
- there’s constantly increasing research into being able to diagnose & analyze what CNN’s “see”: possibly not to the level and accuracy that you’d expect from lldb, but how long did it take to have really amazing debuggers for binary Applications?
That’s not true: there are massive amounts we can glean. Perhaps not as easy as reading the disassembly, but not black box.
That link is far outdated/superseded by modern research:
It's easy to find the problem because you know exactly what the car should have seen, and fix that thing.
A lot of what AI is doing (pattern matching, creative problem solving, etc) could be considered “right brain” activities. Some think that many human mental problems arise out of an unchecked, overactive right brain. We still mistake shadows on the wall for something sinister, or read more into a person’s glance than is really there.
Some posit that the right brain is always hallucinating, in a sense; that psychedelic drugs simply disable the left brain, and allow the right brain to take center stage. Until the corpus callosum developed (which allows the left and right brain to send data back and forth across the divide), it’s possible that right brain insights came to us as “voices” in our heads.
See: “Incognito: the secret lives of the brain.”
“The Dragons of Eden: speculations on the evolution of human intelligence.”
The wiki article is actually pretty accurate: https://en.m.wikipedia.org/wiki/Lateralization_of_brain_func...
It’s only hard to hear on a phone speaker at low volume on the 50dB example. All other examples have what sounds like some sort of static or background noise.
Granted, to an average consumer it may sound like just bad audio, but it’s not imperceptible and thus can be screened for.
Most likely we’ll end up in the same perpetual update cycle as other computer security - someone finds an exploit, that exploit is either reported for a bounty or discovered in active use in the wild, a fix is implemented and exploit is added to the test suite.
For ML that will mean adding the examples of the exploits into the learning sets/providing negative re-enforcement feedbacks.
The question will be whether we can get enough of these caught before some machinery injures someone because it saw something that wasn’t there or accepted a malicious command, and it becomes a media frenzy.
This also leads to a moral and legal questions. If a self driving car injures someone, who will be responsible, the person owning the car or the manufacturer?
Since then I think they've postponed the plans for fully automatic driving though.
So the weak point in this case isn't the fragility of machine learning against adversarial inputs, but old-fashioned network security.
i) The authors and commenters of the article have a strong incentive to get funding for their projects.
ii) The mentioned problems are mostly academic. The real-world implications are not tested, unknown, and likely overblown. Yes, one can create contrived examples. But that isn't same as real life.
The way spoofing examples are created is by feeding small perturbations of the same image to a time-invariant detector. It's unclear if such attacks have much practical value. E.g. a password screen won't allow you to try more than 3-5 values. Attacks must be detectable (due to repeated tries of small perturbations). Plus noise and other perturbations can be added to the detector.
I think the solution will come from marrying a top-down approach to the bottom-up one we're using right now. We need more prior knowledge about the world. We need to be able to simulate situations and understand their effects. Maybe what we're lacking right now is a mental simulator of the world, an imagination module. Coupling perception with imagination would reduce the sample complexity as well.
Interesting to note that the researchers that started the adversarial examples craze and invented the imagination module (GAN - generative adversarial network) are one and the same - Ian Goodfellow. He was right on the spot to identify the weak point of deep learning.
This isn't a hallucination problem. It is a robustness issue.
The outputs of modern AI lack conceptual depth and substance. These algorithms produce very shallow categorizations that are only useful in narrow, constrained contexts. Not surprisingly, it isn't hard to break or hack these fragile categorizations.
Sure, one could potentially argue that this has some similarity to human hallucinations, but I think that is a needless distraction. We know with great certainty that our AI techniques don't have the robustness and generality of animal intelligence. We are much less certain about the causes of human mental illness and any resulting hallucinations, so that analogy doesn't really lead us in a productive direction.
I'll keep hoping that things like capsules and "smarter" network design/training and data augmentation will eventually, de novo, help add "safe" generalizability. Perhaps optimizing accuracy/minimizing loss in a more broadly, uniformly random way rather than in optimizing accuracy, weighted by frequency of observations.
I feel like we're building up to some horrible situation here, because these systems are never going to be worthy of trust in the same way that a human is.
As just one example: as more and more of our lives happen digitally, more evidence of real crimes will be digital evidence. So these AI systems can be easily fooled into thinking you were in the wrong place or requested something that you didn't. And there is absolutely no way to correct the record.
You might want to look into how a "chain of custody" is handed by the courts.
There will be a huge class of things that basically can't be questioned, denying you the ability to confront your accuser.
When gathering evidence that you want to hold up in court, you need to think about how to prove that.
I have this worry that the glue that holds society together is empathy. Which is we feel other peoples emotional state as if it were our own. Most of us at least. You can bet for the governments and corporations that deploy AI technologies in order to control and manipulate people that is exactly the opposite of what they're interested in.
Personally, I think the main problem is some combination of a little bit of over claiming in research, and a metric ton of over hyping and generalizing in both media and business circles.
Robust multi model ensembles with strong generalization ability will show up within the foreseeable future, and will be no more susceptible to optical illusions than human beings.
Also ask any Illusionist / magician.
https://mashable.com/2017/11/02/mit-researchers-fool-google-...
I could just as easily put a sticker on a stop sign that makes it invisible to people.
It's not just machine learning; far too much tech - software and hardware - has this problem. Everyday here on HN you see discussions about the benefits of a new idea without any consideration for how it could be exploited.
Just the opposite. Rarely do you see any discussion of the benefits of a new idea. It's mostly hate.
I suspect we're probably talking past each other, though. Concrete examples would resolve it.
https://news.ycombinator.com/item?id=16542183
Notice the top comment is negative. In general, almost every top comment on HN is negative. It's hard to write well positively, just like it's hard to be a comedian and only make clean jokes.
Tools, like ideas, get feedback when posted to a forum like this. Feedback is rarely going to be all positive, especially in the cases where a tool is still rough.
Without straying too far from the original premise, it's worth pointing out that the idea of using a graph database for traditional tasks is in fact a new idea. Most people would not consider doing it, and that makes it new.
Notice an interesting philosophical thing that happens every time philosophical debates occur: We quickly get mixed up in a debate over the meaning of words. "What do you mean by 'new'?"
It's worth excluding such questions from debate. Questions over the meaning of words rarely go anywhere interesting. http://www.paulgraham.com/philosophy.html
Oh dear, look where we've ended up. But it was an intriguing place, I hope.
The overall point is that if you advocate something non-traditional on HN, you're in for a world of hurt. This place is nothing if not adversarial. You'd better have your I's dotted and your T's crossed. But those constraints are what make the medium so interesting.