Instead, ask yourselves why these deep nets fail after being trained on huge datasets -- and why even more data doesn't seem to help.
The short answer is that mapping directly from static pixel images to human labels is the wrong problem to be solving.
Edit: fixed autocorrect typo
Do you know what proof is? Adversarial examples demonstrate that there is one esoteric failure mode of current deep learning models, one that for all we know is present in human vision (we can't take derivatives with respect to the parameters of our own neurons). It will likely be solved in the next few years. At a minimum you start training on adversarially generated examples.
This response is absolute hyperbole and clearly devoid of any factual knowledge of the nature of deep conv nets and their properties.
Agreed the failure mode may seem esoteric, but note that OpenAI is making a big deal about them.
A non-esoteric way to demonstrate the lack of generalization is to feed a deep conv network real world images (from outside the dataset). Grab a camera and upload your own photo. Roboticists who try to use deep conv nets as real world vision systems see these failures all the time.
"At OpenAI, we think adversarial examples are a good aspect of security to work on because they represent a concrete problem in AI safety that can be addressed in the short term."
https://openai.com/blog/adversarial-example-research/
Hardly proof that deep learning is fundamentally flawed.
Regarding real world issues, these issues come up when you don't separate training and test (and real world) sets properly. My worries would be with implementation.
I'm saying it's not reasonable to expect good generalization in deep convnets that learn mappings from static images to human labels. (Wrong problem.)
No credible machine learning researcher will tell you that deep learning has totally solved "generalizable" computer vision. The only people claiming such a broad statement are usually the media or enthusiasts who have never done any actual research.
So it might be technically correct to say adversarial examples prove (by counterexample) that deep learning fails at generalization, but nobody in the field claimed that in the first place.
It is hyperbole to claim that adversarial examples will be solved in the next few years. That is extremely unlikely, since the reason they exist is due to the linear nature of convolutions (and I don't think anyone is suggesting we get rid of convolutions entirely).