Comment by karpathy | Hacker News Reader

Comment by karpathy | Hacker News Reader

Parent Full thread

This article would not come as a surprise to anyone who works with ConvNets. Sadly, that might not the case for those outside of the field, largely due to media's inadequate coverage of our advances (but this is common outside our field too). No one in the field really believes ConvNets see better than humans. They are very good single glance texture recognizers. It's as if you flashed an image and looked at it for a split second without giving yourself a chance to look around and take some time to gain any higher-level scene understanding. If you tried this with this image you might also think you had seen a leopard. Another point to make is not from modeling side but from data side. If in the training data the leopard texture is highly indicative of leopard, then the ConvNet will learn to strongly associate it as such. As the article mentions, a quick hack would be to make sure that your training data contains many leopard-textured items of different classes. You might then expect the ConvNet to seek other features to latch on to and become less reliant on the texture itself.

Also, we carried out an experiment on ImageNet and the outcome was that "One human labeler (me, incidentally) with a fixed amount of training and a slightly-above average determination reached ~5% top-5 error on a subset of ImageNet test set". The media sees this and it immediately gets spun to "AI now Super-Human. And we're all going to die." It makes a lot of us cringe every time.

Many people in Computer Vision now consider ImageNet "squeezed" out of juice - we're good at texture recognition and when an object is in plain view, and we're now searching for harder tasks and more dynamic range with respect to human performance, in areas such as harder 3D/Spatial tasks, Image Captioning, Visual Q&A, etc. The hope is that these harder datasets might in turn guide us in developing models with more nuanced understanding.

Might you by chance be familiar with Rodney Brooks' work on subsumption architectures [1]? If not, I would summarize the underlying idea (my words not his) as "don't try to jump too many layers of abstraction in one go" [2].

So I wonder to what extent you would consider this a predictable outcome from the classifier in question not being part of a subsumptive architecture --- which at a guess would look like

  - glance/texture responses fed into 
  - boundary-recognition layers fed into
  - object persistence/tracking layers 
  - fed into abstract scene reasoning

It seems to me, as a non-vision researcher (I mainly worked in planning and control), that the most obvious counterargument to the image being a spotted cat is based on boundary/object/scene reasoning, and that it's "reasonable" for the texture/glance layer to say "looks a lot like a cat texture".

[1] https://en.wikipedia.org/?title=Subsumption_architecture

[2] I realize this may seem, superficially, anathema to deep network research, which advocates letting the network find its own intermediate levels of abstraction. But it's actually compatible in my view because Brooks advocates (again, paraphrasing quite a bit) that the separate layers should have different objective functions, and that in fact the need for different objective functions (in a prioritized order) is the cause of emergent layering in nature. "First, don't die. Second, find shelter. Third, find food etc." So one can imagine deep networks each finding their own locally useful abstractions for each objective function in the "Maslow" chain, while still having some macro architecture that tracks human-imposed design principles.

the limit is that training cannot force abstraction. you can only reach abstraction if you have enough neuron space and the data set is big enough to avoid over-fitting textures.

the problem is.. human vision doesn't work just by feeding a bitmap. we have structure to decode space relationships, shapes and maybe even shadow/light relations. no way we gonna see classificator working on color arrays matching our vision capabilities

Seems simple enough to feed a NN with that abstracted data.

However, the advantage to the texture approach is it's abstracted from a lot of other information. You don't want a classifier to say sofa, when it's a picture of a person on a sofa.

but then you're biasing it toward your perception:

http://www.bespokesofalondon.co.uk/assets/Uploads/bespoke-so...

anyway it does work perfectly if that's what you need, but most proponent are trying to use deep nn to classify 'as good as humans do'

No, I actually would not think I saw a leopard. Humans are really good at recognizing things with faces and legs. Those humans that didn't have the ability to recognize a leopard in a split second were already eaten thousands of years ago.

Hmm, in your opinion do you think this would be a good technique then for digitizing paper maps? And if so, could you point in the direction of a library or textbook you'd recommend?

Do you mean cartographic images ? Like a map of your city or state -> machine learning -> some data like google maps?

Next step is Video. Adding temporal dimension will emphasize extrapolating true 3d shapes of recognized objects.

Not just 3D shapes, but understand actions as they develop in time with recurrent neural networks.

What if you took these same Neural Networks as they exist now, and tweaked the input and the parameters slightly. For the input, use individual frames of an hour long video of a leopard (in order), and instead of having it just identify whether or not there is a leopard, have it identify what in each image is the leopard, and have it try to predict the next frame.

It seems that this is more like the way that we learn to identify things. Then once we establish an understanding of a base class (big cat) we can apply that same model to new cats that we have never seen before with just a picture.

I think a recurrent neural network is more suitable here as the size of context for each example might be different, instead of fixing the frame size.

Reply on news.ycombinator.com