Comment by danbruc | Hacker News Reader

Comment by danbruc | Hacker News Reader

So what features are those networks actually learning? What are thy looking for? They can not be much like features used by humans because the features used by humans are robust against such adversarial noise. I am also somewhat tempted to say that they can also not be to different from the features used by humans because otherwise, it seems, they would not generalize well. If they just learned some random accidental details in the trainings set, they would probably fail spectacularly in the validation phase with high probability but they don't. And we would of course have a contradiction with the former statement.

So it seems that there are features quite different from the features used by humans that are still similarly robust unless you specifically target them. And they also correlate well with features used by humans unless you specifically target them. Real world images are very unusual images in the sense that almost all possible images are random noise while real world images are [almost] never random noise. And here I get a bit stuck, I have this diffuse idea in my head that most possible images do not occur in the real world and that there are way more degrees of freedom into direction that just don't occur in the real world but this idea is just too diffuse so that I am currently unable to pin and write down.

> I have this diffuse idea in my head that most possible images do not occur in the real world and that there are way more degrees of freedom into direction that just don't occur in the real world but this idea is just too diffuse so that I am currently unable to pin and write down.

Yes! You're on the right track! The number of degrees of freedom of images of pixels and textures is HUGE. There is not enough data to practically learn directly from those images. So the deep networks are starved for data -- even with the big datasets they are trained on. (It's only thanks to the way they are set up they do well when tested on very similar images, like sharp hi-res photos. But they fail to generalize to other kinds of images.)

So how can you reasonable reduce these degrees of freedom?

It turns out that the continuity of reality itself provides a powerful constraint that can reduce the degrees of freedom. See, when a ball rolls along, this physical event is not just a collection of textures to be memorized. It's an ordered sequence of textures that vary in a consistent and regular way because of many learnable physical constraints (like lighting).

So, it turns out you can reduce the dimensionality by making a particular kind of large recurrent neural net learn to predict the future in video. Our very preliminary testing shows it works shockingly well.

That sounds very interesting is there somewhere I can keep up with the progress?

And... you know these are immune to adversarial examples?

"So what features are those networks actually learning? What are thy looking for? They can not be much like features used by humans because the features used by humans are robust against such adversarial noise."

There are two issues with this - first, we know that the lower feature detectors of neural nets closely mimic the feature detection of the human visual cortex. Secondly, the features could be the same while there could be technical imperfections with the later stages of current neural nets.

Reply on news.ycombinator.com