Hacker News Reader

This shouldn't really be surprising. Machine learning is specifically not magic. The reason CNNs have seen so much success is precisely because they build in translation-invariance, which massively cuts down on parameters while forcing the final function to have the desired structure regardless of wherever gradient descent takes the weights.

Also why most papers in deep learning are network architecture innovation.

View on HN

One more relevant note - (Olshausen and Field, 1997) showed that the filter employed by V1 simple cells could be learned using some simple assumptions about sparse coding and a single image. Translation invariance built in by way of the sampling scheme of the image, small patches.

The filters learned by the first layer of CNNs is usually of the same type, Gabor filters. Not a coincidence.

That was twenty years ago. What's old is new?

What do Gabor filters have to do with this?

Parent is saying Gabor filters are typically effectively recapitulated by the first layer of of the network anyway, as they are a natural representation.

But what does have to do with smoothness and translation invariance which this paper is a demonstration of? You even learn Gabor filters with local connectivity without spatial weight sharing.

Deep learning researchers rediscover Compressed Sensing. News at 11.

Reply on news.ycombinator.com