>FWIW, many vision researchers believe that the resemblance of the first convolutional layer to Gabor filters is perhaps more a case of selection bias than anything else. The argument goes that were they not the output of the first layer, that paper wouldn't get accepted =)

well, i can see the temptation - the orientation and spatial frequency selectivity are the major characteristics of cells in V1 and the receptive field for the first layer there does look like Gabor

http://www.scholarpedia.org/article/Area_V1#Receptive_fields

I agree that such a good resemblance of the learned kernels to Gabor is too good, this is why i used "uncanny" :) If it is real then i think it manifests very interesting and, no pun intended, deep emerging properties of the neural net learning process (something along the lines "maximum entropy kernels while still doing the job" as the asymptotic state)

Btw, is it really selection or confirmation bias?

And to expand on previous point of convoluting the input with many-many kernels - happens to be at the order of 40 per "pixel":

"V1 contains a vast number of neurons. In humans, it contains about 140 million neurons per hemisphere (Wandell, 1995), i.e. about 40 V1 neurons per LGN neuron. Such divergence gives scope for extensive processing of the images received from LGN."