The point is: the process that humans go through when they learn is completely different than the process that contemporary neural nets go through. No one has yet come up with a theory that combines all of the features of human learning into an implementable algorithm. It will surely happen eventually, but there are at least a few more conceptual breakthroughs that will need to happen. Minor tweaks to back-propagation won't do it.
But I think "feature detectors" are exactly what the earlier comment was referring to, e.g. a Gabor-wavelet-style decomposition of the retinal image. Deep learning systems have to learn those; we're born with them.
Well, that's one theory. But I think it will turn out to be a lot more complex than that. One thing that I haven't seen anyone pay much attention to is feature detectors in the time domain, which we clearly have. We notice movement as a fundamental feature. Our movement detectors can actually be triggered by static images [1]. One of the ways we distinguish dogs from cats (I believe) is by the way they move. It would be a very interesting experiment to use CGI to make a dog move like a cat and vice versa and see how those are perceived.
It makes more sense to interpret the comment as saying that humans don't learn an internal image representation. Humans do learn representations of bridges, aircraft, cats, etc. But those are built on top of an image processing/representation system that we are born with, analogous to raster graphics?
Edit: Maybe I'm misreading the comment. What's definitely built in at birth is things like edge and orientation detectors. A zebra detector would be a surprise.
Actually, that is what I meant, though I don't have a reference for cat detectors per se. But there is ample evidence for innate feature detectors of comparable complexity (e.g. human facial expressions), even if the actual target is something other than cats.