"we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like."
"...you get an output from the end of the layers, and you propagate a signal backwards through the layers to change all the parameters. It’s pretty clear the brain doesn’t do something like that. "
So why can't the brain do some kind of backpropagation?
When we don't know how a thing works that does not preclude us being able to eliminate some of the possibilities. Our ignorance is profound, but not total.
As an aside, are they really trying to patent a slight twist on backpropagation? That seems pretty counter-productive to me.
For some neural nets, you still have a gradient, but the concept of back or forward propagation is not defineable. Based on the topology and structure of biological neural nets, what would you think is the case?
Overall though, I think there is a lot of truth to his message.
Hebbian learning .... maybe. I never thought too much about contrastive divergence.
It doesn't really matter; brains most certainly don't work like ANNs, except maybe in some weird mean field sense for a few things like liquid state machines. It would be a huge coincidence if LeCun or Hinton or whoever magically wrote down the brain equation....