Hacker News Reader

I keep saying this to neural net enthusiasts: brains and neural nets don't have anything in common other than some nouns. Not network topology, not even level encoding (bio neural nets are pulse rate encoded -closest "neural net" to this is something like a liquid state machine, which is not easily simulated on a Von Neumann computer). We actually know an awful lot about how things in the visual cortex work. Nothing like a "deep network."

View on HN

Which makes the compelling results of deep networks even more mysterious. (https://gandissect.csail.mit.edu/) How could we have captured the seemingly enormously complex act of visual object recognition, with all the neurons and synapses and chemicals, in something so crude and simple as backpropagation and stochastic gradient descent with a bunch of ones and zeros? Or perhaps both processes, biological and artificial, point to a more fundamental emergent phenomenon of the computational universe we inhabit?

I'm pretty sure if you threw the number of grad students and GPUs at PGMs or various ensemble techniques, the results wouldn't be very different. Fundamentally prediction as we do it is just a form of data compression.

I don't think brains work in a way we understand at all (though pieces like visual cortex are well understood).

Reply on news.ycombinator.com