Hacker News Reader

Comment by bmcooley | Hacker News Reader

Question from someone who is not knowledgeable in ML or AI: Do current implementations of ANNs allow trying out these different types of organizations/structures? Is the structure and workings of neurons sufficiently similar to ANNs that the architecture transfers, or are there huge differences that make understanding and development orthogonal?

View on HN

Most ANNs are not similar enough to biological NNs to attempt to recreate these structures.

A feed forward network is just matrix multiplication with a nonlinearity. It could, at best, be described as biologically inspired.

Also SNNs seem to be an active field of research. Namely research center Jülich (human brain project) is conducting experiments on modeling & simulating biological neuron populations.

https://en.wikipedia.org/wiki/Spiking_neural_network

http://www.nest-simulator.org

The short answer is no, the architectures don't transfer. At least not today. Biological neurons are much more complex, and therefore behave quite differently from neurons in neural networks.

To put things in perspective, we've fully mapped the connectome (map of neuron connections) of the simplest animal, C. Elegans, which has only about 300 neurons, yet we still can't simulate this organism's behavior computationally.

Last I knew, OpenWorm was making good progress on the nervous system simulations, though. IIRC they’ve demonstrated swimming, retracting when bumping into walls, and food-seeking.

Feed-forward CNNs pretty much describe V1 to V5 in primates. There are multiple papers on these architectures in the last decade.

And another question on top of that: is there any biological theory that corresponds to backpropagation?

A really interesting question. It would seem far fetched at first that backpropagation could be used by the brain, because it's unclear what the mechanism for transmitting the error at every synapse backwards, to the synapses in the previous layer, would look like physically.

This is a really interesting lecture given by Geoffrey Hinton (https://www.youtube.com/watch?v=VIRCybGgHts) where he discusses the various issue commonly raised with "biological backpropagation" and proposes a solution based on Spike Timing Dependent Plasticity (STDP). Basically he argues that you can interpret the STDP learning rule as a derivative filter on a firing rate and get backpropagation in this way. This is just on wild idea though and has not been shown to work experimentally or through simulations.

There are also a couple of interesting pointers in this Stackexchange thread: https://cogsci.stackexchange.com/questions/16269/is-back-pro...

You should check the neuromorphic community, really interesting work/people. They go from analog electronics to diehard neuroscience. I loved their Telluride summer school.

E.g. http://journal.frontiersin.org/article/10.3389/fnins.2016.00...

I once attended some talk or other and that question came up - the answer given was flippant, but I always thought it was oddly perceptive...... "Dreaming"

Reply on news.ycombinator.com