Hacker News Reader

Comment by audunw | Hacker News Reader

I seem to remember research stating that an individual neuron has very complex behaviour that requires several ML “neurons” / nodes to simulate. So if you do a comparison, perhaps the brain is deeper than you’d think by just looking at the graph of neurons and their synapses.

Could we construct a neutral net from nodes with more complex behaviour? Probably, but in computing we’ve generally found that it’s best to build up a system from simple building blocks. So what if it takes many ML nodes to simulate a neuron? That’s probably an efficient way to do it. Especially in the early phase where we’re not quite sure which architecture is the best. It’s easier to experiment with various neural net architectures when the building blocks are simple.

View on HN

> I seem to remember research stating that an individual neuron has very complex behaviour that requires several ML “neurons” / nodes to simulate.

This is probably what you're remembering: https://www.sciencedirect.com/science/article/pii/S089662732...

Yeah, biological brains could be remarkably more powerful than digital neural networks if the have primitive functions that we haven't accounted for. For example, some networks seem to encode information in the firing rate, rather than just the presence of a signal. If neurons could, e.g. do frequency-based calculations (and not just threshold-based, like spiking neural nets), they could be orders of magnitude more powerful and efficient. I am thinking particularly about neurons involved in, e.g. audio processing.

the entropy rate goes way up if you consider spike timing dependent signals as well. but the difference in computational capacity between the brain and ML lies less in the brain's inherently time-dependent dynamics and more in the impressive computational capacity of single neurons. Dendrites compute, electrochemical dynamics during action potentials compute, synapses compute. All in complex time-dependent ways. check out izhikevich's dynamical systems in neuroscience for a taste of the computational capacity of the electrochemical dynamical system alone

My guess is that the firing rate of biological neurons more or less simplifies to the activation in an artificial neuron. Higher firing rate = higher activation.

> Could we construct a neutral net from nodes with more complex behaviour?

Well there's spiking neural networks (SNN)[1], which are modeled more closely to how neurons actually work.

Main obstacle is still, as far as I know, that there's no way to train a SSN as efficiently as a "regular" neural network, which lends itself very nicely to gradient descent and similar[2].

[1]: https://en.wikipedia.org/wiki/Spiking_neural_network

[2]: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9313413/

The brain backprops??????

there is no evidence to support this

Reply on news.ycombinator.com