Hacker News Reader

Comment by delecti | Hacker News Reader

It's about 100B neurons in a human brain [0], but neurons are also much slower. Individual neurons run at under 200Hz [1], which is probably a half dozen orders of magnitude lower than these processors are running at. Maybe it balances out?

[0] https://www.verywellmind.com/how-many-neurons-are-in-the-bra... [1] https://en.wikipedia.org/wiki/Neural_oscillation

View on HN

Neurons do a lot more than 1 calculation for each of those Hz.

Dendritic branching can be extensive and in some cases is sufficient to receive as many as 100,000 inputs to a single neuron. https://en.wikipedia.org/wiki/Dendrite

A more conservative 10,000 * 200 Hz ~= 2 Mhz * 100B ~= 100 Million 2 Ghz processors cores.

Though each dendrite is again doing far more than one calculation per each of those cycles.

While true, the same is also true for many processor architectures. They're not just doing a single mathematical calculation per cycle, and additionally they also take multiple bits of input. A 32 bit processor could be thought of as having 32 "dendrites" of input, depending on what specific calculations are being performed to emulate the functions of neurons.

I recognize that the specifics are very different between a single neuron and a single processor, but the processors are so much faster that I'm inclined to give some benefit of the doubt to the people actually in possession of the array of 1M processors.

So I guess by that analogy a neuron is equivalent to a rather slow processor with a 100-kilobit word size?

No, Dendrites are far more complex than bits. They have multiple chemical pathways and operate in real time.

A you can think of a single dendrite as ~1,000 bits of information though it's hard to say how much of that information is useful as it's rather complex chemical signaling. Which gets into why opioids get people high for example.

> Individual neurons run at under 200Hz [1]

maybe but that doesn't account for extensive nonlinear computation happening along dendritic arbors. Think vast decision trees resolved in parallel, per neuron, at 200Hz.

The processor cores in SpiNNaker run at 200MHz (yes, this is slow!), and can usually issue one instruction per CPU cycle (that's quite nice). However, the trick to getting lots of neurons in is in keeping the number of CPU cycles per synapse event down, since there's a lot more of those than there are neurons.

Yes, bits of the synapse processing code are in assembly coded to waste not one cycle at all. It turns out to be vital to do that in order to keep the efficiency high (and that has many key knock on effects; the synaptic density is a critical parameter for overall model scaling).

In any case, to say that neurons process information at 200Hz is wrong. Or rather it is not even wrong. The individual neurons don't really do very much, but the overall network does a lot and it isn't limited to 200Hz at all. It's just that it handles time in a totally different way to conventional computers...

> Individual neurons run at under 200Hz

So anything your brain can think in 1 second needs at most a depth of 200 layers.

Each neuron operates on it's own clock cycle which messes with these calculations.

anecdotelly every neuron has ~7000 synapses so 200 layers among 7000^200 neurons?

(its probably a meaningless analogy)

Reply on news.ycombinator.com