Why Is the Human Brain So Efficient? Massive Parallelism | Hacker News Reader

Why Is the Human Brain So Efficient? Massive Parallelism

nautil.us

129 pointsnbmh52 comments

"a professional tennis player can follow the trajectory of a tennis ball after it is served at a speed as high as 160 miles per hour"

This is false or at least highly misleading. A reader might imagine that the player's eyes track the ball in flight. This is not what happens. A pro player reads the position of the serve and begins moving before the ball is hit. The predictive power of the brain is much more important than the speed and precision the author was trying to highlight here.

Agree. As someone who was playing table tennis for a bit time, where basically you can't see the ball due speed, prediction training is essential and long term process. At some point you get so versed you know where the ball will be just by looking at opponent elbow or arm.

The way other articles describe it, a pro player is not really aware of the incoming ball, lower level non-conscious (reflex) systems track the environment and react before you are aware of it, just like when a regular person dodges a sudden obstacle coming at them and only a bit later they realize what's going on.

That's a meaningless distinction when talking about the brain. The brain is very much doing calculations that we are not aware of, but they still occur.

Maybe I misread it, but I think the poster was talking about conscious vs. subconscious processes, not brain vs. non-brain processes.

Yea, I get they think 'really aware' is a meaningful distinction. But, from an energy standpoint if your doing calculations and you need to pay for them. Further even at 160 mph it still takes the ball 1/3 of a second to travel the court so tracking it is useful to update how your hitting it even if you need to move your body based on predicted location.

Computationally, predicting the location is actually worse as it involves tracking the ball's location and then adds on it's likely location. It's basically stuck going ok my lagg is ~1/20th of a second so I see X, it's actually at Y, but my reaction is going to be anther ~1/20 of a second so I need to try and hit it at Z. And then continuously updating both Y and Z based on new information. Then on top of that you have multiple Z's for different things etc etc.

Consciously you don't really perceive that lag because your stuck with it.

PS: And of course each part of the nervous system has different amounts of lag. To the point where many decisions are made outside of the brain.

Sounds like one of the concepts that inspired the book Blindsight by Peter Watts. The subconscious 'reptile brain' is able to process certain old-school dangers, like an object moving quickly towards us, faster than our conscious brain, and can cause involuntary, potentially life-saving, reactions to the stimulus.

The conscious mind processing these things can often get in the way but, in the case of sports, alter the instinctive behaviour into something advantageous. An augmentation of the conscious and subconscious into better performance.

A converse example, being conflict between conscious and sub-conscious, is if you drop a mug of boiling water, the instinctive reaction is to try and catch 'thing you dropped', but the conscious reaction is 'don't burn yourself' so don't catch it and jump backwards to not get splashed.

Very interesting stuff.

https://www.goodreads.com/book/show/48484.Blindsight

https://en.wikipedia.org/wiki/Blindsight_(Watts_novel)#Consc...

Beware (or since it's the HN crowd, Recommended!): hard sci-fi.

One thing that has always bothered me about many AI approaches is that instead of having multiple layers that deal with different levels of urgency everything is delegated to a "rational" mind that thinks vs a combination of thinking predicting and reacting.

The parallel is this obsession with rationality over intuition. Though somewhat ironic because black box AI is just accepted.

Building something that mimic how it appears to work, as opposed to how it does work. Like building the symptoms whilst assuming we're building the disease.

Having said that, however, gotta start somewhere. Having said that, however, we might have done a decade of work in the wrong direction by the time we find out what direction it should go...

'tis a conundrum.

There is a related anecdote in "The Sports Gene" that suggests that MLB hitters do not track the ball in flight, but instead react to the pitchers' arm motion. As a result they have trouble hitting a motion they haven't visually trained against:

http://www.eugenewei.com/blog/2013/8/23/the-pitchers-who-con...

https://www.imdb.com/title/tt5434972/ is an _excellent_ documentary about just this topic, science and all...and it's funny.

I recently realized the (obvious in hindsight) fact that general intelligence better that brute force doesn't exist, as intelligence is equivalent to compression.

Given the recent discoveries about neurons using mRNA capsids to communicate [0] it's not that farfetched to posit that we are really dna computers [1]. The processing time (for new problems) seems human-like: "The slow processing speed of a DNA-computer (the response time is measured in minutes, hours or days, rather than milliseconds)"

The evolutionary argument: as DNA computing is already used by microbes [2] how could the nervous system made of (relatively) dumb neurons compete with that? Synapses still make sense - as a way to request a rna packet and/or inform that it's coming and from where.

One neuron with capability of ~10M pattern matches per second (encoded in dna/rna) would mean that the human brain executes ~2^60 pattern-matching operations per second, utilizing zettabytes of imperfectly copied data. Enough to brute force its way through lots of problems.

Memory as dna would explain high-level memory quirks: each read would be destructive, by splitting dna into rna, interacting with other rna under the presence of appropriate enzymes, then copying and disseminating the resulting rna, transforming the memory each time it's retrieved.

It would also explain urban legends about people's personalities changing to resemble their organ donors in some way - as a donor's memory packets that somehow ended up on the donor's organ and, with the help of immunosuppressants, managed to infect the receiver's brain.

[0] https://www.nature.com/articles/d41586-018-00492-w

[1] https://en.wikipedia.org/wiki/DNA_computing

[2] https://en.wikipedia.org/wiki/Microbial_intelligence

> mRNA capsids to communicate

In this case "communicate" is in a narrow sense - this communication is very slow and Arc has more to do with plasticity rather than the electrical response of neurons.

> general intelligence better that brute force doesn't exist

What do you mean by this?

It sounds like you are assuming nature requires exactness and are really making a philosophical argument about the nature of computation, namely an np solution in p 'doesnt exist'

> Memory as dna would explain high-level memory quirks

If we are conjecturing than so too could approximate results 'explain high-level memory quirks'

we already have many approximate algorithms that are significantly better than brute force, and I would argue that any read procedure would necessarily be algorithmic, which then would require an explanation as to why this natural process failed to evolve over time

If such an explanation is simply, though arguably counterfactual, 'nature requires exactness' and so is unable to utilize the incremental improvement of evolving algorithms for approximate results, I would argue this implies p!=np because otherwise I think, if it were able, nature would tend toward improving on exactness over the 13B+ years it's been expressing mathematical truths

Being as my intended inference in regard this specific unsolved problem is to develop an algorithm to show p=np, I wonder if the process we refer to as conciousness may be such an algorithm

> neurons using mRNA capsids to communicate

I wonder if the rna is raw memory data or architectural plans for nuerons which, when constructed, express memory

>What do you mean by this?

The goal is to reduce n bits of data to x<n bits. Because there are less variables you gain predictive capability of n-x bits. Or stated differently, the goal is to get closer to the kolmogorov complexity of whatever you're trying to model.

Yet it's not possible to compress n bits in the general case. That's because the kolmogorov complexity is a function of your assumed knowledge (assumptions). All you can do is start checking every possible transformation from your assumption starting with the most probable one - the probabilities are based on your knowledge itself.

>we already have many approximate algorithms that are significantly better than brute force

Yes - but that means the algorithm itself, along with its execution, is the shortest (in the used metric, which can include execution time) answer for a particular problem. How do you generate the algorithm in the first place?

> Yet it's not possible to compress n bits in the general case.

Abstract bits sure, 2^4 objects are unable to represent 2^5 objects, simply due to 16!=32

But what does it have to do with'general intelligence' and brute force?

We were originally talking about rna communication.. where does kolmogrov come in? In the data representation in the rna? But what of it when the mechanism that encodes and decodes is unrestricted in its upper bound complexity? If the disparity between the upper bound of memory being encoded and the mechanism encoding it are great enough then that system could 'compress n bits in the general case'

> All you can do is start checking every possible transformation from your assumption starting with the most probable one - the probabilities are based on your knowledge itself.

This just sounds like you're saying every algorithm is brute force but with different possible states due to assumptions

Would you call Euclid's gcd 'brute force with assumptions'? I would argue algorithm is antonym to brute force

> How do you generate the algorithm in the first place?

Ah, I think I see what you're saying.. are you conjecturing the process of evolving conciousness was itself a brute force process?

Where understanding it's underlying process and being able to implement it ourselves, perhaps even more thermodynamically efficient, is inconsequential due to our efforts being only possible by the original conjectured brute force process that allowed us to abstract to such a degree..? This process being the 'assumption' to be appended to the proof?

But this again seems like a philosophical debate.. one of life and negative entropy

How do you define general intelligence? How do you defend the statement that 'intelligence is equivalent to compression.'?

If you would have to consider all of existence as assumption, then a bitwise representation of our own intelligence would be a significantly small subset of the bitwise representation of all things; expressing this would seem to imply some process substantially more efficient than brute forcing every possible state, or luck is real?, or we underestimate the complexity of 'general intelligence' and in actuality the search is ongoing? Or some undiscussed other?

Not sure how you are defining "general intelligence".

Why wouldn't AIXI count under your definition? (I recognize that AIXI can't be truly implemented, but I don't see where your explanation fits in a "unless you have a halting oracle")

The article seems to be about how it is that the brain can process so much information at so slow a cycle speed, but it doesn't really address power efficiency at all. Even assuming 100% parallel operation, our current chip designs use something like a factor of a million more energy to do the same amount of computation. I wish I knew why- it's not like we're ignoring power efficiency. Depolarization of an axon must be incredibly power efficient.

A litte online research reveals that the depolarization potential is on the order of one or two millivolts, whereas silicon has a bandgap of 700 millivolts. That's plausible; since power is proportional to voltage squared, a 500 fold voltage improvement would equate to a 250000 fold power improvement.

I think it's about context. If I say "order pizza" a ton of information is transferred via shared environmental context. I'm not transferring information directly but manipulating contextual levers to map certain types of information.

An example is if I say something to you in say japanese, I also need to teach you japanese in the same phrase unless there is shared context. computers run on a very impotent form of language, logic, that is rigorous and general purpose but has no "escape hatch". (basically the sentiment of Godem, Escher, Bach with regards to rigorous systems. )

Oh ok you're talking about data compression. That's also not what efficiency means, in a strict sense.

We don't know what operations the brain is actually doing, so declaring that it is performing them efficiently is pure speculation.

Similarly, the brain surely has limits to parallelism, for the same sorts of reasons a computer does. You only have one mouth, so if two parts of the brain tried to speak, fully parallelized, you'd get nonsense at best. They have to agree on what to say, which is effectively serialization.

The mouth as an example with regarding to speech doesn't work very well. People can say one thing, but say it in a way that means the complete opposite. So already we have tone incorporated into our speech. Who knows what kind of subconscious thoughts we embed into our speech that are not yet understood well.

Then there's the combination of tone and the words, accent, body language and every other context embed into our language that produces a very complex meaning, even for one word.

It doesn't matter what was factored into the speech, you still can't speak in parallel. It's not physically possible. At some point you need to get agreement as to what the output should be.

The brain definitely has an edge on compute power. The computer has an edge on storage power. I would bet the brain doesn’t store more than a Terabyte of information.

We just need more cores without spiking energy usage.

Interesting observation. I think most of us rely on some sort of world view or paradigm to operate. Probably as a result, the actual info stored is even less than a TB.

The human brain has over 100 trillion synapses, which is somewhat analogous to storage or memory (in both senses). Some estimates for the storage capacity of the brain are as low as a terabyte. Others are an order of magnitude or two higher.

Being electrochemical, the brain is “slow” compared to a computer (around 10hz). But it’s massively parallel and an interesting combination of digital and analogue.

Erm, and we just need the software to run it all :)

The computer also has an edge on precision and determinism.

Considering the kinds of experiments that have occurred in the past on electrical stimulation of the brain and memory recall, I would say that the amount of memory being used in the brain exceeds any limit we may well put on it by many, many orders of magnitude.

The major problem is that we are not trained today to recall most of what gets laid down. We are far to dependent of external storage mechanisms, books, videos, etc, that we no longer train ourselves to actually recall and pass on our knowledge.

I am not saying that external memory is bad, just that we should be using all options available to us.

Our understanding of the brain and its mechanisms is very much in the very early stages - we have just scratched the surface.

A brain is not equal to a computer or a processor.

You will never get the answer if you look at it from a narrow minded perspective.

A brain is equal to billions of processors connected in a highly naturally efficient network functioning almost effortlessly.

Something along these lines seems like a plausible comparison to a brain.

What is a brain is not equal to a processor but a neuron is equal to a processor?

How many milliseconds does it take for one layer of neurons to adjust its output? Let's call this number X.

Can we conclude that if a task takes a human Y milliseconds, then a neural network with Y/X layers is sufficient for that task?

AIUI/AIBT there aren't really discrete layers in brains and even the same type of neuron can act in very different ways, so I wouldn't be surprised if this question doesn't translate to brains.

A biological neuron is many magnitudes more complex than a typical neural network node.

The side-to-side comparison is not very accurate.

The brain consists of many structures, many of them involved in non-cognitive tasks.

The "speed of basic operation" column is given as "10 GHz" for a computer and 1 kHz for a human brain, but later is clarified that the 1 kHz refers to the maximum observed frequency of a neuron's output. Arguably these are unrelated metrics. I don't think a thing like "speed of basic operation" can even be defined for a human brain, because fundamentally it's a programmable computer metric that simply doesn't transfer to a brain.

Of course, one could go ahead and simply compare the conscious compute power of a human to that of a computer (e.g. FLOPS), but then that number would be something like "1/100 to 5 FLOPS".

>maximum observed frequency of a neuron's output.

IIRC the frequency of a neuron's output is used to encode the magnitude of the signal.

Take for example a neuron transmitting a pain signal, low frequency is used to transmit low levels of pain.

High frequency, extreme pain.

Another problem with the analogy in this article is that there is no division between memory and processing element in a nervous system. They're essentially the same.

Imagine running a program on a computer where everything is somehow memoized, on a system that is always powered on.

I wonder how this would compare against quantum computing, as that allows maximum parallelism too if in correct.

Quantum computing is not parallel computing: https://physics.stackexchange.com/a/3400 briefly talks about the mechanism of quantum computing and how it is not general parallelism.

Question: Has there been a comprehensive study on the activation function of a single (biological) neuron?

Yes, see the Hodgkin Huxley model [1].

[1] https://en.wikipedia.org/wiki/Hodgkin%E2%80%93Huxley_model

My brain isn't efficient. I forget basic things sometimes.

Human brain mechanism is a mystery

I feel like there must be some way to 'clean' our brains. As we get older it must accumulate 'gunk' that slow it down or weaken connections.

I guess sleep is the closest thing we have but to me that's like saying drink water to fix tooth decay.

We already do (in a multitude of ways) when we sleep. There's research that indicates that sleep is used to clear away neurotoxic waste.[0][1] The role of deposits of tau and beta-amyloid proteins in Alzheimer's[2] and aging are also of particular interest. Other papers supporting the synaptic homeostasis hypothesis suggest that sleep is when synaptic strength is renormalized.[3][4] Basically, memories are reinforced (the signal) in relation to existing knowledge while daily noise is effectively pruned (among other processes).

The analogy is flawed, but the brain already has some pretty intricate garbage control methods. As we learn more about them, it'll open up new paths for clinical treatments for a host of problems.

0. https://www.ncbi.nlm.nih.gov/pubmed/24136970

1. https://www.nih.gov/news-events/nih-research-matters/how-sle...

2. http://www.sciencemag.org/news/2018/01/alzheimer-s-protein-m...

3. http://science.sciencemag.org/content/355/6324/507

4. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3921176/

Drinking water (instead of sugary and/or acidic drinks) is actually a very good way to prevent tooth decay.

Even if you drink sugary/acidic drinks, water helps dilute them and rinse them away.

Taking your question conceptually, rather than physically, memories / experience would be 'gunk' if they're outdated (don't relate to how the world works anymore, or don't relate to how your body works anymore).

Your memories and experiences (potentially one and the same) colour your experience of the world and make you react in different ways, not always beneficial - but the phenomenon wouldn't exist if it wasn't historically, evolutionarily beneficial.

Like a quantum computer, computing thousand calculations at once and the true answer is what remains? I guess there's a ... parallel there. And I know some people don't like this diductic reduction, analogy, metaphor, callitwhatyouwant. I'd welcome corrections.

The problem with trying to correct this is that it's not wrong, because it doesn't really state a hypothesis.

Are you trying to say that brains are quantum computing devices? Maybe, but we don't have any particular evidence for that, so it's on a par with saying "The Flying Spaghetti Monster has a tiny noodly appendage that reaches through hyperspace into each neuron, and that's how we think."

We can observe brains in a lot of different ways - active electromagnetics, chemical sampling, microscopy, MRI... and we can say that injuring certain areas of the brain will impede certain functions. We know that excesses or droughts of some chemicals are associated with emotions, depression, and some diseases.

But we don't have a good model for how brains think, and among the good models we don't have, quantum computing is one of them.

I was being sarcastic, but then wasn't sure whether the comparison was actually wrong.

Reply on news.ycombinator.com