Hacker News Reader

Comment by iaw | Hacker News Reader

How far are we from fully modeling the human brain? I mean besides an easy way to identify all the neuronal connections...

This makes me feel like we're close to that one terrifying short-story.

View on HN

We are massively far away from modeling the human brain. First of all, no one can agree what level is necessary to model the brain, and that varies tremendously by scientific question. Personally, my lower limit would be something like the computational package Neuron which models voltages across axon compartments and distribution of ion channels, My upper limit confidence bound is we don’t care about anything subatomic.

At the upper bound: In molecular dynamics, which is used extensively in modern day neuroscience to understand the function of ion channels and GPCRs, a single H100 can model 70ns/day of compute for 1M atoms. There are 8.64e+13 nanoseconds per day. There are ~10^26 atoms in a human brain. Therefore, an upper limit back of envelope is you need fewer than 10e+26 atoms / 10e+9 atoms * 8.64e+13 ns / 70 ns = 1.23e+29 H100 GPUs.

Calculating the lower bound is more difficult, but let’s start by saying you can get away with a fp16 for each synapse. Storing the weights of that model for 100 trillion synapses is 200 Terabytes, and if you figure weight size * 4 or so to do anything useful then this is in spitting distance. Note that this example lower bound is massively less complex than the Neuron model I suggested, as the entire field of neuromodulators, homeostatic mechanisms, glia, and more are thrown out, which are all important for modeling how the brain works under certain computational regimes.

That is a spectacular response!

My bio knowledge is very basic, so forgive naiviety in these two questions.

First, I'm not asking you to go through the math on the spot, but I'm guessing that lower-bound capability is well understood in 'the field', but is it documented against various species? Perhaps mapping against current / projected GPU/compute systems capabilities? (I know there's a project to model a worm's brain, IIRC down to molecular level. But I'm picturing a 'we are 3 years away from being able to emulate a basset hound, 4 years for a border collie' - that kind of roadmap.)

Second, you said upper bound is to ignore sub-atomic. I thought we had proton and electron gradients, at least in metabolism. I believe proton there is a synonym for Hydrogen (atom), but electron would imply some potential need to emulate at sub-atomic? Have I misunderstood the bounding / chemistry involved?

The electron and proton are considered parts of the realm in atomic physics still. When physicists speak of sub-atomic they tend to mean any physics below the Aufbau model or electrons and nuclei.

For the lower bound there is a dark horse factor that has spooked Geoffrey Hinton. He thinks that biological brains aren't able to do backpropagation effectively through multiple layers, and so differentiable programming frameworks are much more powerful than what the brain has, at an algorithmic level. In other words, he thinks that computers are able to learn more effectively than any neuron-based biological brain. Of course right now there are caveats. The brain appears to have more 'statistical efficiency' meaning it appears to learn more from less data, and the brain is obviously more energy-efficient. There is also the possibility that Geoffrey Hinton is just wrong.

Biological brains also don't really operate layer by layer and can have connections between random neurons, so it's probably a lot more space efficient. Impossible to say if any of that actually matters though.

Can you cite a reference for Geoffrey Hinton (and or others) on this line of thought on this "dark horse factor"? I think this resonates with a lot of people, having a focal point Schelling point to refer to in discussions would be handy.

I believe the situation is a lot more extreme than the absurd efficiency of differentiable programming. I have been meaning to write up (but been too busy to do so) an insight where I believe training can be made ridiculously cheap computationally speaking (in a way that combines with differentiable programming, not replaces it). I am agnostic if this is what the brain does, but wouldn't be surprised at all if the brain does in fact do back-propagation (or uses the insight that I've been meaning to write up).

We will never simulate the entire brain atom-by-atom, we won't need to, the same way we never simulate atom-by-atom and we don't even place structural atoms by hand when we build a bridge, a rocket, or a tree house, we can be way more intelligent than that [1]. In the limit, the entire thing could be even more simple than we currently can imagine [2]. But yes, before we start leveraging equations, we must find the principle of gravitation for collective intelligence first [3].

[1] https://en.wikipedia.org/wiki/Hodgkin%E2%80%93Huxley_model

[2] https://en.wikipedia.org/wiki/Reaction%E2%80%93diffusion_sys...

[3] Michael Levin | Cell Intelligence in Physiological and Morphological Spaces, https://www.youtube.com/watch?v=jLiHLDrOTW8

I think a long ways away, if I understand this article about the difficulties involved in accurately modeling even a single biological neuron:

https://www.quantamagazine.org/how-computationally-complex-i...

> "If each biological neuron is like a five-layer artificial neural network, then perhaps an image classification network with 50 layers is equivalent to 10 real neurons in a biological network."

The complexity explodes quickly because each biological neuron's behavior is modulated by a large number of biochemical neurotransmitters, on top of all the dendritic connections (up to 15,000 each, apparently).

When AI becomes good enough we will maybe we will stop thinking about trying to imitate human brains. If we viewed our brain's decision making power objectively we can find several flaws, for example our heuristics to make quick decisions for mundane things is also our greatest weakness(short sightedness). We are poor at incorporating data to make good decisions and constantly bias due to some external stimuli.

Why would you want to make anything close to the brain? What real scientific or engineering or humanitarian uses does doing that even have? AI is already and going forward should strive to be a groundup of redesign of intelligence.

> Why would you want to make anything close to the brain? What real scientific or engineering or humanitarian uses does doing that even have?

To have models of the human brain that we can poke at and change and tinker with and etc., so that we can get better ideas of how therapy techniques, medications, ... will impact the actual real people that might benefit from them.

You would need to define what you mean by 'modeling the human brain'. If it means AGI or anything similar, then we're very far.

To paraphrase an analogy I've heard somewhere (in similar context) - We're building better and better ladders, maybe even lifts with this last push in ML field. But the brain is on the moon - even the best lifts won't get us there.

Sorry if I'm missing an obvious reference, but what short story do you mean?

(edit: thanks for both replies already, and any others that might fit; I understand now the reference was likely to Asimov)

https://users.ece.cmu.edu/~gamvrosi/thelastq.html

https://qntm.org/mmacevedo

Wouldn't modelling the human brain mean we'd be using less power? We're using brute force to try get similar results to what the brain does.

The question that interests me is how far are we from modelling a human cell, neuron or not? Because that's how we cure cancer.

Is human consciousness just a product of neural networks, or is there other additional mechanisms we're not aware of?

The brain is analog and chemical, AI will be digital and silicon. We have no idea how the map from one to the other.

Is the Quantum computer hypothesis dead?

I don't see how quantum computers are relevant? We can't build them, and there certainly isn't any interesting quantum computation in the brain.

What do you mean we can’t build them?

To your second point, we have little to no ability to understand yet what quantum effects may or may not be active in brain/consciousness function. We certainly can’t exclude the possibility.

I mean lots of people have tried to build quantum computers, and so far no-one has succeeded in anything that's describable as a "computer", instead of "half a dozen logic gates". Perhaps in the future.

We can fairly well exclude the possibility of interesting quantum effects in human consciousness, because the human brain is a hot, dense environment that might as well have been literally designed to eliminate the possibility. It's the exact opposite of how you want a quantum computer to be built.

Which doesn't mean there aren't plenty of quantum effects involved in the molecular physics, but that isn't what is normally meant by 'quantum computer'. Transistors would also meet that definition.

I do not agree with either of your assertions:

1) That 433 qubits does not make a computer and is instead “a half dozen logic gates.” I agree a half dozen logic gates is not a computer. 433 qubits is not comparable in terms of information capacity or processing capacity to a half dozen logic gates. This number is also publicly doubling annually now — I would bet the systems we don’t know about are more complex. Importantly, a computer in this context is not something you would attach a monitor to — it is just an electronic device for storing and processing data.

2) That we have any good idea of the limits of how biological systems might be influenced by quantum effects within specific temperature ranges. You certainly wouldn’t construct a human brain to interface with quantum effects given the present state of our knowledge in constructing these kinds of systems. But then we can’t even construct a self-replicating cell yet, nevermind a brain. It’s hard to imagine we understand the limits at work there.

> The brain is analog and chemical, AI will be digital and silicon.

Says who?

Sure, if you assume that “AGI is just scaling up GPT”, it will be digital and silicon. But that’s a big assumption.

For all we know, AGI will only ever, if it exists, be analog and chemical.

> We have no idea how the map from one to the other.

Plus, even if we had an easy one-to-one mapping function between them, we don’t understand the source well enough to do the mapping.

It does not need to be, but today the computers we use are overwhelmingly based on silicon. Also OP mentioned AI, not AGI.

An analog-to-digital and digital-to-analog converter is less than a cup of coffee in some places [1].

[1] https://protosupplies.com/product/pcf8591-a-d-and-d-a-conver...

Oh god so my background is CE/ECE stuff and you managed to trigger me. I don't want to be rude... just bluntly saying you triggered me. Doing something really small for A/D D/A with 8bit and not worrying much about resolution and data loss is one thing. For something massive scale the problem is a lot less trivial and a lot more mathematical.

Haha, sorry, was more of a tongue-in-cheek reply to "We have no idea how [to] map from [analog] to [digital]".

For many applications there is no need to fully model the human brain. An approximation of a particular aspect would be sufficient in most cases. We didn’t build aeroplanes by fully modeling a bird, we just need aerodynamics.

Reply on news.ycombinator.com