Original author of the paper here, happy to answer questions. It's a little strange refreshing HN and discovering new popular press about your paper.
It's as silly as expecting to get a good theory of artificial intelligence by studying artificial neural networks.
(I'm aware of the irony in the above statement, but stand by it earnestly. An excellent engineering artifact whose functioning we can't explain is not scientific understanding. An excellent engineering artifact whose functioning we refuse to explain is bad philosophy, too.)
Also, there exist brain areas and regions where we do in fact have a few good good models, and connectomics has the potential to help us resolve them -- see http://www.nature.com/nature/journal/v500/n7461/full/nature1...
Turns out, even where they should connections don't constrain circuits to a sufficient degree.
I guess the key lesson is -- don't rely on a single approach, because its limitations may well lead you astray. Applies to connectomics, physiology, modelling, etc.
Everyone who's paying attention understands that individual neurons have the potential for very complex, but precise, behavior.
But knowing the connections across the brain (at some resolution) is helpful for plenty of reasons. If I want to understand how areas in the brain communicate, it's immensely useful to know where they're connected, for instance. Let's say I have 200 sensors I can place in the brain wherever I want. Placing them at crucial nodes or connected areas could be tremendously useful, since we can't yet put sensors everywhere for most spatial and temporal resolutions we want.
It looks like many analyses were performed by binning transistor switching as you might spikes from neurons, or by linearly combining transistor activity across transistors or time.
This is slightly puzzling to me, because transistors would appear to not work on the basis of average activity at their level of computational composition. I would be surprised if you could really understand circuit activity at a computational level with most of the measurements you describe.
On the other hand, I suspect the methods used work slightly better in a real brain. Of course, the computational purpose of many neurons is difficult to parse or even construct useful hypotheses about. But at the very least, neurons in the outer periphery seem to behave in a way that is at least consistent with firing rate, e.g. impulses at the neuromuscular junction to cause muscle contraction, and retinal ganglion cells to fire at higher rates when the input matches their certain luminance features.
For the next paper, though, we're taking a more careful look at a lot of these encoding questions, as well as trying to use processors with a greater degree of functional specialization so we can more accurately capture what's going on.
For the processor I can see the properties of individual transistors being (near) constant, but do you have a reason to think so for neurons in the brain as well?
It is as though someone declared, "Powered heavier-than-air flight will never be possible." Uh, of course they will? It's totally obvious?
Like how can any reasonable or even poorly-informed, uninvolved person who is in a hurry, possibly not see how completely obvious this is? Help me understand this. Never is a long time.
-
EDIT: case in point, for some inexplicable reason this comment, a direct question to you stochastician, in the above form (I didn't edit except to add [serious] and the last sentence) was just downvoted twice. (I take it by people who consider everything I just said to be BS.) Just to be clear none of the above is sarcastic, etc. To me - and I suppose you - it's pretty clear what will eventually be happen, and "never" is a strong claim. I completely fail to see how anyone can claim "never" when we're ahead in terms of hardware, today.
Contrast AI with, say, interplanetary space travel. We're a long way away from that too, but there's a more obvious path to getting there. We need better control of nuclear energy, more robust robotics designs, and more efficient environmental control. These are engineering challenges: for the most part, the science is already laid out.
With AI the science isn't there. We don't know which direction to move in, let alone how to move in that direction.
"If you had a near perfect hydrogen -> helium fusion engine, it'd take about 6 million tons of fuel (about the mass of the Pyramids of Egypt or 2,000 Saturn V rockets)".
Does this sound like a roadmap? "If you had a near perfect hydrogen-> helium fusion engine"... "mass of the Pyramids of Egypt"?
Contrast this with the hardware I'm talking about: by my estimation there are 1,000 clusters today, today, that can run the appropriate software if we had it. The idea that someone would call it "impossible" at a time when conventional human thinking challenges such as playing Go, are being mastered by machine using machine learning, seems crazy.
I absolutely do not agree with you that we have any roadmap for interplanetary, certainly extrasolar, travel.
Contrast this with the Human brain. We can count the neurons (100 billion or so). It uses as much energy as a 20 watt lightbulb[1] but doesn't have light-speed interconnects, unlike server rooms. The roadmap we are following, simulating neural nets on bigger and bigger hardware, is pretty much the exact thing you'd expect us to be doing. It's yielding amazing results in machine learning.
Why would anyone think that the biological, slow-firing chemical analog process these huge, super-fast digital processes replicate, will be out of reach literally "forever".
It is literally possible that not one human being will ever travel outside the solar system - not in the next 1,000 years; 5,000 years; or ever.
The idea that we will "never" simulate a hundred billion analog neurons running at 20 watts seems childishly naive. It seems like saying "no machine will ever fly - period" when machines are already doing more than enough work to do so. Come on.
Anyway I am really interested in stochastician's answer here - this is why I wrote them after they offered to answer any questions ("appy to answer questions.").
[1] 20 watts
Supercomputing instances built w/ EC2 are in the 0.25Petaflop range. Assuming that's only a fraction of their machines, maybe the major cloud providers play on the same scale - and at this point, you have to deal with communication across DCs, which might make realtime simulation utterly infeasible. That's 2 yes, and 3-5 maybes, not "over 1000 organizations".
Then you're looking at the fact that recent research indicates dendrites carry processing power as well, which would move any brain simulation well into the exascale range. And Moore's law is shakey by now.
But given all that is overcome - we seem to be doing OK at the scaling up thing - you would need a complete connectome of the brain. So far, we've succeeded doing that for roundworms. (And 6502's, as the article shows :)
The human brain is a far cry off from that.
But let's be optimists, and stipulate that is at some point done as well. We're still in the dark, because we have no idea if a fully connected brain will work without being in a specific starting state. We're nowhere even close to map the state of a brain. And at this point, we're definitely in the realm of questions where we don't even know yet if we will find an answer at all.
Want more curveballs? There's a non-zero probability that the brain can only be fully explained using QM explanations. You've just significantly shifted the scale of the simulation.
I don't know NEVER is an appropriate word, but it's not totally obvious, either. Using the word "obvious" in this context (just like the word "never") means you've reduced the problem to pop-science, where everything is either black or white.
Where are you getting this? Did you account for the fact that the average cortical neuron fires 0.16 times per second? [1] Put in terms of gigaherz, that is -- haha just kidding about gigaherz, that is 0.16 herz per neuron. Times 100 billion neurons. "Combined with the observation that 90% of neurons rarely fire" so you actually might not need to touch all 100 billion of them in a cycle.
100 billion * 0.16 gets you literally 16 gigahertz. This is four cores running at 4 gigaherz. Memory can be heavily compressed if necessary, without slipping out of the realtime budget.
People don't seem to get that these are ANALOG SIGNALS. These neurons aren't etched in silicon. Simulating that is like simulating a mechanical watch or something - with 100 billion pieces. Your guess is ridiculously too-high.
Do you know how long it takes for a signal to cross from one half of the brain to the other? 100 m/s is widely reported progation speed limit, and the average brain length is 167 mm[2] . That means you get 167 mm / 10000 mm = 1.67 milliseconds. That's long enough to cross a server room approximately 25,000 times.
(I divided 20meters by C, getting 66 ns - the time at speed of light to cross a 20 meter room, and this value goes into 1.67 ms 25,032 times.)
All of my numbers here are super-conservative, above-all the fact that I said only 1,000 server rooms were able to do this today. It's considerably higher.
We're ahead of the computational budget required, not behind it. I simply disagree with your calculation.
The rest of this paragraph is sarcastic: When mentioning that there's a non-zero chance that neurons use quantum mechanical effects (the way our etched silicon does, since gates are so small now that this has to be accounted for), you forget to also mention that there's a non-zero chance that souls are connected to brains using an as-yet unidentified organ and therefore nothing in the brain can be simulated given any amount of computational power. I mean, the chances of that aren't zero, right?
[1] "Based on the energy budget of the brain, it appears that the average cortical neuron fires around 0.16 times per second" - http://aiimpacts.org/rate-of-neuron-firing/
[2] https://www.disabled-world.com/artman/publish/brain-facts.sh...
Second, the complexity does not lie in the neurons, there one gets away with something like 100 billion neurons * 100 Flops/ GHz (Prozessor) ~ 100 Prozessors per dimension if we completely neglect the gradient.
But what we need is the dynamics of the connections. And based on that, the estimate of 100 PFlops simulation gives roughly 10^5 clock cycles per connection, which is not very much if you compare it to the latency of a standard database in the microsecond range. (Network latencies are in a similar order of magnitude.)
About your argument, that it would be enough to only simulate firing neurons, well we have to simulate a neuron to detect if it is firing.
For completeness, Kurzweil mentions in The Singularity is near much lower estimates for functional simulation of a brain in the TFLOPS range, however that is with optimized algorithms and we probably need a singularity first to get these optimized algorithms.
This page I linked before , https://www.disabled-world.com/artman/publish/brain-facts.sh... , says "The brain has a processing capacity of 0.1 quadrillion instructions per second" (though that phrasing is bizarre), saying that the cluster Roadrunner at 1.06 petaflops did 10x more computation.
Roadrunner was built in 2008.
1) Neurons are independent spatially and temporally.
2) Dendrites are not computationally active, again false.
3) State is not important.
4) Neurons are used to simulate a binary function and each only one.
For me the most authoritative answer is what the Human Brain Project in europe is able to achieve: http://www.nature.com/news/fragment-of-rat-brain-simulated-i...
31k cells with 37M synapses. That's a start, but pretty far from a mouse brain (100M cells) or a human brain (80B? ).
Also if Moore's law really is dead, this all gets much more depressing.
You mention membrane dynamics, past spiking history, etc - but neural nets accomplish a lot without these things.
A simplification surely works. You are not thinking about simulating a brain using a large cluster with a lot of back and forth.
Why not? Don't you get how incredibly slow 100 m/s is? This is 7 orders of magnitude difference in speed. Silicon is seven orders of magnitude faster at signal propagation than the brain, which isn't electrical.
To you this doesn't unlock vast possibilities for simulating the analog in real time? It just doesn't seem that permanently impossible, given the results you are already claiming are occurring.
Also although I directly addressed you, I guess you don't have any answer as to why someone would choose a target date like "never" or "300 years" as opposed to some reasonable extrapolation of the results being achieved today.