Sequoia: 20 Petaflops, 1.6 million cores, 1.6 Petabytes RAM
hpcinfo.wordpress.com
hpcinfo.wordpress.com
My own personal interest is protein folding.
Proteins are folding right now, as you read this, in every cell of your body, and they're doing it in linear time.
But computationally we're stuck with exponential time.
The day we find an algorithm for linear time protein folding, is the day the world changes at least as much as when penicillin was discovered.
> But computationally we're stuck with exponential time.
It's not that bad and it "recently" got a lot better.
Search for "shaw protein folding neutral territory".
That may mean that proteins quickly collapse the probability space or have some other "trick" by which they are not NP-complete, or whatever, it is not magic.
There is a way and I'm hopeful we can find it.
http://www.princeton.edu/~rajchak/
the other day and he was talking about using optimal control theory to optimize protein sequences for specific enzyme activity. He had a very sophisticated model for how the amino acid sequence determines the shape of the protein's active area. It was way over my head but he treated it as a solved problem. You may find his papers interesting.
The combinatorial search results on protein folding are theoretically interesting, but I don't think that they are show stoppers for finding, or optimising the folded configuration of a protein. As you mentioned, most of the degrees of freedom which lead to the exponential size of the search space are irrelevant. A detailed understanding of the physics of how amino acids interact in water with ions can allow one to make approximations that vastly reduce the size of the search space.
Plus how much time does it take a protein to find it's native conformation? Let's say the folding time is 10 ms. I think ultrafast pump probe spectroscopy has shown that alpha helices and other secondary structural motifs form on the time scale of 100 ps. Then the protein explores about 10^8 configurations before it finds the native one.
Sequence optimization and designability of enzyme active sites - PNAS v. 102 (2005)
Full text is here:
http://www.pnas.org/content/102/34/12035.full
""" Since it was first observed that all known natural proteins adopt one of only ≈1,000 distinct protein folds (1) [emphasis mine], researchers working in the field of computational protein design have endeavored to understand the basis of this biophysical phenomenon and apply the resultant knowledge to the construction of unnatural proteins. Motivated by the observation that certain folds are associated with more natural sequences than others, the principle of fold designability (the number of unique sequences compatible with the backbone structure of the fold) was introduced (1). To place the problem on a mathematical foundation, an analogy has been drawn between sequence space and phase space in statistical physics, where the backbone structure represents a macroscopic state of the system and distinct sequences are microscopic states whose occupation of the macroscopic state is associated with an energy, the folding free energy of the protein. In this language, a fold's designability is correlated with its sequence entropy (2). """
To use the TSP as an analogy, think of a protein as a configuration of cities. The proteins in our body are the solutions of the easy configurations, i.e. a group of cities arranged in a circle. Nature hasn't been able to find the really hard solutions.
We would like to explore the full configuration space, to look for new proteins, and that's still hard. Or maybe I misunderstand.
That said, we need better physics to represent these interactions. The fields been relatively static when it comes to forcefields that really work.
If someone is excited about that, what about the human brain itself? Just a simple information:
The human brain is the most complex thing in the whole universe, or: all of physical universe is less complex than a single human brain.
(since "all of physical universe is less complex than a single human brain" and the physical universe contains around 6.7 billion brains (that we know of))
If true, then it has an undefined and immeasurable complexity. You can't make all these statements about it, you can't measure it, and you have no context to even begin to talk about it. You can't even make assertions about what the measurement of complexity itself would be ("infinitely more" is just as nonsensical as a finite number more because we do not know what the units of a complexity measurement would be -- if "units" even makes sense in the "beyond").
It's not me saying this, but science.
You keep using that word. I don't think it means what you think it means.
But neurons aren't necessarily the optimal substrate for "neural" computation. They have a cycle time of about 100 Hz. Electronics are potentially a factor of 10^8 faster. Neurons also have a lot of noise which surely reduces their information transmission capacity. Also there are major constraints on the neural tissue involving power consumption and stability against siezures. Also there are vast sections of the brain e.g. the cerebellum which are apparently devoted to problems like stabilizing posture and making smooth movements which can be solved much easier with electronic controls.
It's very plausible that we could achieve a factor of 100 or even 1000 over the brain. The bottleneck is human imagination; we don't have a clue how the thing works.
The human brain is quite "infinitely" interconnected, and this is something which not only would be really, really hard to solve with a physical machine: it's simply impossible.
I understand the fascination of a real super computer, but it's simply nothing compared to a single human brain...
Hmm. How does this "infinitely interconnected" thing work if not by physical means? (In physical, I include electromagnetic waves without a dedicated conductor, not just "wires".)
The interesting thing is: some do, some do not.
(We have physics, chemistry, and others, exactly to distinguish different levels...)
If some of the communications are not by physical means, how does the "not physical means" communication work?
If all of the communications are by physical means, how does the brain have "infinite" interconnect? Am I wrong in believing that one can't have infinite connectivity with physical means?
You say that the human brain would (in theory) be able to create something more intelligent than itself?
(Are you aware of the irony?)
If you follow evolution, that's how we came into being in the first place. Though there was no specific entity working with that explicit goal in the case of evolution.
Are you aware of the empty rhetoric?
However, I'm very close to certain that your post is basically a lot of hand-waving that is ultimately not only poorly reasoned but also actually incorrect.
- - -
The question we're trying to answer is basically, "What's the computational capacity of the brain?"
But that's not really a good question to work with, because "computational capacity" isn't well-defined. [Actually, I'm not sure about that. I know that either it's not well-defined, or its definition is something that's clearly not what we want in this situation.]
So let's ask something that we can actually reason about, like, "Assuming infinite memory, roughly how many floating point operations per second would a computer need to be able to execute in order to direct a human body in a way that's indistinguishable from what a real person does -- in real time?"
And, okay, we really only want the brain's conscious thought and maybe some of its unconscious thought, but it's hard to make a well-defined question involving just those things, and I doubt that the answer to our question will be more than a few orders of magnitude off of what we're really after.
- - -
Anyway, your post does a really, really bad job of answering that question (or anything like it). Your worst offense: > "So the human brain has about 10^11 neurons and about 10^4 synapses per neuron. Naively representing the state of each synapse with one byte requires 10^15 bytes that's about 500,000 PCs."
Come on. Did the fact that you had to use the word "byte" not tip you off? YOU'RE DISCUSSING MEMORY. Nobody cares about memory, because the amount of memory needed to store the brain's state is vastly less than the number of operations needed to simulate its function in real time.
I seriously am not able to articulate how frustrated I am that at least 12 people felt justified in upvoting you without taking the time to process the first two sentences of your post.
Before I go, another treat: > "Electronics are potentially a factor of 10^8 faster."
No. Electronics are a factor of 10^8 "faster", when "faster" is taken to mean "more rapid in executing a cycle." Which is an absolutely pointless irrelevent piece of shit statistic.
What the fuck.
- - -
Anyway, let me wrap up this post by actually answering the above question as best we can, given the current state of scientific understanding (or, more precisely, my understanding of that state of understanding):
There are on the order of 10^10 neurons and on the order of 10^11 glial cells in the adult human brain. These cells are directed by untold numbers of chemoattractants. We have no good theoretical model for the behavior of these cells in huge numbers of situations.
Therefore, WE HAVE ABSOLUTELY NO FUCKING IDEA how much processing power it would take to simulate the brain's macro function. We can put an upper bound on it: we know that if we could simulate every atom in the brain to a reasonable degree of accuracy, we'd get human behavior. Obviously the number of FLOPS it would take to do such a thing is unreachable. And beyond that, I don't think it's fair to say we know anything at all.
I don't think I can really comprehend what any of your objections are, but I can certainly think of some major flaws in the toy argument I gave, and maybe these are what you were getting at even if you couldn't really articulate them.
first, just saying the state of a brain is the state of each synapse assumes a fixed topology of the brain. This is obviously false - there is a lot of evidence that the topology of the brain is changed by experience e.g. the occular dominance columns. Okay, so how much extra space would we need to deal with this - not that much actually: how much space do we need to store an arbitrary weighted directed graph with 10^11 nodes and 10^15 edges? Well for each edge we need to know the two vertices and the weight. Give each neuron a unique id and this requires about 10-12 bytes per synapse instead of 1. This is only one order of magnitude difference between a fixed and arbitrary topology! And this is probably a gross overestimate of the storage space required because the topology of the nervous system is far from arbitrary - after all the thing is generated by a genetic code of very modest size. See my posts on why the large search space doesn't kill attempts to solve protein folding.
Obviously I ignore the functioning of the glia - I know for a fact that these are very important for the growth and development of the brain but I'm not sure they are the critical part of its computational function. If you know otherwise please point me to some references, I would love to know!
The reason that I mentioned the speed difference of electronics and neurons is that this is what makes it possible to update the state of those 10^15 synapses in real time. Of course it would be wasteful to use the standard PC architechture for this process, but still.
Hope this helps clear things up.