How Computationally Complex Is a Single Neuron?
quantamagazine.org
quantamagazine.org
First of all, they didn't try to model the behaviour of a biological neuron, only of a model of a biological neuron, and an incomplete model at that. The article points that out itself.
More importantly, the claim rests on the assumption that a deep neural net architecture is somehow a sensible measure of the complexity of a completely unrelated process (as far as I can tell, deep neural nets are not in anyway related to simulated rat neurons). But this is a big, huge, gigantic assumption.
Suppose I train a bunch of humans to predict the behaviour of a simulated rat neuron. Suppose I find out that I need to train at least 1000 humans to accurately predict the simulated rat neuron's behaviour. What does that mean? That 1000 human brains are as complex as a simulated rat neuron?
Come to that- can a single human brain predict the behaviour of a simulated rat neuron? I'm prepared to wager that, no, it can't, because human brains are rubbish at prediction tasks like that. What does that mean? That a simulated rat neuron is more complex than a human with an entire human brain?
The entire premise of the work is absurd nonsense and it probably only got a mention in Quanta because it's something released by DeepMind. Irritating.
I mean, all the ways that even a micro transistor interacts with voltages and heat make it potentially more complex than an AND/OR gate. But that's often what a given transistor is used for.
I'm not saying all the complexity of a neuron doesn't matter or that neuron isn't much more complex than a physical chip piece - just that treating parts in isolation, if you don't understand the system, can very easily result in wrong conclusions.
EDIT: I was completely wrong about the source and application. Can't believe I was able to find it, because I was way off. This was it: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50....
Other reading indicates that this technique has been successfully applied in electronics design in the time since 1996. https://en.m.wikipedia.org/wiki/Evolvable_hardware
The NASA thing I was thinking of was this: https://en.m.wikipedia.org/wiki/Evolved_antenna
You have seen today's xkcd, right?
(2510 - for you people from the future...)
This is exactly where complexity hides. Simplicity of models relies on abstractions, which in the real world are invariably leaky. The complexity of making a robust NAND gate is very much observable at some level, and only goes away once you ignore the messy details. The more we look, the more this seems to hold for pretty much everything in our observable universe, from galaxies to quarks. The more you dig the more worms you find. There are thousands of sub-fields of molecular biology which try to understand how a single cell actually works, and we still are not done by a large margin. Of course we will always ignore what we can to make workable human models that we can actually reason about.
The parent comment about varying transistor combinations was not quote correct in my opinion, as these variations in receptor makeups DO change how the neuron and circuits respond to stimuli.
Yet you mention observable reasons at the beginning, before abstracting it right past spherical cows on friction less planes to its purely mathematical concept.
Especially with attacks like row hammer one could argue that redundancy or the lack thereof has a significant observable impact on how modern systems behave.
*translated
it was fascinating. :)
one of my favorite talks at a really fun conference.
- The goal of artificial neural networks is not generally to simulate with 100% accuracy the firing of real neurons, but to perform their function. The results from ANN applications so far show impressive abilities that match equivalent (but specific) functions of hundreds of thousands of real brain neurons
However, if we are modeling particular functions e.g. motor control of the limbs, we can black box a lot of that complexity.
There is no question that a neuron uses a large fraction of its atoms for its functionality, as is every other cell. What I think you're postulating is that these atoms form a functionally uber-complex network with mind-boggling combinatorics - but that position does not align well with current science. Single neurons are not that smart. For starters, they don't have the I/O address space or bandwidth to be, nor do they have the energy budget to behave like a block of Computronium.
Neurons have not been opaque black boxes to us for a long time now. We understand a lot of the biochemical processes going on in there. We may not have a complete picture in many areas regarding many neuron types, but there is not enough unexplored space in there to allow for a cell-sized quantum supercomputer or anything like that.
The upper bound on a neuron's computational complexity is dictated by the intricacies of its protein machinery, which is many many orders of magnitude lower than the number of atoms making up the whole.
So given ~20 types of receptors, ~8 response types ranging from inverse agonist to full agonist, and conservatively, 20 relevant concentrations per ligand, we have over 3000 states dictated both by macro and local conditions.
But I'm just not convinced chemistry is fast enough for thought to happen any other way than quantum.
Life has been far ahead of human state of the art for a long time.
Often, we don't even know what to look for until we learn basic physics ourselves.
For example, barely 100 years ago, we figured out that light is energy delivered in photon packets. Only then could we understand processes like photosynthesis. Something life figured out billions of years ago.
As we learn more about quantum mechanics, we discover that photosynthesis seems to harness quantum effects to improve conversion efficiency.
I'm convinced the brain is using advanced physics we don't fully understand yet.
Yet, that the brain and neurons are quantum is something I haven't seen much of in literature.
Once we figure out room temperature quantum computing, I wouldn't be surprised if we find that the brain has been doing it all along.
We just don't know what to look for, until we learn enough about physics ourselves.
Disclaimer: I believe there is an intelligent creator. (https://www.jw.org/en/bible-teachings/science/)
[0] https://socratic.org/questions/calculate-the-number-of-colli...
How fast do you think? I know my reaction time to unexpected external stimulus is up in the 200ms+ range, and sometimes even basic things can take multiple seconds to think through with my conscious mind. This is an enormous amount of time chemically speaking.
If you are proposing that quantum processes aren't really random, that somehow human thought affects how they work, I doubt we have any evidence of that one way or the other. I would call that a fringe theory, but I can't say you are wrong.
I've recently being seeing more about a theory that consciousness is inherent in matter, and that particles choose which quantum path they take. This is way beyond anything we currently know about physics though.
Your analogy is like saying my computer has X number of atoms, so they might all be used for computation! Instead, we know that, for example, the case is not used for computation. The power supply is not used for computation, etc.
Mathematically, over some finite interval, approximating y=sin(x) using a Taylor series, or approximating y=x**n using a Fourier series, can both require a large number of terms. But this doesn't imply that a sine/cosine function is more computationally complex than a monomial, or vice versa.
In the opposite direction, assuming we had manipulation techniques create new neurons, position them and stretch out axons arbitrarily and control synapse linking strength (which we can't now), then IMHO there could be a 1-to-1 correspondence; the same octopus neurons that McCulloch-Pitts model tried to describe could also each approximate one artificial neuron for inference. Of course, learning is a different issue, brains don't exactly do back-propagation and gradient descent.
You only need to put all your synapses at the soma and voila
1) The computations we want to do may still be expressed better without consciousness. Think Chinese Room thought experiment, or philosophical zombies. In other words, we may be able to make something that can understand and execute complex commands, perceive and react to its environment, and communicate information — without having feelings, desires, or consciousness per se.
2) I'm less worried about creating new things to be unethical to than I am about the dis-ethical treatment of currently sentient beings.
> 2) I'm less worried about creating new things to be unethical to than I am about the dis-ethical treatment of currently sentient beings.
Both of these relate to one of my major concerns:
We don’t yet have a testable definition of “feelings, desires, or consciousness” [0] such that we could even tell if an AI did or did not have this characteristic.
[0] and any of the other various words frequently thrown around as attempts to create such a definition. Every time I make this claim I get responses along the lines of “yes we do: $foo!”, but in my experience so far, all of these turn out to be one of (1) circular definitions, or (2) things humans don’t have, or (3) things which plants and the enemy characters in Wolfenstein 3D do have.
I'd be curious to read more about these 3 arguments and their flaws.
I don't think that's a circular definition. I think humans have it. I don't think plants or video game characters have it.
However: it is not testable, and it's not obvious that it could be. If someone claims to have subjective experiences, I don't think there's any way to know whether they're conscious or whether they're just an automata acting conscious.
If consciousness is just being able to experience experiences, then that's a function that doesn't have any outputs, how could you possibly test for it?
And this applies equally to humans, c.f. solipsism. It could also apply to all matter in the universe, it's just that most of that matter has no computational capability, and no way to express itself even if it did.
> However: it is not testable, and it's not obvious that it could be.
If it isn’t testable, do you really have a definition rather than a tautology? What does it mean to have “subjective experience”? One instance of a game AI is subject to different inputs than another instance of the same AI, which you reject (as would I: I listed it as an example of how bad the definitions have been and definitely not as a claim they’re sentient).
For example, you could ask me any question about the contents of my mind. Let's say you ask me if my favourite colour is green, and I say it is. You don't know whether my favourite colour really is green or if I'm lying. The fact that you have no way to ascertain my true favourite colour doesn't mean favourite colours are ill-defined! You just can't know what anyone else's favourite colour is.
(Not that I'm claiming my "subjective experience" definition is well-defined -- I accept that it's pretty vague as definitions go).
I assume you are also a vegan? I would put my energy into fighting animal meat and animal husbandry today than worrying about a level of technology we probably won't reach for at least a few decades if not a few centuries.
You can see a scan of an ad for the system here: https://bit.ly/3gYXs3v
50TB disk
128GB RAM
Not too bad for 1984, Castle Wolfenstein was probably running reaaallly smooth.
[1] https://www.biorxiv.org/content/10.1101/613141v2.full.pdf
> We next applied our paradigm to a morphologically and electrically complex detailed biophysical compartmental model of a 3D reconstructed layer 5 cortical pyramidal cell (L5PC) from rat somatosensory cortex (Fig. 2A). The model is equipped with complex nonlinear membrane properties, a somatic spike generation mechanism and an excitable apical nexus capable of generating calcium spikes
Mistaken idea: genes are like a program in code that defines what biology does.
More accurate idea: genes are like the NVRAM of a running program that has run continuously with in-place updates for 4 billion years.
They also represent the NVRAM of the compiler that builds both the program and the compiler itself :)
It is a process somewhat like studying a CPU by shocking prongs and occasionally slicing it real thin to see inside it.
(my interpretation of the comment) For example, if there are like 8 free variables on neuron formation from the genetic blueprints, and then we find they produce reasonably different neuron types about 16 times over the range of that variable. We could end up with 4 billion possible models that we would need to simulate all possible neurons; then I guess you'd want to reduce those by saying that these two were like these other two and starting condensing them down; I'm not sure where this would go, but maybe we would find something cool or find a new insight by figuring out all possibilities
It might be helpful to know that (some | many | most) people think of biological neurons as being sort of poor-quality artificial ones, or at least that there is a direct correspondence between an organic neuron and an artificial one. This paper is making the argument that a given biological neuron is more like a network of artificial neurons, which is interesting and important because a lot of research into intelligence right now is going into making 'biologically plausible' models and testing those. If their results are essentially that each neuron has the sophistication of a network then say a future model of a cortical 'cell' (a cluster of neurons we think have a purpose as a group) should take this into account.
Genetics are interesting but neurologically we're still at the phase of knowledge where we are making up wild-ass conjectures and shooting holes in them for want of anything better to be doing.
Just as transistors are fundamental to processors, neurons are fundamental to brains, and it is important that we understand these basic building blocks if we ever hope to understand the brain and intelligence as a whole.
Propagation delay and circular loops. No it doesn't have to be "external" stimuli. There's no explicit input and output layer in how a real brain is constructed. That is an artificial constraint put on artificial neural nets to make a programmers job easier. A real brain can and likely does have self stimulating loops.
>Just as transistors are fundamental to processors, neurons are fundamental to brains
Transistors are also quite simple as individual units and the computational complexity comes about in how they are connected.
This is what I understand causes epilepsy. At least when they are out of control.
Furthermore, the brain is far to wet and warm for quantum mechanics to be relevant. Coherence times would suck.
Not to mention, even if inside-neuron computation turned out to be largely irrelevant for the whole-brain computation, it's still fascinating to look at how much computation, sensing and planning is happening inside any cell just for the cell to live in its environment. We tend to make a huge distinction between single organisms and their constituent parts, but human cells are not all that hugely different from bacteria in terms of their need for doing smart work to live.
I think brainwaves are probably mostly an epiphenomenon of the electrical signals that make up our brains' computation. Of course, EM waves being what they are, there is feedback between that computation and EM waves. But I think a good way to think about it might be like a computer giving off radio signals during certain computations, like [].
[] https://news.softpedia.com/news/emitting-radio-waves-from-a-...
For example old CRTs gave off RF signals that could be decoded to read the contents of the screen but the CRT display was not designed to be a radio transmitter.
From one published paper:
"We demonstrate that a single brain-neuron-extracted microtubule is a memory-switching element, whose hysteresis loss is nearly zero. Our study shows how a memory-state forms in the nanowire and how its protein arrangement symmetry is related to the conducting-state written in the device, thus, enabling it to store and process ∼500 distinct bits, with 2 pA resolution between 1 nA and 1 pA. Its random access memory is an analogue of flash memory switch used in a computer chip. Using scanning tunneling microscope imaging, we demonstrate how single proteins behave inside the nanowire when this 3.5 billion years old nanowire processes memory-bits."[1]
As he discusses in an interview[2], the idea that the membrane is the only important part of the neuron is an idea from 1907 and is fundamentally incorrect. While he does not claim to prove Orch-OR and has his own theories, he is sure the internal structure of neurons with protofilaments of various sizes capable of up to terahertz frequencies has vital functionality that needs its place in neuroscience.
There are about 5,000 microtubules per neuron. If these are quantum devices, we are many thousands or millions of years away from AGI, if it's even possible to achieve it without replicating those structures.
[1] https://aip.scitation.org/doi/abs/10.1063/1.4793995
[2] https://www.closertotruth.com/series/quantum-physics-conscio...
If there are 5,000 intercommunicating structures with quantum properties * 80 billion classically connected neurons instead of just the 80 billion node neural network, we may have not even approached the capabilities of a single neuron with classical binary supercomputers. That would explain why there isn't a single example of successful AGI, even to emulate the behavior of simple bacteria.
As far as efficiency, quantum computers require 1,500 square feet and lots of electricity to preserve the state of tens of qubits. They can solve problems with less overall power than classical alternatives, but you can't pack a whole lot of them into a neuron.
And taking an "outside" view, it's certainly not the case that humans are particularly good at solving problems where the only efficient algorithm we know of runs on a quantum computer... so even if there were any interesting quantum computation going on, it's not clear why that would be needed for AGI.
Going back to "cells are not computers," there are some (apparently good) arguments that plant photosynthesis relies in some essential way on nonclassical electron behavior to achieve its high efficiency for the purposes for which plants use it (biomass production), but that doesn't mean we can't achieve higher performance for the purposes of energy production with solar panels using a much simpler process. Even if you believe that the complexity in living organisms is mostly essential (which I do!) it doesn't mean much for the design of machines for humans to use.
I agree with you that the current methods for trying to achieve AGI are dubiously related to how actual brains work and probably aren't going to be successful without some major changes, but that's a very different point!
The structure of a microtubule is a fibonnaci geometry, and the theory is that different pathways along the structure provide superposition using a chain of hydrophobically isolated pockets of benzene molecules within the tubulin protein walls. This provides error correction for part of the state of the quantum system, somewhat misleadingly called a qubit, as well as protection against decoherence. I think consciousness depends on these structures and states because it explains why anesthesia, which bind to aromatics, stop consciousness without killing the brain. That doesn't mean all of Orch-OR is correct, but it is an extremely important piece of evidence, and the only testable hypothesis that I know of for why anesthesia works.
And I agree, living cells have very different requirements. They need to draw as little power as possible, rewire themselves as the environment/computational needs change, and be capable of repairing themselves.
> And taking an "outside" view, it's certainly not the case that humans are particularly good at solving problems where the only efficient algorithm we know of runs on a quantum computer... so even if there were any interesting quantum computation going on, it's not clear why that would be needed for AGI.
There's no evolutionary advantage to solving complex math, but people with different neurology are able to perform incredible calculations. The entire principle behind AGI is that if you can model the network effect of the brain, you get AGI. If that model is completely wrong, the theory behind AGI isn't going to work.
> Going back to "cells are not computers"
Correct, cells cannot be classical computers. They would require too much power for too little benefit. That's why paramecium have microtubules instead of processors, and rely on fundamental aspects of quantum physics instead of comparably primitive turing machines. And to borrow a phrase from Hameroff, proponents of AGI should try modeling the behavior of single celled organisms before they try the brain.
> there are some (apparently good) arguments that plant photosynthesis relies in some essential way on nonclassical electron behavior to achieve its high efficiency for the purposes for which plants use it (biomass production), but that doesn't mean we can't achieve higher performance for the purposes of energy production with solar panels using a much simpler process. Even if you believe that the complexity in living organisms is mostly essential (which I do!) it doesn't mean much for the design of machines for humans to use.
The latest breakthroughs in solar efficiency are literally based on inspiration from biology:[1] "Although we can’t replicate the complexity of the protein scaffolds found in photosynthetic organisms, we were able to adapt the basic concept of a protective scaffold to stabilize our artificial light-harvesting antenna.”
The newest research of building quantum computing devices is moving away from trying to wrangle individual atoms and instead moving to storing and manipulating molecules.[2] So, for the next generation of computing, the complexity of living organisms might be absolutely essential for designing machines. As quantum computers grow in capability, we will be able to model more of the quantum world, because "predicting the behavior of even simple molecules with total accuracy is beyond the capabilities of the most powerful computers."[3] Each generation of quantum computers will bootstrap the next.
Once the hubris and arrogance of people who think nature couldn't have possibly evolved to take advantage of quantum properties is finally over, I think there will be a revolution in every field as it gets cheaper and cheaper to model the planck scale world. The first victim will be AGI, and there is a lot of money and ego desperate to keep that marketing scheme viable.
[1] https://scitechdaily.com/breakthrough-in-stabilization-of-bi...
[2] https://www.sciencedaily.com/releases/2020/09/200902095130.h...
[3] https://www.scientificamerican.com/article/how-quantum-compu...
Whether or not "the next generation" is "biologically inspired" (which doesn't mean using exactly the same mechanism), solar panels right now outperform plants on a pure energy production basis, and do this without any complex molecular or quantum machinery. This is because energy production is much easier than biomass production. That's my basic point, if you don't care about beating life on literally every axis (particularly energy efficiency, replication, and use of cheaply available materials) it's entirely conceivable you can do better. So if people want to build nanobots there's basically no chance they're going to beat life, but that's pretty different from AGI (even though it seems like the same people are invested in both for whatever reason).
Anirban is awesome and his lab is even crazier
“Realization of universal quantum gates is rather challenging in the case of spin defects because the rigorous conditions needed for confining quantum decoherence are likely to limit the coherent exchange of information between qubits as a result of scarce control over qubit-qubit distances. In this respect, a chemistry-driven bottom-up approach to qubit scalability is more appropriate. Molecules are highly versatile, enabling their electronic structures and spin environments to be tuned at will with the use of simple synthetic chemistry tools. Moreover, they can be replicated in large number, functionalized in the desired way, and organized in a controlled manner for the production of large qubit arrays. Undoubtedly, chemical design offers endless opportunities for magnetic molecules to be tailored for specific technological tasks.”