Could a computer scientist build a brain?

> The program must fit in a genome-sized instruction set of about 1 gigabyte

True but this can be a lot larger, maybe even a 1 exabyte instruction size since it depends on he programming language used to express the program. And even accounting for the invariance of runtimes at scale*, the constant factor might be gigantic since we have to build up a lot of cellular machinery first. Reminds me of the Carl Sagan quote: To make a sandwich,you must first construct the universe!

* For example, to translate a program from language A to language B, you can dedicate a constant size to write a language A to B translator. Thus at large program sizes , the kolmogorov complexity (i.e instruction set size) is fairly similar between programs since the size of the program dominates the size of the translator program which is constant. But the constant factor for cellular machinery might be gigantic**

** Or not if we only need a rough simulation

I think the question and article is fascinating. I do think that one should not only look at the genetic representation, and the algorithm a cell runs, but also the formalized environment that runs the simulation.

I’ve been thinking about this for a few years and I truly believe these cells need to be able to “move” in either a 3d lattice similar to a cellular automata or become nodes in some cyclic graph (or maybe both).

For the genetic representation, I had an insight that DNA/RNA looks strangely similar to SKI calculus (combinators) and have been using a Church encoding to translate the SKI program to an “action” per turn (move, connect, spike, divide, etc.). It has a nice property that the cell’s “program” and the spikes between them, and the input/output to the simulation itself can be the same thing (just a string of combinators).

Either way the author is spot on when it comes to this sort of thing having the properties 1) “always on” (no separate training mode), 2) local interactions, 3) everything stems from a genome, 4) each cell is its own little mini-program with the same genetics as everyone else.

All I can say is that evolutionary program is hard and takes either massive compute or large timescales to run the simulations. Fascinating stuff though

You'll need a simulated environment because brain development depends a lot on the environment, it's not purely genetic.

Imo, there isn't anything fundamentally special about the brain as a cognitive substrate. Even a pool of water could have cognition (reservoir computing), i feel that it's a general property that can emerge in any sufficiently high-dimensional physical system with enough internal degrees of freedom to explore many configurations, and enough coupling and stability to remain coherent, and enough plasticity for experience to leave persistent changes in its future dynamics.

And subjecting such a system to sustained external objective pressure...where some internal organisations perform better than others and it would progressively accumulate some useful structure about its environment.

The brain is just a really old artefact of billions of years of blind environmental weathering of matter being repeatedly shaped by the requirement that its host continue to exist as a coherent, persistent entity.

In short the brain is not fundamentally special in the principles that make cognition feasible.

What i'm trying to say in response to this article is that he's working at a far higher level of abstraction than is required. It's putting software before physics.

Also this somewhat chimes the same tune from the bitter lesson; it’s extremely non-trivial to reverse-engineer from the top down the residue of eons of evolutionary search. You’re basically trying to infer some smarter way of achieving this one absurdly overfit artefact it left behind...won't be betting against the full brunt of mother nature's massively parallel search.

https://en.wikipedia.org/wiki/Evolved_antenna

This isn't quite the question the article is asking, but if you expand it just a bit to a brain rather than a simulation of a biological brain specifically, the specification complexity of a modern AI is not that large. You can get a good idea of the foundation of the system with some 3blue1brown videos. While just watching those and deeply understanding isn't enough to score a job at a frontier AI lab, I'd expect that a lot of what the labs do is similarly not that complicated to specify... it's coming up with the ideas, tuning them, and then trying to test them at the now-staggering scale it takes to prove them out for commercial use that is most of the issue.

The complexity of the AI itself, the final weights, is vastly higher, because that incorporates all the data that was flowed through the "brain".

In a decent encoding you could probably still fit the initial state of the frontier models comfortably in just a few kilobytes of code-golfed code, including the update functions and every necessary for the actual training. Recall the specification complexity is the smallest program that can create the initial state, not the initial state itself; the specification complexity of "give me a trillion 64-bit numbers generated from this psuedo-random number generator" is on the order of that very English sentence in size, not 8 tebibytes of specification complexity.

Is the result a biological brain? Obviously not. Biological brains do not do what the LLMs are doing. But unlike someone asking this question 20 or 30 years ago, where "yeah, but what if some other architecture could work too?" was still largely hypothetical, and neural nets were still largely toys, now it isn't. We may argue if an LLM is as smart as a human but I would say that at least on its playing field it is quite clearly smarter than most biological brains in existence on most measures we care about. I have to qualify "on its playing field" because it is fundamentally a text completion engine, so for instance even very very tiny biological systems still beat it on things like "ability to drive an ant body around to do useful tasks". That's a separate field of AI right now. Stay tuned on that matter, but it's not how things work today for sure.

There is still something to what biological brains do that we are not matching; as I like to say, humans do what they do without the entire contents of the internet being poured through their head multiple times over. But the idea that maybe we don't have to exactly match biology to get something useful is no longer just a theory.

The human genome is essentially a billion-year-old quantum engineering document that we can read but can’t “yet” write.

Asimov’s positronic brain was the conceptual framework with the wrong formula.

One aspect missing from TFA is that in biological brains a lot of the training is performed during query time. If we assume 1 GB of DNA is enough to encode the brain's overall structure we still do need training data (e.g. visual/auditorial/tactile) to build out the strength of the synaptic connections.

I have this science fiction story in my head that consciousness can evolve from organic to mechanical and back to organic as it is perhaps the most energy efficient method to encode complexity.

I agree that the program to create brains will be small, but the logic presented to explain that is false because it ignores the size of the compiler.

For example you can have a program that is 1 bit long and encode either a browser or an OS, provided the compiler/interpreter is big enough.

Jeff Hawkins and his team have been trying to understand human brain and postulate a theory of intelligence based on neocortex structure.

It's if statements all the way down, except you run all of them in parallel and then flip a coin. Let me get my keyboard.

there is a theory by the brilliant Roger Penrose and extended by others that the brain is just not neurons but also quantum effect

which would make it impossible to emulate via current technology

excellent PBS Space Time on that

* https://www.youtube.com/watch?v=xa2Kpkksf3k

"Consciousness" is tick based and it does not require a massive human brain. Even the smallest of animals react in real time to stimulus. There's a filter that lowers the amount of tokens the next tick has to process. That's why you can watch a video and two different people will notice entirely different things about it. Forgetting is our brains culling what they process. Being awake eventually degrades that process. You can see this in every study on tokenized llms showing a decrease in accuracy as the context fills up. So too does your brain. That is why we sleep.

Short answer yes. We can build a brain.

As for quantum effects. It's just the machine. Not the mind itself. It's sort of like asking how a CPU works and then expecting that to inform your science on cognition.

This is a very insightful article. That said, I think it has a potentially fundamental blind spot: it largely neglects the role of external information in constructing the developing nervous system itself.

The article treats information arriving from the environment mainly under the heading of “learning”—for example, learning English vocabulary or learning to program—as though development first produces a nervous system and environmental input subsequently supplies it with acquired knowledge. But this seems to rely on a much stronger assumption: that, given the genomic program alone, a human zygote would unfold into something recognizably close to a normally wired human nervous system even in the absence of the appropriate external inputs. I am not convinced that this separation between “development” and “learning” is biologically justified.

My own case is one reason I find this distinction questionable. I have unilateral amblyopia caused by congenital anisometropia. Amblyopia is not primarily a defect of the eyeballs. It is a neurodevelopmental disorder in which chronically unequal visual input during a critical period leads to abnormal experience-dependent refinement of the visual system, particularly in visual cortex. Processes such as synaptic competition, strengthening, weakening, and pruning proceed under systematically distorted input and consequently produce maladaptive circuitry.

In ML terms, the problem is not that a fully developed visual system later fails to learn how to see. Rather, the developing system is being trained on the wrong input distribution, and the resulting model itself develops differently. The sensory data are therefore not merely “examples” presented to an already constructed learning machine; they are part of the process by which the learning machine is constructed in the first place.

More generally, I am not aware of evidence that a human embryo deprived of normal tactile, thermal, proprioceptive, vestibular, and other forms of sensory feedback—or systematically exposed to abnormal versions of them—would nevertheless develop an entirely normal nervous system. Many capacities that we casually describe as “innate” may therefore not be specified by the genome alone. They may instead be specified jointly by the genome and an evolutionarily expected environment.

If that is true, then the article's framing of the spatial complexity of nervous-system development as being bounded by the information content of the genome may be incomplete. The developing nervous system has access to an additional information source: signals from body and external world. Sensory input begins remarkably early, carries a great deal of information, and can have causal effects on the formation and refinement of neural circuitry itself.

Therefore neural development should involve two complementary processes. The article focuses primarily on decompression. But there is also a process running in the opposite direction: the developing organism continuously takes in a very large stream of structured environmental and bodily input and compresses regularities in that input into the organization of its neural circuitry. The information incorporated into the mature nervous system need not all have passed through the genome.

In that sense, the genome may not encode a complete developmental program so much as a program that is designed to operate under a particular distribution of environmental inputs. What we call “normal development” would then be a property not of the genome alone, but of the interaction between the genome and the environment that evolution could statistically “expect” the organism to encounter.

This is why I find the article's treatment somewhat too narrow. By placing environmental information largely in the later category of “learning,” it risks drawing too sharp a boundary between development and learning. In at least some neural systems, experience is not merely information written into an already developed network; it is one of the inputs to the developmental algorithm that builds the network.

I am only a bioinformatics undergraduate so my understandings of both bio and infomatics are somewhat slop. Would be grateful to see any criticisms.

TL;DR: I think that “the program must be small enough to fit in a genome” is incorrect. [Environment / external information / sensory input] whatever you call it is actually one part of “the program” and it's huge.