Hacker News Reader

Comment by vbezhenar | Hacker News Reader

How newborn brain works is absolutely fascinating for me. I just don't understand how is it possible.

Human DNA contains 1.5 GB information.

Human body, including brain, gets built using this information only. So our "preconfigured" neural networks are also built using this information only.

And apparently it's enough to encode complex behaviour. That's not just visible things. Brain processes a humongous amount of information, it basically supports living processes for entire body, processing miriads of sensors, adjusting all kinds of knobs for body to function properly.

I just don't understand how is it possible just from a purely bit size approach. For me, it's a mystery.

View on HN

It somewhat makes sense if you think of it in terms of a really complicated 1.5GB metaprogram with a huge pile of conditionals that are triggered by the programs it itself writes (proteins). The final you is made up of an incomprehensible huge number of copies of the metaprogram, running different configurations, and spitting out programs to each other which then do more stuff. Our human brains can't really conceive of a configurable metaprogram that writes programs by interacting with itself in different configurations that it itself sets up.

Something similar: Kolmogorov complexity.

There is a finite size program that can generate infinite digits of pi (in infinite time). Kolmogorov's complexity of pi is finite even when the object is infinite.

It's not very surprising that it takes a few GB of a program to encode conscious 'us'. Humbling to think about it though...

For a demonstration of Kolmogorov complexity, it's good to watch "A mind is born"[0] by lftkryo. It's only 256 bytes, but can generate over 2 minutes of complex music and video. Also, the name is appropriate for this topic :D

[0] https://www.youtube.com/watch?v=sWblpsLZ-O8

Yeah but pi digits are essentially random noise, but any human is a precisely build system. E.g. there are exactly two identical eyes with nerves going to this precise area of brain, every time.

It's more like mega-efficient archive utility that unzips a few GB into a human, I just can't fathom it.

That's exactly the wrong way to think about it, and I'm surprised that so many devs think of it that way. We already have programs that works exactly like that (i.e. producing rich, complex output that would be many times the size of the input code + data if encoded raw): procedural generators. It's emergent complexity, not compression.

> It's emergent complexity, not compression.

They might be the same thing.

That comment does not have anything to do with what we are discussing (i.e. R/DNA).

It was also developed “iteratively” under extremely harsh selection criteria over a time scale that is so long it is almost impossible to reason about. An old geology textbook I had used the analogy of a geologic timeline that stretched from LA to NYC. Life appears really early (in CA somewhere IIRC) and human existence is about the width of a crack in the pavement just before you hit the Atlantic Ocean.

Using a timeline from LA to NYC, since you made me curious:

- life formed 3.7B of 4.5B years ago, which is 700km towards NYC from LA; or about Colorado

- proto-humans formed 2M of 4500M years ago, which is about 1.7km “from” NYC; a distance hard to compare with the whole way

- human lifespans are about 70 of 4.5B years, which is about 6cm “from” NYC; a distance hard to compare with either 1.7km, 700km for life to form, or the whole 3966km.

Ha! thanks! My memory of this was way off. But I guess I liked the idea at the time if I remembered it at all decades later. Life has been evolving for a LONG time.

Nature recently posted an interesting video [1] about what causes developing hearts to have their first beat. The gist is that eventually random electrical noise triggers a propagating wave which is then continued and repeated by the cellular automation nature of heart tissue. You don't need as much software if your system is composed of emergent properties.

[1]: https://youtu.be/SIMS2h5QsZU

> For me, it's a mystery.

For me, it's one of the last true mysteries! We've figured out damned near everything else, nothing has this level of "unknown" to it.

It's simply mind-blowing to me how such a tiny block of data can encode such high-level behaviours so indirectly!

Genes code for proteins, not synapse weights!

Those proteins influence cell division, specialisation, and growth through a complex interplay of thousands of distinct signal chemicals.

Then those cells assemble into a brain, apparently "randomly" with only crude, coarse patterns that are at best statistical in nature. Some cells are longer, some shorter, some with more interconnects, some with less, but no two perfectly alike.

Then, then, somehow... waves hands... magically this encodes that "wide hips are sexually attractive" in a way that turns up fully a decade later, well into the "pre-training" phase!!!

What... the... %#%@!

How does that work!? How does any of that work?

Y'all work in AI, ML, or adjacent to it. You know how hard it is to train a model to learn to detect anything even with thousands of examples!

PS: Human DNA contains only 750 MB (62 billion bits) of information, of which maybe 0.1% to 1% directly code for brain structure and the like. Let's be generous and say 10%. That is just 75 MB that somehow makes us scared of snakes and spiders, afraid of heights, attracted to the opposite sex, capable of speech, enjoy dancing, understand on instinct what is a "bad" or "good" smell, etc, etc...

There is still a big discussion of nature vs nurture. Did not follow the subject you mention but many things can be in fact just learned.

Also, as mentioned previously, there is more than the DNA at work - like at least epigenetics, but I guess the fetus is influenced a lot by the mother's body.

Epigenetics and mother's body influence feel - to me - like magic more or less the same. And the nature vs nurture regarding tastes developed either early or later on, well, as a father of 2 siblings who are radically different in certain tastes, I don't really know where I would have nurtured them into being different. I try to introspect a lot on that, maybe we did something but honestly... I don't think so.

With humans, we can even imagine that mother body teaches child brains via placenta or something (I don't think that's what happening, but whatever).

However think about birds. They lay eggs. So there's no direct connection between mother body and child body. Yet it works somehow...

The yolk (used directly in the embryos development) is generated during 10 days (https://www.purinamills.com/chicken-feed/education/detail/ho...). This could give the opportunity to pack a lot of "indirect information" to be used by the future embryo.

Regarding "teaching" the child while in the womb, it is exactly what is happening, see: https://www.americanscientist.org/article/baby-talk

I do agree that some organisms will transmit more "information" (via multiple ways, chemically, mechanical, etc.) than others (like maybe the birds) but the fact is the DNA is just a part of the development process and even if maybe it is "the first one", it will not "pack" everything.

The brain absolutely and biology in general when one starts digging.

Discovery of DNA was positioned as a "Biology: Mission Accomplished" - it's far from true. We don't understand all of DNA and epigenetics. We don't have a good understanding of how life began.

Back to the brain, it's power consumption to capabilities, weight to capabilities is just insane. The link to brain size and intelligence is a mystery as well - jumping spiders, octopus, corvids, parrots ...

For us it's hard to train a model because our compute and resources is nothing compared to nature's "compute" the whole universe: "it" has absurdly more resources to run different variations and massively parallel compute to run the evolutionary "algorithm", if you think about all the chemical building blocks, proteins, cells, that was "tried" and didn't survive.

From that angle our artificial models seem very sample efficient, but it's all hard to quantify it without know what was "tried" by the universe to reach the current state. But it's all weird to think about because there is no intent in natures optimizations it's just happens because it can and there is enough energy and parallel randomness to eventually happen.

And the real mystery is not how evolution achieved this but that the laws of chemistry/universe allow self-replicating structures to appear at all. In an universe with different rules it couldn't happen even with infinite trial and error compute.

Thanks for saying this. I wish people regarded the unimaginable vastness of the state space represented by the time scales involved and relatively small size of the interacting molecules. The inherent "parallel compute" is dizzying beyond our comprehension.

I wish we could know if our universe is an aberration.

Sure, the sheer volume of trial, error, and feedback that’s gone on in evolutionary history is mind boggling, but human intelligence is relatively recent and has had only a few hundred thousand turns at that wheel with a population of maybe a few million.

To be fair, we have few traits that are truly unique, but even going back along our branch of the tree of life all the way to the first recognisable mammals is not as many generations as you’d think. Certainly nothing like what goes on with fast breeding life like bacteria!

The enormity of effort also doesn’t explain how the end result works.

The way our genes encode for high level instinctual behaviour is akin to controlling the specific phrasing of a company’s quarterly report next year by changing out the coffee beans at the cafe that the accountant’s roommate frequents.

Even if I told you that I’m Doctor Strange and that I tried this ten million times before I got the exact right varietal of bean, you’d still be impressed and have a long series of follow questions!

As a person who knows next to nothing about how the brain or the genes that configure it work, I tend to think of this in terms of 80s video games like River Raid. The level data for these games, if stored naively, would fill the computer's available memory many times over. So they just store a pseudorandom number generator seed along with a few other parameters. Coupled with a few rules to make the level playable, it can generate a seemingly impossible number of levels with very little stored data.

Maybe the genes just encode a few crucial rules and the rest just emerge from that.

Oh, and I know even less about how the universe works. But I tend to think of it in the same terms: Emergent phenomena stemming from simple rules à la Game of Life.

Ever since I read about Rodney Brooks and his idea of the Subsumption architecture I've been convinced that something like this is going on in our minds - likely with some other mechanisms too. It just clicks for me - I'm mostly likely completely wrong, but it's a pretty cool idea, and I've used it to create some really interesting simulations.

https://en.wikipedia.org/wiki/Subsumption_architecture

That's mostly how I think of it, albeit the analogy I use is procedural texture or music generation in 4K demos.

There are very simple algorithms that generate (or maybe just expose) complex structures already "present" in the universe.

> gets built using this information only

No they don't. There is plenty of external stimuli that also serves as input, e.g. the process of raising a child and complex thoughts that may only be transferred from grown human to grown human.

Try raising a human in a barren cell without human contact or as part of a pack of wolfs and you'll see how much a human brain is built from "DNA only".

Certainly, and I don't think anyone really doubts this.

Still, people are sometimes surprised by how DNA may affect more parts of behavior than they previously thought.

Not necessarily by directly coding for the behavior. In many cases, the DNA will just modulate how we learn from the environment. And if the environment is fairly constant, observed behavior can correlate more strongly with DNA that one might have expected.

Yeah compared to animals we have a lot of extra bootstrapping outside of physics/chemistry alone via culture and stored information similar to how cell DNA bootsraps via physics, human mind boostraps via stored information in human "network" (talking, internet, books) after being born.

I think you're underestimating the role epigenetic information plays. 1.5 GiB encodes every protein used to build us, sure, but which genes get switched on when and how are sensitive to factors not encoded for in DNA, including the environment of the cell and the fundamental chemistry of biology. Epigenetic information is hard to capture but can profoundly affect how an organism develops; cloned cats, for instance, may show a vastly different fur color and pattern from the original, to cite just a highly visible example.

That's not additional information. It's a kind of codec for sure, but it's not magic information from nowhere. Like a compression algorithm.

Not information from nowhere, no. But information from outside the genome. To use cats again, colorpoint cats such as Siamese are subject to a temperature-sensitive mutation in the genes which code for fur pigment, so the fur at the coolest parts of their body (face, ears, paws, tail) is the darkest. The colorpoint pattern is not coded for in DNA. It needs input from the environment in order to be expressed.

It's not really compression. It's more like, you can write a much shorter Lisp program to do the same task as a C program, but you need the entire Lisp runtime to get it that short.

Have you looked into the amazing things people do with procedural generation with only a tiny bit (kilobytes, often) of source code? My intuition is that this is vaguely analogous.

Here's an example from 2003, where the entire source code, from music to visuals, fits in 64 kb: https://youtu.be/HtJvSvQnep0

Here's a good gallery of such demos: https://64k-scene.github.io

The even crazier thing is that DNA does not encode any of that. Behaviour and morphology is not directly encoded in there, you'll only find recipes for proteins. The zigote will divide into billions of cells that share that same recipe book. Depending on the electric and chemical signals surrounding cells are sending, individual cells get their "personalities" or function. This cell colony forms an organism which emerges from the sum of morphology and behaviour of all cells. But you'll find no recipe for an arm in DNA, it is the result of the work of the collective intelligence that is your body.

I'm not sure in what sense there isn't a recipe for arms in our DNA. To me, it seems the DNA does encode that stuff, but in a highly compressed format that is then "unzipped" through the laws of physics and biology into a living and breathing being with arms.

I mean, the information has to be in there somewhere, right?

I don't know either, maybe epigenetics play a part in this (Some information transferred from the mother cells to the child)?

If the DNA does not encode it then it's an astounding coincidence that it happens so reliably and repeatedly.

I don't think human DNA generally codes for the behavior derectly. Rather, DNA can code for how the brain learns from incoming data streams.

If the brain naturally tunes into some sources or patterns of input rather than others, it may learn very quickly from the preferred sources. And as long as those sources carry signals that are fairly invariant over time, it may seem like those signals are instinctual.

For instance, it may appear that humans learn to build relationships with kin (both parents and children) and friends, to build revenue streams (or gather food in more primitive societies) and reproduce.

Instead, the brain may come preloaded to generate brain chemicals when detecting certain stimuli. Like oxytocin near caregivers (as children) or small fluffy things (as adults). When exposed to parents/babies, this triggers. But it can also trigger around toys, pets, adopted children, etc.

Friendship-seeking can be, in part, related to seretonin-production in certain social situations. But may be hijacked by social media.

Revenue-seeking behavior can come from dopamin-stimulus from certain goal-optimzing situations. But may also be triggered by video games.

And the best known part: Reproductive behavior may primarily come from sexual arousal, and hijacked by porn or birth control.

Each of the above may be coded by a limited number of bytes of DNA, and it's really the learning algorithm combined with the data stream of natural environments that causes specific behaviors.

A lot of animals are born "hardcoded" with most of the instincts they need to survive, so some behaviors are clearly innate.

And "how the brain learns from the incoming data streams" is, in part, driven a set of behaviors too.

A baby's eyes are trying to detect and track certain preset features long before the primary visual cortex learns to make sense of them. That's a behavior, and it exists for a reason. As the baby develops, the baby would try to seek out certain experiences to learn from them, which is a behavior that exists for a reason too.

There's a hypothesis that certain mental disorders are caused by this innate learning process going off course, but it's just a hypothesis, of the kind that's hard to prove conclusively.

Indeed. But most instincts involve elements of learning. Meaning the instincts may be stored using a much smaller number of bits than if they were stored as traditional IF-THEN-ELSE computer program.

For instance, the pattern the brain seeks to optimize to learn to work may be much smaller than the full algorithm for walking.

And if the brain learns quickly enough (and if a newborn animal started learning elements such as balance, moving legs, etc, before even being born), learning to walk may be learned in minutes instead of months.

I think it's a wrong way to look at it. In addition to DNA information content, one should count also the complexity of the proteins and higher-level structures in the gametes.

It’s like one computer with program (DNA) and helper programs (RNA) creates second generation of computer and programs (proteins, lipids, carbohydrates) that essentially create their own version of computer system in which they govern things like enzymes, hormones etc

But keep in mind that humans are not created in vacuum. After those two levels of computer create third level that is brain, actual programming of brain is done by other living humans.

So actual „humanity” is what persists in living population and would reset when population is culled and newborn must live and learn on their own.

Even if such newborn would live long enough to have access to things like books, computers, even sound and video records… those would be completely useless to them because they won’t even know language and skills required to use those.

> 1.5 GB information

Well, nature has a big advantage over us in that it doesn't need to "make sense" of that code :). So it can, for example, do crazy reusage optimization patterns. A "subroutine" that is used in one place could also be part of a "data piece" of another part. A "header" part can also double down as a "validator" of another part. Doesn't need to make sense, it just needs to work. The only limits are the laws of physics. I would not even call it compression at this point. It's more like heavily optimized spaghetti code.

I find using the term "junk DNA" for non-coding DNA to off-putting for exactly this reason. There's most certainly "cruft" accumulated in any evolved organism's DNA, but the very presence of that "cruft" might just as well be serving another purpose.

To me, it feels similarly impossible/spooky to how image models work.

Consider a model like SDXL:

- each image is 512x512, plenty of detail

- max prompt length is 77 tokens, or a solid paragraph

- each image has a seed value between 0 and 9,999,999, with each seed giving a completely different take on the prompt

I can't begin to calculate the upper limit on the number of possible human-readable prompts that can fit in 77 tokens, but multiply even an (extremely conservative) estimate of a million possible prompts by 10 million seeds and it's clear that this model "contains", at minimum, literally tens of trillions of possible meaningful images -- all in a model file that's under 7 GB.

I suspect it works similarly to the biological side -- evolutionary pressure encoding complex patterns into hyper-efficient "programs" that aren't easily interpretable, but eerily effective despite their compact size.

"Human DNA contains 1.5 GB information."

Human DNA is tightly integrated with its environment. Instead of stand-alone, think compressed, source code of a high-level language running in an interpreter and with a standard library with 10-100x more functionality.

There's also how networks have combinational effects, some things in the body use temporal encodings, and who knows what else. We can't really estimate the information content of all of this put together since we don't even understand it. It is amazing, though.

This is why I’m so insistent that LLMs aren’t the best way (if they are a way at all) to getting to human level intelligence. The maximum amount of energy and input data required for training and inference is many orders of magnitude less than we are currently using.

~25 years from conception to maturity, millions and billions of years of brute force development... There is a lot of energy involved in typing this sentence to you. I am not sure LLMs use more.

Yes, but the inference cost of humans is extremely low. We're constantly making decisions and generating thoughts, most subconscious, while using extremely little energy. It's remarkable how energy efficient the human body and mind, and animals in general, are.

Yes it is impressive but the front loading shouldn't be dismissed

But LLMs don’t get a pass on the billions of years of front loading either. I mean, I see what you’re saying, and I get your point for sure, but my point is that we should be able to find a more clever way to accomplish this that builds off our billions of years of front loading. LLMs seem like the lazy way out in a way.

> Human body, including brain, gets built using [DNA] information only

I think there is a good chance there are other substantial information transfers from one generation to the next. The total genome of all that gut bacteria is orders of magnitude larger for example.

Well technically yeah but consider that it takes ~9 months for the product to function without constant life support, at least a few years until majority of the basic functions work and ~15 years until it is fully functional.

Talk about compile time.

9 months is caused by head size to how far you can stretch the exit ratio. In a way, we are born prematurely, to lessen the probability of death in childbirth (for both the mother and child).

The longer I think about it, the worse it gets.

It’s not foolproof, but I can easily transmit a huge amount of information to someone by saying “Titanic prow king of the world scene.” In seven words, which could be fewer if I were really trying, the recipient has a moving image and sound in their head (as long as they’re the right age group- every example I could think of made me feel old).

> Titanic prow king of the world scene

Well that was transmitting "a pointer" more than anything else, but yes I agree that nature could be doing the same thing. Not hardcode behaviors, but certain chemical reactions to some "pointers" that are totally environment related. Arachnophobia apparently could have a genetic component, so there could be a "spider pointer" somewhere.

"Human DNA contains 1.5 GB information"

If compressed, then there is room for more. (but afaik much is rather unused)

And for me I cannot say, that life is not a mystery to me, but this specific part I have less trouble with imagining it. As little code can create complex worlds and simulate them. (a minecraft wasm build for example is just 14 mb, but fully working)

> I just don't understand how is it possible just from a purely bit size approach.

It's ~750MB (3 billion base pairs). But anyway, that's a size of a decent Linux distribution with tons of software.

Actually the DNA is very inefficient with many areas that appear to do nothing. 1.5GB is a ton of "source code".

There is no significant evolutionary pressure to erase unnecessary parts.

You can make a brainfuck runtime in less than a kilobyte and it can run any program known to man.

Makes one curious about epigenetics!

Remember the body had 9 months to "learn" a bunch of stuff already.

I'll probably get downvoted for this, but recent (last ~100 years) evidence and phenomena suggest that consciousness might be fundamental to reality, and thus there could be some other information transfer we would currently consider "woo" going on here. This is hard (if not impossible) to prove, of course, but quantum mechanics has totally bewildered many aspects of the materialist ("reductionist") model of the universe. There is a large and increasing number of physicists and other reputed scientists/researchers who are adopting some variation of the consciousness-as-fundamental stance.

That's been shown not to be true. Concious observers are not required for reality.

Recursion

Reply on news.ycombinator.com