Hacker News Reader

"DNA can encode everything the brain does with N bits of information"

It doesn't directly encode the structure of the brain. Development is required.

"Involved" is a serious weasel word when it comes to the brain, especially when AI is under discussion; everything in your skull is "involved" in cognition, but that doesn't mean it's doing anything particularly important, and it definitely does not mean that every detail of the dynamics is an absolute necessity to obtain Cognition. But I think we might be arguing different things...

We are arguing different things. My point is that you can't write a little equation for a neuron (which is an entire living cell!) and wave your hands and say "done!"

Kurzweil is way way way too hand-wavey for me.

It doesn't directly encode the structure of the brain. Development is required.

Your Java program doesn't directly execute on the CPU, compilation and interpretation are required.

Totally different thing. The java program encodes each decision and computation that it performs directly.

No a Java program requires an environment to run, so does a brain .

It's just that a brain alters it's behavior and structure as a result of processing inputs (development).

We don't (normally) write Java programs to change their behavior as they process information from their environment during execution.

You're confusing literal translation with development. Here's a very loose conceptual analogy:

Translation: translating Lord of the Rings into Chinese.

Development: hearing a short plot synopsis of Lord of the Rings and writing your own fantasy novel based on the same theme.

Development really does do something analogous to that. It takes a collection of proteins and rules and, through embodiment within the laws of physics, constructs the phenotype. The phenotype contains vastly more information than the genotype, and two identical genotypes will not produce absolutely identical phenotypes.

There is something fundamental about development that we do not understand. This is widely acknowledged in the developmental biology, evo-devo, and evolutionary computation fields. A closer analogy than the loose one above might be some of the behaviors we see with fractals and cellular automata, though development is less deterministic than that.

Evolution and development are somehow related. We don't quite get that either. But both processes add vast amounts of information and both involve adaptation.

Hand waving: the adult brain is the output of running a program stored in DNA for twenty years on a runtime (the physical universe) we can barley understand. To produce a brain from the data stored in DNA, we would need to simulate the runtime environment. To do that without simulating the whole universe, we would need to work out which subset of the universe to simulate. We are far away from knowing what shape that subset takes, let alone what the minimal number of bytes required to simulate it is.

My point is that you can't write a little equation for a neuron (which is an entire living cell!) and wave your hands and say "done!" Kurzweil is way way way too hand-wavey for me.

Yes, we're in full agreement on this, especially the part about Kurzweil - I don't think he advances the science of this at all, and his poppy PR approach rubs me the wrong way, for sure, as do many of his specific ideas.

In particular, I think it's a terrible idea to hinge our hopes for AI on the idea that we'll be able to carry out a perfect enough simulation of a full brain to call it "done". Not only does it seem to be a pipe dream at present, depending on a whole bunch of technological advances that may not come for a long time, it would be almost completely inextensible and teach us very little - once we had such a simulation running, it's unlikely that we'd understand enough about how to modify it to improve upon it.

I don't quarrel with the idea that DNA encodes the brain extremely indirectly at all, including a substantial development process that depends on a lot of things other than the pure genetic code, and I also agree that the dynamics of a brain are very dependent on the details of all sorts of cells, not just an oversimplified logical representation of them. Where I think we start to diverge is that I have issues with the idea that the particulars of any of those processes can somehow "piggy-back" any non-trivial amount of functionality into the system that's not coming from the genetic code.

Here's the clearest way, I think, to state my information-content claim: if we were to randomly change the details of the way neurons function in detail (still leaving them with the same highest-level capabilities), and randomly change the way brain development happens, and randomly change all steps of entire process that leads from DNA->brain (again, subject to the restriction that the whole thing still works), and even - I daresay - randomly change the laws of physics, then...

...if all of these changes still permitted us to write down any string of DNA (or whatever our new version of DNA was) that ultimately resulted in the growth of an intelligent human brain, the amount of DNA required would, on average, be pretty darn close to the amount that we see used today. And that has implications for other implementations of the same logic, namely that if we could find some way to optimize towards whatever solution evolution has found, we could probably end up with a shorter code for it than evolution has because we can allow all parts of the DNA->brain process to optimize for compression whereas the real physical process is largely frozen against change.

That's the sense in which I think Kurzweil has a reasonable estimate, nothing more, nothing less; where he takes it from there is another story altogether. :)

Sure, but Kurzweil's claim is merely that if DNA can encode everything the brain does with N bits of information, then regardless of how the DNA behaves, we have at least a loose estimate of the level of complexity in the brain.

The problem is that DNA is not sufficient. Organisms don't grow from naked DNA in a vacuum; the DNA is always contained within a cell which is enclosed within a more complex bio structure (egg, womb, etc) which may be contained within the parent organism in the case of mammals.

You need all of this information plus knowledge of the various interactions of the different environments to pursue Kurzweil's approach.

The universe of necessary and sufficient information is much, much larger than the 3 billion base pairs in your DNA.

For the moment we seem to have enough trouble modeling (relatively speaking, to the brain) simple proteins reliably.

It's quite a leap of faith to make these statements about the technological future with so little in terms of progress to show in these fields for the last decades. I think the most impressive a-life demos are now almost 10 years old, the best we can really simulate is (drumroll) a cockroach. And personally I think that's a milestone achievement because it means that at least we have a principle that works.

Going through the DNA route to get to a working brain seems to be a very roundabout way of getting there, it will require all of the embryonic mechanisms to be modeled accurately as well as something like the first several years out of the womb before you'd know if you had created something insane or something resembling intelligence.

Assuming you'd recognize it as intelligent even if you succeeded, there may be more ways of being intelligent than we know about.

Absolutely, I never said that I think modeling the brain would be easy (I don't) or that building it up from DNA is anywhere near the best way to attempt AI (IMO it's not).

I simply don't buy the argument that the algorithms of cognition inherit any substantial amount of functionality from their physical implementation, and hence, I see Kurzweil's complexity estimate as somewhat reasonable.

> I simply don't buy the argument that the algorithms of cognition inherit any substantial amount of functionality from their physical implementation

I'm somewhere in the middle on that. I wish for things biological to be clear-cut and deterministic enough that we can fully understand them the way we understand mechanical systems.

But precisely because the brain is encoded in precious little DNA there is some evidence that there is more to it than meets the eye, after all if 50M of gzipped data can encode the whole thing why do we have such a hard time understanding it.

There is enough repetition in there that some died-in-the-wool reverse engineer would have put 2 and 2 together by now if the secret was in the wiring or in some simple algorithm (ANNs for instance).

Apparent order appearing from chaos is a field that has seen some study and the amount of complexity that can arise form simple starting data is quite amazing, witness the mandelbrot set and other fractal forms.

It may be very hard to short-circuit such understanding and to 'divine' the workings of the formula without first going the long way around to understand the whole system rather than the 'seed' from which it grows. This is not simple mathematics where a simple equation on complex numbers gives you the mandelbrot set, it's possibly machinery interpreting an equation with 50 million terms.

In different terms, given a very distorted (dissected) picture of the 3 dimensional mandelbrot set would you be able to figure out the formula that gave rise to it without prior knowledge of the mathematics involved?

http://www.skytopia.com/project/fractal/2mandelbulb.html#epi...

FWIW, I'm not too far away from you on this - I don't think we are going to find a very simple algorithm to do whatever general pattern recognition the brain does. All indications suggest that while there may be such an algorithm programmed in and repeated (a lot more times in humans than other animals), it's far too complex for us to simply read off or guess at.

But it does suggest that we should be considering the functions that such higher level units might have, such that they become more intelligent as we compose them. Easier said than done,of course, especially since it's very difficult to actually observe brain dynamics in any detail.

You need all of this information plus knowledge of the various interactions of the different environments to pursue Kurzweil's approach.

Absolutely, and this is one reason why I think Kurzweil's approach overall is pretty foolish.

But I do think that his information-content claim is within the bounds of reason (I've explained my stance on that many other places in this thread, the gist is that it's highly unlikely that the whole developmental process supplies a huge amount of information "for free", in the same sense that it's unlikely that you'll achieve a 10x compression in code size by using Python instead of Ruby to write a program) and it at least tells us that a solution to the problem of intelligence does exist that doesn't require (for instance) explicit hard-coding of every neural connection inside a brain. In fact, it requires massively less information to specify than that, so there's some hope that in the end we'll be able to come away with reasonable approximations to that algorithm since it must be "fairly simple" (meaning: more complicated than anything we've tackled before, but maybe still within the realm of possibility).

That hints at the fact that AI researchers (at least those that don't buy Kurzweil's ridiculous "simulate-the-whole-thing!" approach) should be looking more into building out small "genomes" into massive structures rather than focusing too much on individual explicitly specified processing networks, because nature is clearly doing something like that, and it seems to work pretty well, letting it achieve a remarkably compact solution, given that evolution tends to produce very bloated code as a rule. There has been tentative research along these lines (http://en.wikipedia.org/wiki/HyperNEAT, for example), but there needs to be more, particularly to figure out what sorts of things we actually want these massive structures to do, what sorts of processes we should be focusing on to control that building process, what types of units we need in order to make the processing that we're doing feasible, etc.

if DNA can encode everything the brain does with N bits of information, then regardless of how the DNA behaves, we have at least a loose estimate of the level of complexity in the brain

Non sequitur.

Solution to unified field theory. That's about 32 bytes of information, uncompressed. There, since that statement is so low in information content, it must be easy to find. We have a loose estimate of it's complexity with my statement, right?

The problem here is that an adult brain has a much higher Kolmogorov complexity than the DNA that codes for its parts.

http://en.wikipedia.org/wiki/Kolmogorov_complexity

Sure, and a shag rug has a much higher Kolmogorov complexity than the instructions for how to weave it. If measured carefully enough you'd find every individual strand of yarn in it has a unique length, orientation, curvature, amount of twist and amount of fraying that can be measured to an arbitrary degree of precision. So if you wanted to reproduce or store a description of that exact rug, you couldn't do it. That doesn't mean you can't make another rug that's just like it or close enough to serve the same purpose and be recognizable as a shag rug.

Like that rug, the brain contains oodles of complexity that doesn't matter at all for our purposes.

There is a huge part missing here though. A rug simply has to 'be', a look alike will suffice. A brain has to work.

The question on the table, I think, is whether a brain that's pretty darn similar to a working one will still work or not - does it just have to have similar statistical properties, or are the explicit details of the connections important, does the chemistry have to be just-so, if we replaced neurons of type X with pseudo-neurons of type Y that do almost the same thing when viewed at scale Z but are completely different underneath, would it still work, etc.?

A lot of these questions are completely unanswered. We do know at least that the brain is extremely resilient to changes in chemistry and can work quite well even in the face of extensive damage, which is an indication that we might be surprised how little of the overall arrangement is actually necessary to keep it working properly.

We need to make sure that we're being careful with our language, too: the complexity necessary to specify any brain is a whole lot lower than the complexity to specify one particular brain. In AI the goal is the former, but for the latter, we really do have to worry about each strand in the carpet. An AI researcher might not give a rat's ass about reproducing your memories, and will be more than happy to construct something that can form any memories; on the other hand, to you, your memories (and the detailed wiring inside your head, which we might be able to alter substantially without "breaking" the brain) are vitally important.

Absolutely true.

That leads to 'seed AI', http://en.wikipedia.org/wiki/Seed_AI when run in reverse, so you'd have to implement the 'minimally viable self improving brain', and take it from there.

And even there maybe not all the strands in the carpet have to be just so, but it may very well be that there have to be certain amounts of each colour in roughly such and such a pattern with interconnects between these larger groups and so on. Some of that information is known but definitely not all of it.

The damage angle is a tricky one, some damage seems to be absolutely not problem at all, even if it is major, in other cases the smallest bit of damage seems to be enough to cause terminal failure. There are a lot of clues in there about the organization of the brain.

if DNA can encode everything the brain does with N bits of information, then regardless of how the DNA behaves, we have at least a loose estimate of the level of complexity in the brain.

No even close. Imagine a kilobyte filled with alternating zeros and ones. We have two commonly used methods for describing the information content of this kilobyte. Shannon information content is closely approximated by today's compressors - and 1k worth of alternating zeros and ones compresses to next to nothing. Kogorov Complexity is the length of the minimum necessary program to produce the 1k worth of zeros - again, a simple loop, incrementing a pointer then dereferencing the pointer to fill the address with alternately a zero or a one suffices - a mere handful of bytes are necessary.

Now imagine that the RAM chip holding the zeros and ones is hit by a bunch of cosmic rays , flipping about 10% of the bits. Both the size of a compressed version of the new kilobyte, and the Kogorov program to generate the new sequence are both going to dramatically increase in size.

Now, the thing is that DNA produces a brain to a fairly homogenous pattern - it's the equivalent of my series of alternating zeros and ones, although admittedly it's a bit more complex than that - this is to be expected, there's more Shannon information in DNA for encoding a brain than there was in my simple pattern of zeros and ones.

The cosmic rays are the equivalent of a brain learning, encoding information by weighting connections between neurons. This process massively increases the amount of information stored in a human brain, but this information must be copied if you actually want the copy to behave like the original. I would expect this amount of information to be several orders of magnitude bigger than that found in the DNA, which is why Kurzweil's claim is just completely off the wall.

We're arguing different things. You're talking about the information content required to reproduce a particular brain, which I agree, has a massive Kolmogorov complexity (even if we decide, as we might want to, to factor out many equivalence classes based on behavior). Far more than a DNA estimate would hint at.

But I'm just talking about (as are most AI people, and I think Kurzweil, as well) the informational content required to build a brain. Any old brain, could be yours, could be mine, is probably neither. The information required to do this is much lower, and corresponds loosely to your first example, a kilobyte with alternating ones and zeros.