Hacker News Reader

Comment by api | Hacker News Reader

I'm a life-long programmer who presently works in software, but I studied biology in college... mostly because I wanted to learn about learning by studying how living systems do it.

Biological systems are nothing like anything we would ever engineer, and to understand them we must remove our "anthropocentric engineer goggles" and look at them for what they are. Analogies between biological systems and computers, software, machines, etc. are very loose analogies meant to illustrate a point. Never take these analogies too seriously.

DNA is not a program-- it is a molecule, and one that may very well do things at the quantum level that are biologically important.

The brain is not a neural network. It is an interconnected colony of living cells of a variety of types, and it has been shown that all types of cells in the brain are involved in cognition.

We are nowhere near anything with the parallelism or information density of the brain. Getting close would take an advancement in computer technology of the magnitude of the transition from vacuum tubes in individual boxes to a 32nm Core i7. Nature has had billions of years, and it is way ahead of us. There's a nascent field called quantum biology that suggests that the brain may very well be a quantum computer, so maybe quantum computers are the vacuum tubes->ICs scale transition I am speaking of.

I think it's now possible with a big rack of multi-core machines in an MPI cluster to approach the capabilities of a fruit fly's brain.

Cells are not machines. I don't think we have the language to really describe what they are perfectly, but the closest I can come is "stochastic quantum probability field device." A bacterial flagellum is not a "motor," it is a quantum-scale chemo-electro-motive... well... our language breaks down. Like Feynman said, don't tell stories. Just speak literally and then use math.

Actually, come to think of it, there's one engineering analogy that might work for biology. Biology is quantum-scale nanotechnology. Yeah, that's pretty close.

(On a related tangent, I've found that many engineers are sympathetic to intelligent design type arguments against evolution. This is because they try to think about biology like engineers and take these machine analogies literally. It just doesn't work like that.)

View on HN

Getting close would take an advancement in computer technology of the magnitude of the transition from vacuum tubes in individual boxes to a 32nm Core i7.

...so about 35 years? ;)

well, if you're going to use that as a baseline you should also factor in that the rate of advancement is exponential. so maybe 10?

That probably helps more if the problem complexity is increasing linearly. If it's going up exponentially with problem size, especially if complexity increases at a faster rate than technology, then you're not going to make much progress.

ie, if emulating a human brain is only N times as hard as emulating a flatworm, Moore's Law might do the trick.

But if emulating a human brain is more like (flatworm complexity)^(number of cells in human brain - number of cells in flatworm brain) then Moore's Law is unlikely to help for a very long time indeed.

Maybe, though I am concerned about processing power stagnating.

Advancement is governed by economics as well as technical capability. There must be demand for new technology, or a field stagnates. Witness aviation as an example... utterly stagnant outside of military niche applications.

People seem to no longer want faster and faster computers, and the market seems to be moving toward lighter-weight lower-power portable devices like netbooks, the iPad, etc. Those have slower CPUs than current-generation desktops. I suppose the extreme gamer and server/datacenter markets are still driving performance, but for how long?

One problem is that programmers are not using the capabilities of current-generation processors, partly because the dominant OS (cough Windows cough) makes it horrifically painful to deploy desktop apps. This drives all development to the web and turns desktops into thin clients. In the end this kills demand for performance outside the datacenter market.

You are correct for consumer electronics. But commercial and scientific computing are driving increases computational power and density that do not seem to be letting up. In the 4 years I have worked in hosting the amount of server you can get in a 4u box has gone from 8 2.5GHz cores to 24 2.5GHz cores and from 32GB to 256GB. The storage requirements and compute requirements for applications are also increasing substantially.

You're confounding peoples preference for usability (including portability) with a preference for applications with low computational demands.

If you could pack the "extreme gamer" capabilities of a Playstation or an Xbox into a format as "usable"[1] as an iPad... Then you would of engineered the next iPad.

The iPad was able to come into existence because we've finally hit the point where we can cram that much computation into a small factor form (along with all the other engineering advances like wireless networking, reducing power consumption, improving display and improving battery life).

Most of those advances are directly descended from the pushing of the bleeding edge. Companies / people are not simply going to go "oh we've got iPads now. So no need to make anything faster / better / bigger".

[1] By usable I'm not talking about some magical Jobsian property of the device. I'm not even talking about the software interface. I'm talking about being able to surf the web / post to your blog / whatever while on the toilet. Try doing THAT in 1995.

Or 500, or 10,000, who is to know. The problem is that the way the brain seems to be organized is around very simple (at face value) processing components that operate at a ridiculously low frequency in a massive parallel manner that we programmers can only dream of, at a power budget that would make Ebenhaezer Scrooge look very happy indeed.

If 35 years (according to Henry Makram, linked below it's only a decade) was all it took then we could simulate the brain today at a reduced speed and get meaningful output, after all, all you'd have to do is slow down the inputs accordingly.

We're as far away from having a universally teachable computer (not programmable!) as we were in the early 70's when true AI was only about a decade away.

Some interesting reading about the 'state of the art':

http://spectrum.ieee.org/tech-talk/semiconductors/devices/bl...

http://bluebrain.epfl.ch/

http://www.technologyreview.com/biotech/19767/

The "simulate at reduced speed" theory appeals to me, but I think the actual numbers make it implausible. Assuming Moore's Law, 30 years is a 1,000,000 speed up, plus let's say 4.5 years for 3 more doublings, giving us 8,000,000. To simulate 1 second would take 3 months. Debugging would be frustrating. (assuming 35 years from now; seems an arbitrary figure.)

> Assuming Moore's Law

That alone may already be a mistake, it's an observation, not a law after all.

Besides, compared the 35 years ago we can now do things 1,000,000 times faster than back then, but computers are not 1,000,000 times 'smarter' they just give the same answers that you could compute back then but faster and on fewer computers.

The future is parallel anyway, so it isn't Moores law (increase in density of transistors on-chip) per-se that will drive this, more likely there will be a switch to increasing chip packing density with smaller chips (bigger yield) and better communications between the chips (think computing fabric).

We need a huge advance in programming languages before we can really contemplate building an AI by taking advantage of such a structure though, simply simulating the organic soup that forms a brain is going to be a much harder problem computationally and may simulate a dead or an insane brain much more easily than it will simulate a live and thinking one.

This is exactly what I was thinking.

Actually, if that last shift took 35 years, the next one of that magnitude will be even faster.

This is Kurtzweil's fundamental insight: exponential growth is faster than people realize. We consistently underestimate it because our brains are pre-disposed to think linearly.

If it takes you 1 year to solve 1% of a problem, your brain feels like you're 99 years away. In reality, you're only 7 doublings from completion.

Where that falls down is that it fails to consider limits. Exponential growth cannot continue forever. At some point, something will limit it.

It may be a physical limit, a supply-side resource limit, or an economic demand limit, but there will be a limit somewhere.

Without limits, a single bacterium could fill the entire universe in a few years.

Sometimes things do grow like that for a while, but Kurzweil's attempt to turn this into a universal law and neglect limits is hand-wavey and silly.

True. However decades long periods of exponential improvement in technology is the norm, not the exception. The most spectacular example is Moore's law, but it is hardly alone. If you read The Innovator's Dilemma you will find plenty of other examples, ranging from the maximum range of a steam ship to the volume of dirt a backhoe can scoop per hour.

The interesting questions are how much computing power you need to perform equivalent tasks to a human brain, and whether current technology will reach that before it plateau's out.

The idea of "things leveling out" evoke a metaphor of there being some finite amount of resources that can be gotten out of somewhere.

Exponential growth of tools opens up an exponential number of different avenues of exploration - if computers didn't advance at all for ten years, we'd still come up with many more ways to use them. With them advancing exponentially, we can not only find different ways of using them but new fields where different forms of exponential growth can happen. And so-forth. There's no fixed frontier but a moving process.

This isn't saying it's all wonderful but it's all likely to be a bit beyond our ability to encompass it - to draw a circle around it.

You can solve a problem 1% in 1 year if it is a linear problem then you are right. But if the 1% was the easy bit and the 99% remaining are very hard then you still don't have a solution.

If the solution is 99% easy and 1% hard then you may find out after completing 99% of the problem.

Many problems are like that, simulating the brain is an excellent candidate for being such a problem. If it was just a matter of throwing more computer power at it then we'd have solved it years ago, it's that big a prize. But there is still a large part of our understanding missing and understanding does not yield to Moores law.

Intel predicted the end of Moore's Law would come with 16 nanometer manufacturing processes and 5 nanometer gates, due to quantum tunnelling.

I think DNA is actually best described as a 1970's era binary in a running mainframe. It rewrites its own source code depending on outside information. And in order to understand what happens you need to look at a core dump that includes all the interesting bit's floating around and attacking/enveloping the DNA.

PS: Don't forget this source code has been hot patched (http://en.wikipedia.org/wiki/Patch_(computing)#Hot_patching) over 3+ Billion years.

Edit: To continue the analogy the boot loader has been lost to time (or sacrificed to free up memory), so getting a working system requires copying not just source code, but also much of the current state of the system.

Not too bad, but sort of like trying to describe an elephant as a bunch of balloons and a water hose glued onto a bail of hay on a table.

I also dislike those sorts of analogies because they are loaded with aesthetic value judgements that do not apply. Programming code that looked like that would be ugly. The genetic system is beautiful and elegant.

> The genetic system is beautiful and elegant.

Only because you don't have to maintain it

That's what's so beautiful and elegant about it. When "executed" it evolves, so it maintains itself.

The better you understand the way DNA works the less elegant it looks. It's a steaming pile of Hacks on top of Hacks and frighteningly buggy.

This encodes protean X, however it only folds up correctly 15% of the time. However, Y bumps things up to 70% and Z get's you to 90%. The other 10% well that depends on the shape some of these are used by Z to do... Why do we know this? Well both Y and Z are defective 2-3 percent of the time resulting in... etc.

PS: Don't get me wrong the happy path works well most of the time and when it fails early it's just a non viable embryo so no problem. However, saying it's elegant is like saying all airplanes in the sky must be easy to maintain because you never see anyone outside fixing them.

What you call a bug, with protein's being defective 2-3% of the time, is the most important feature of the biological program.

It's more elegant thank you think.

Can you elaborate?

Your code would evolve and maintain itself too, if you could afford to wait millions of years and let it randomly break for a large portion of your users.

And by 'large portion' we are referring to 99.999999% of them

"the brain may very well be a quantum computer"

This is the minority view in neuroscience.

> In quantum terms each neuron is an essentially classical object. Consequently quantum noise in the brain is at such a low level that it probably doesn't often alter, except very rarely, the critical mechanistic behaviour of sufficient neurons to cause a decision to be different than we might otherwise expect... —Michael Clive Price

You may be right about that, and I don't think the quantum point is essential to what I wrote above.

But... I do tend to sympathize with the minority view here. The problem is that I don't think the majority are looking at the complete system. Yes, the macroscopic activation and conduction behavior of neurons probably can be modeled classically. But that behavior, as well as things like where the axons and dendrites connect, is governed at the meta level by the genetic regulatory networks and metabolic machinery of neurons and their support cells. All that involves at least thousands of genes and a lot of interactions that may very well extend down to the quantum level.

It's a living cell that grows and changes over time, not a simple gate that can be modeled by an equation. Modeling a neuron like that is like modeling a star as a single point source of light because it looks like that from far away. You can model the way stars look through a telescope like that, but that does not accurately describe what a star is.

"I think it's now possible with a big rack of multi-core machines in an MPI cluster to approach the capabilities of a fruit fly's brain."

This seems so unlikely. Isn't it more likely that it takes a rack of multi-core machines to simulate a fruit fly's brain using our extremely primitive algorithms for approximating intelligence?

In other words, it's a software problem.

You're right.

It's a software problem and a hardware problem. Our hardware is not up to the task, and even if it was we wouldn't know how to program it.

Evolutionary computation and non-von-Neumann architectures such as stochastic data flow architectures might be where to start. We would not write the code. We would build the right kind of architecture and then evolve the code within that architecture.

> We would not write the code. We would build the right kind of architecture and then evolve the code within that architecture.

That sounds more like an approach that I think would produce results.

The key here is 'right' though, what is right?

Look - all the talk of the brain as a "quantum computer" seems to be more full of "bull" than the talk of mind uploading and 700 year lifespans. Quantum interactions happen in all matter that we observe, so they happen in the brain. There is not that much more to it. Neural computation depends on neurons and synapses, communicating with hormones and electrical impulses. If we can model the significant algorithms and feedback loops that determine these interactions, we can model the brain. No quantum computing is needed.

Kurzweil himself would admit that we do not know the details of how all of our technology works. Humans at this point understand very little about atomic physics (we still account for the majority of mass by calling it dark matter and hiding it in a formula constant) yet we can produce atomic explosions. I think that one thing alot of people are missing here is that you only need very little theory before you can apply it. Understanding WHY the theory works is a much harder problem, unfortunately.

As far as I know, Roger Penrose first proposed the idea that quantum effects on neural processes will make computational modeling of said processes impossible. It is highly speculative, and the strongest evidence it's got going for it is that an eminent physicist wrote a book based on it (The Emperor's New Mind.)

Penrose certainly popularized the idea. Kurzweil addresses this issue in his book "The Singularity is Near."

The Emperors New Mind is an excellent primer in physics but it failed to sell the quantum idea of the brain in a convincing way.

I expected a rather stronger argument, especially from someone as high up in the hierarchy of science as Roger Penrose, and after laying such a huge foundation.

Maybe there is one but if there is I haven't found it in that book.

The Emperor's New Mind was (IMO) abysmal, and I was really embarrassed to see someone with such indisputable genius as Penrose put out such drivel. You're very right, the strongest evidence for his theory there is that an eminent physicist wrote on it. That's not a good thing.

Please, if you've subjected yourself to that nonsense, cleanse your palette with The Road To Reality - math/physics is where Penrose shines, and that book is him at his brightest.

http://en.wikipedia.org/wiki/Philosophical_zombie

What if the background stuff that you neglect from your model is where all the interesting stuff happens?

Some people see the philosophical zombie as an argument against consciousness being material at all. I don't really see that, but I do see it as an argument against the idea that you can achieve sentience by emulating a coarse-grained approximation of the brain. I think you would have to either really totally understand the brain or harness evolution and let it build you a sentient being whose structure captures the essence of organic life within whatever its embodiment happens to be. You might end up with something that looks nothing like the brain superficially, but that embodies what the brain does somehow.

What if there is no "really interesting stuff"? What if the stuff in the world was all "regular stuff"?

Could be.

It sounds like I'm making a new-agey argument against reductionism, but I'm not. What I'm arguing against is overzealous reductionism... the idea that you can get a grainy image of something and quantize it and you're done. You might be able to do that in some areas, but in biology you can't get away with that. Very small causes can be just as important or more important in living systems than very large ones.

To avoid the philosophical zombie problem, I think at the very least we need a solid theoretical understanding of the phenomenon that we are trying to capture. We have to know what we are trying to do, otherwise we do not know how to start to go about doing it or whether we have done it or not. If you want to land on the moon you need to know what the moon is, where it is, and where you are in relation to it.

That means that we need:

1) A quantitative, hard, solid definition of life. Right now what we have is a qualitative phenomenological definition of life as it exists on Earth. I'm imagining a definition of life that's as solid as the thermodynamic definition of entropy or enthalpy. Based on what I've read in this area already, I would say that it's almost certain that a definition of life will be stated in terms of thermodynamics. Google Ilya Prigogine and dissipative structures to get started.

2) A definition of some sort for consciousness. Right now we have basically nothing here... not even a qualitative set of criteria like we have for life. We know that we are conscious, and we suspect that at least some other living things are conscious. We do not know whether all living things are conscious or not. We do not know whether the set of all conscious entities is entirely contained within the set of all living entities or whether something could be conscious but not alive.

IMHO dismissing #2 is chickening out. Dismissing #1 is definitely chickening out. Not answering the questions means you don't know where the moon is and you might be landing in New Mexico instead.

DNA is not a program-- it is a molecule, and one that may very well do things at the quantum level that are biologically important.

Sure, but Kurzweil's claim is merely that if DNA can encode everything the brain does with N bits of information, then regardless of how the DNA behaves, we have at least a loose estimate of the level of complexity in the brain. I don't necessarily think that he's right that AI will first be achieved in that manner, but I think I have a bit more faith in human ingenuity than he does, and I think we'll get the software there "by hand" before brain scanning hardware can actually do what he needs it to do.

Of course it's true that the decoding scheme and the dynamics of the end result could provide a whole lot of additional complexity, just like it's possible to build a decompression algorithm that allows us to compress a Linux distribution into 1 byte (put the whole program in the decompressor). But quantum effects or no, it seems extremely unlikely that the physics, chemistry, and biology behind all of this are somehow conspiring, magically tuned to provide an awesomely efficient set of basis functions to encode the algorithms that the brain needs to apply - sure, some parts of the brain's work probably exploit physical coincidences that make the overall job easier, but as far as we know, quantum chemistry doesn't offer a "do intelligence" utility function, that will have to be built up from much smaller sub-units. Far more likely is that the bulk of the brain's function is more or less explicitly coded for somewhere within our DNA.

The brain is not a neural network. It is an interconnected colony of living cells of a variety of types, and it has been shown that all types of cells in the brain are involved in cognition.

"Involved" is a serious weasel word when it comes to the brain, especially when AI is under discussion; everything in your skull is "involved" in cognition, but that doesn't mean it's doing anything particularly important, and it definitely does not mean that every detail of the dynamics is an absolute necessity to obtain Cognition.

But I think we might be arguing different things...

On a related tangent, I've found that many engineers are sympathetic to intelligent design type arguments against evolution. This is because they try to think about biology like engineers and take these machine analogies literally.

[Aside: I don't know too many engineers that buy into intelligent design, but I guess YMMV...]

The mistake that biologists make is that they always assume we're thinking about biology, and that our goal is to understand biological systems.

But when we're talking about AI, the goal is not to understand the brain, it's to figure out how to do something pretty close to what the brain does. Chances are, that's a much simpler goal than figuring out the dirty details about what every cell in the thing does. But it means that we have to be very careful not to get lost in the muck, worrying about the form of the brain's computations rather than the function. This is tricky, because it may be that the way the brain gets things done is not very well abstracted or comprehensible, so it's possible that we'll need to pick the level from which to draw inspiration from the brain very carefully (I think the apparent uselessness of neural nets in strong AI is a good indication that we might need to abstract at some level other than "neuron").

Again, though, Kurzweil has a very specific (and I'd argue, peculiar, at least relative to most people in AI) view on this, and thinks that full scale detailed brain simulations are the way that we need to do this (or at least that this will be the quickest route to the goal). I suspect even he doesn't think we'll simulate the physics of every neuron in detail, though, preferring to instead abstract away the most important bits of functionality.

"DNA can encode everything the brain does with N bits of information"

It doesn't directly encode the structure of the brain. Development is required.

"Involved" is a serious weasel word when it comes to the brain, especially when AI is under discussion; everything in your skull is "involved" in cognition, but that doesn't mean it's doing anything particularly important, and it definitely does not mean that every detail of the dynamics is an absolute necessity to obtain Cognition. But I think we might be arguing different things...

We are arguing different things. My point is that you can't write a little equation for a neuron (which is an entire living cell!) and wave your hands and say "done!"

Kurzweil is way way way too hand-wavey for me.

It doesn't directly encode the structure of the brain. Development is required.

Your Java program doesn't directly execute on the CPU, compilation and interpretation are required.

Totally different thing. The java program encodes each decision and computation that it performs directly.

No a Java program requires an environment to run, so does a brain .

It's just that a brain alters it's behavior and structure as a result of processing inputs (development).

We don't (normally) write Java programs to change their behavior as they process information from their environment during execution.

You're confusing literal translation with development. Here's a very loose conceptual analogy:

Translation: translating Lord of the Rings into Chinese.

Development: hearing a short plot synopsis of Lord of the Rings and writing your own fantasy novel based on the same theme.

Development really does do something analogous to that. It takes a collection of proteins and rules and, through embodiment within the laws of physics, constructs the phenotype. The phenotype contains vastly more information than the genotype, and two identical genotypes will not produce absolutely identical phenotypes.

There is something fundamental about development that we do not understand. This is widely acknowledged in the developmental biology, evo-devo, and evolutionary computation fields. A closer analogy than the loose one above might be some of the behaviors we see with fractals and cellular automata, though development is less deterministic than that.

Evolution and development are somehow related. We don't quite get that either. But both processes add vast amounts of information and both involve adaptation.

Hand waving: the adult brain is the output of running a program stored in DNA for twenty years on a runtime (the physical universe) we can barley understand. To produce a brain from the data stored in DNA, we would need to simulate the runtime environment. To do that without simulating the whole universe, we would need to work out which subset of the universe to simulate. We are far away from knowing what shape that subset takes, let alone what the minimal number of bytes required to simulate it is.

My point is that you can't write a little equation for a neuron (which is an entire living cell!) and wave your hands and say "done!" Kurzweil is way way way too hand-wavey for me.

Yes, we're in full agreement on this, especially the part about Kurzweil - I don't think he advances the science of this at all, and his poppy PR approach rubs me the wrong way, for sure, as do many of his specific ideas.

In particular, I think it's a terrible idea to hinge our hopes for AI on the idea that we'll be able to carry out a perfect enough simulation of a full brain to call it "done". Not only does it seem to be a pipe dream at present, depending on a whole bunch of technological advances that may not come for a long time, it would be almost completely inextensible and teach us very little - once we had such a simulation running, it's unlikely that we'd understand enough about how to modify it to improve upon it.

I don't quarrel with the idea that DNA encodes the brain extremely indirectly at all, including a substantial development process that depends on a lot of things other than the pure genetic code, and I also agree that the dynamics of a brain are very dependent on the details of all sorts of cells, not just an oversimplified logical representation of them. Where I think we start to diverge is that I have issues with the idea that the particulars of any of those processes can somehow "piggy-back" any non-trivial amount of functionality into the system that's not coming from the genetic code.

Here's the clearest way, I think, to state my information-content claim: if we were to randomly change the details of the way neurons function in detail (still leaving them with the same highest-level capabilities), and randomly change the way brain development happens, and randomly change all steps of entire process that leads from DNA->brain (again, subject to the restriction that the whole thing still works), and even - I daresay - randomly change the laws of physics, then...

...if all of these changes still permitted us to write down any string of DNA (or whatever our new version of DNA was) that ultimately resulted in the growth of an intelligent human brain, the amount of DNA required would, on average, be pretty darn close to the amount that we see used today. And that has implications for other implementations of the same logic, namely that if we could find some way to optimize towards whatever solution evolution has found, we could probably end up with a shorter code for it than evolution has because we can allow all parts of the DNA->brain process to optimize for compression whereas the real physical process is largely frozen against change.

That's the sense in which I think Kurzweil has a reasonable estimate, nothing more, nothing less; where he takes it from there is another story altogether. :)

The problem is that DNA is not sufficient. Organisms don't grow from naked DNA in a vacuum; the DNA is always contained within a cell which is enclosed within a more complex bio structure (egg, womb, etc) which may be contained within the parent organism in the case of mammals.

You need all of this information plus knowledge of the various interactions of the different environments to pursue Kurzweil's approach.

The universe of necessary and sufficient information is much, much larger than the 3 billion base pairs in your DNA.

For the moment we seem to have enough trouble modeling (relatively speaking, to the brain) simple proteins reliably.

It's quite a leap of faith to make these statements about the technological future with so little in terms of progress to show in these fields for the last decades. I think the most impressive a-life demos are now almost 10 years old, the best we can really simulate is (drumroll) a cockroach. And personally I think that's a milestone achievement because it means that at least we have a principle that works.

Going through the DNA route to get to a working brain seems to be a very roundabout way of getting there, it will require all of the embryonic mechanisms to be modeled accurately as well as something like the first several years out of the womb before you'd know if you had created something insane or something resembling intelligence.

Assuming you'd recognize it as intelligent even if you succeeded, there may be more ways of being intelligent than we know about.

Absolutely, I never said that I think modeling the brain would be easy (I don't) or that building it up from DNA is anywhere near the best way to attempt AI (IMO it's not).

I simply don't buy the argument that the algorithms of cognition inherit any substantial amount of functionality from their physical implementation, and hence, I see Kurzweil's complexity estimate as somewhat reasonable.

> I simply don't buy the argument that the algorithms of cognition inherit any substantial amount of functionality from their physical implementation

I'm somewhere in the middle on that. I wish for things biological to be clear-cut and deterministic enough that we can fully understand them the way we understand mechanical systems.

But precisely because the brain is encoded in precious little DNA there is some evidence that there is more to it than meets the eye, after all if 50M of gzipped data can encode the whole thing why do we have such a hard time understanding it.

There is enough repetition in there that some died-in-the-wool reverse engineer would have put 2 and 2 together by now if the secret was in the wiring or in some simple algorithm (ANNs for instance).

Apparent order appearing from chaos is a field that has seen some study and the amount of complexity that can arise form simple starting data is quite amazing, witness the mandelbrot set and other fractal forms.

It may be very hard to short-circuit such understanding and to 'divine' the workings of the formula without first going the long way around to understand the whole system rather than the 'seed' from which it grows. This is not simple mathematics where a simple equation on complex numbers gives you the mandelbrot set, it's possibly machinery interpreting an equation with 50 million terms.

In different terms, given a very distorted (dissected) picture of the 3 dimensional mandelbrot set would you be able to figure out the formula that gave rise to it without prior knowledge of the mathematics involved?

http://www.skytopia.com/project/fractal/2mandelbulb.html#epi...

FWIW, I'm not too far away from you on this - I don't think we are going to find a very simple algorithm to do whatever general pattern recognition the brain does. All indications suggest that while there may be such an algorithm programmed in and repeated (a lot more times in humans than other animals), it's far too complex for us to simply read off or guess at.

But it does suggest that we should be considering the functions that such higher level units might have, such that they become more intelligent as we compose them. Easier said than done,of course, especially since it's very difficult to actually observe brain dynamics in any detail.

You need all of this information plus knowledge of the various interactions of the different environments to pursue Kurzweil's approach.

Absolutely, and this is one reason why I think Kurzweil's approach overall is pretty foolish.

But I do think that his information-content claim is within the bounds of reason (I've explained my stance on that many other places in this thread, the gist is that it's highly unlikely that the whole developmental process supplies a huge amount of information "for free", in the same sense that it's unlikely that you'll achieve a 10x compression in code size by using Python instead of Ruby to write a program) and it at least tells us that a solution to the problem of intelligence does exist that doesn't require (for instance) explicit hard-coding of every neural connection inside a brain. In fact, it requires massively less information to specify than that, so there's some hope that in the end we'll be able to come away with reasonable approximations to that algorithm since it must be "fairly simple" (meaning: more complicated than anything we've tackled before, but maybe still within the realm of possibility).

That hints at the fact that AI researchers (at least those that don't buy Kurzweil's ridiculous "simulate-the-whole-thing!" approach) should be looking more into building out small "genomes" into massive structures rather than focusing too much on individual explicitly specified processing networks, because nature is clearly doing something like that, and it seems to work pretty well, letting it achieve a remarkably compact solution, given that evolution tends to produce very bloated code as a rule. There has been tentative research along these lines (http://en.wikipedia.org/wiki/HyperNEAT, for example), but there needs to be more, particularly to figure out what sorts of things we actually want these massive structures to do, what sorts of processes we should be focusing on to control that building process, what types of units we need in order to make the processing that we're doing feasible, etc.

if DNA can encode everything the brain does with N bits of information, then regardless of how the DNA behaves, we have at least a loose estimate of the level of complexity in the brain

Non sequitur.

Solution to unified field theory. That's about 32 bytes of information, uncompressed. There, since that statement is so low in information content, it must be easy to find. We have a loose estimate of it's complexity with my statement, right?

The problem here is that an adult brain has a much higher Kolmogorov complexity than the DNA that codes for its parts.

http://en.wikipedia.org/wiki/Kolmogorov_complexity

Sure, and a shag rug has a much higher Kolmogorov complexity than the instructions for how to weave it. If measured carefully enough you'd find every individual strand of yarn in it has a unique length, orientation, curvature, amount of twist and amount of fraying that can be measured to an arbitrary degree of precision. So if you wanted to reproduce or store a description of that exact rug, you couldn't do it. That doesn't mean you can't make another rug that's just like it or close enough to serve the same purpose and be recognizable as a shag rug.

Like that rug, the brain contains oodles of complexity that doesn't matter at all for our purposes.

There is a huge part missing here though. A rug simply has to 'be', a look alike will suffice. A brain has to work.

The question on the table, I think, is whether a brain that's pretty darn similar to a working one will still work or not - does it just have to have similar statistical properties, or are the explicit details of the connections important, does the chemistry have to be just-so, if we replaced neurons of type X with pseudo-neurons of type Y that do almost the same thing when viewed at scale Z but are completely different underneath, would it still work, etc.?

A lot of these questions are completely unanswered. We do know at least that the brain is extremely resilient to changes in chemistry and can work quite well even in the face of extensive damage, which is an indication that we might be surprised how little of the overall arrangement is actually necessary to keep it working properly.

We need to make sure that we're being careful with our language, too: the complexity necessary to specify any brain is a whole lot lower than the complexity to specify one particular brain. In AI the goal is the former, but for the latter, we really do have to worry about each strand in the carpet. An AI researcher might not give a rat's ass about reproducing your memories, and will be more than happy to construct something that can form any memories; on the other hand, to you, your memories (and the detailed wiring inside your head, which we might be able to alter substantially without "breaking" the brain) are vitally important.

Absolutely true.

That leads to 'seed AI', http://en.wikipedia.org/wiki/Seed_AI when run in reverse, so you'd have to implement the 'minimally viable self improving brain', and take it from there.

And even there maybe not all the strands in the carpet have to be just so, but it may very well be that there have to be certain amounts of each colour in roughly such and such a pattern with interconnects between these larger groups and so on. Some of that information is known but definitely not all of it.

The damage angle is a tricky one, some damage seems to be absolutely not problem at all, even if it is major, in other cases the smallest bit of damage seems to be enough to cause terminal failure. There are a lot of clues in there about the organization of the brain.

if DNA can encode everything the brain does with N bits of information, then regardless of how the DNA behaves, we have at least a loose estimate of the level of complexity in the brain.

No even close. Imagine a kilobyte filled with alternating zeros and ones. We have two commonly used methods for describing the information content of this kilobyte. Shannon information content is closely approximated by today's compressors - and 1k worth of alternating zeros and ones compresses to next to nothing. Kogorov Complexity is the length of the minimum necessary program to produce the 1k worth of zeros - again, a simple loop, incrementing a pointer then dereferencing the pointer to fill the address with alternately a zero or a one suffices - a mere handful of bytes are necessary.

Now imagine that the RAM chip holding the zeros and ones is hit by a bunch of cosmic rays , flipping about 10% of the bits. Both the size of a compressed version of the new kilobyte, and the Kogorov program to generate the new sequence are both going to dramatically increase in size.

Now, the thing is that DNA produces a brain to a fairly homogenous pattern - it's the equivalent of my series of alternating zeros and ones, although admittedly it's a bit more complex than that - this is to be expected, there's more Shannon information in DNA for encoding a brain than there was in my simple pattern of zeros and ones.

The cosmic rays are the equivalent of a brain learning, encoding information by weighting connections between neurons. This process massively increases the amount of information stored in a human brain, but this information must be copied if you actually want the copy to behave like the original. I would expect this amount of information to be several orders of magnitude bigger than that found in the DNA, which is why Kurzweil's claim is just completely off the wall.

We're arguing different things. You're talking about the information content required to reproduce a particular brain, which I agree, has a massive Kolmogorov complexity (even if we decide, as we might want to, to factor out many equivalence classes based on behavior). Far more than a DNA estimate would hint at.

But I'm just talking about (as are most AI people, and I think Kurzweil, as well) the informational content required to build a brain. Any old brain, could be yours, could be mine, is probably neither. The information required to do this is much lower, and corresponds loosely to your first example, a kilobyte with alternating ones and zeros.

I recommend reading "The Singularity Is Near".

Sure, maybe Kurzweil doesn't understand the brain on any deep level and indeed maybe even those who understand the better than him don't understand it well enough at this point.

But Kurzweil's basic argument really isn't about that, it's about the exponential advance of tools and technologies and understanding on multiple level. Will the blue brain project succeed? Will some lesser known project succeed? Will the process take twenty instead of ten years? All unknown but not crucial to the implications of exponential change. When you have tool that are improve exponentially, what you can do tends to improve also. And then the whole process builds on itself. Do I know where this will go? No but I don't you do either.

Your post is remarkably obtuse. By your same logic I can say a program is not executing code, it is a series of electrons moving down different circuit paths. Also true, but such behavior can be described at a higher level as "executing code."

Take another example, your post could be considered a message, or it could just be considered electrons emanating from an LCD monitor and getting received by a brain, which in turn generates certain behaviors in a large meat machine. Also true, but your post can also be described as a message about the need for thinking about reality literally instead of abstractly in order to understand truth.

However, if we can only access truth by talking about the world literally, then no one, ever, has access to truth. Even today we cannot describe the world literally, at all. An electron is merely an abstract representation of even more fundamental quantum physic dynamics. And, who's to claim quantum physics is really at the bottom of the reality stack? For instance, we have no idea why waveforms collapse to their particular determinate. There is something even more fundamental behind this phenomena we currently have no clue about. Therefore, by your criterion for truth, everything discussed and thought about throughout history has been nothing but gibberish, including your own post.

So, clearly you do not even agree with your own claim, since you seem to think you are communicating something to us.

I call BS.

relevant to your comment is POSIWID!: the Purpose Of a System Is What It Does

That makes the term "purpose" rather useless, doesn't it?

If you think about it, that's evolutionary theory in a nutshell. Organisms don't evolve due to any desire or goal to change; rather, random processes cause mutations, and traits that happen to result in more copies of themselves being made tend to become more common, and that's it. There is no purpose to the changes; they just are, and "better" ones happen to become more common because they duplicate more.

I would argue that this just means that some things and processes, like evolution, do not have a purpose, rather than that their purpose is whatever actually happens. If purpose just means result, then you've defined purpose such that it no longer applies to lots of systems we actually already understand the purpose for.

So much nonsense in one place ...

>>Biological systems are nothing like anything we would ever engineer

You really believe, you can predict what human technology will look like in 50, 100, 10000000 years?

>>DNA is not a program

A hard disk platter is not a program.

>>Nature has had billions of years, and it is way ahead of us.

That's why birds are so much faster than planes.

>>Cells are not machines..."stochastic quantum probability field device."

All modern computers are quantum machines.

>>Biology is quantum-scale nanotechnology

And so is today electronics.

>That's why birds are so much faster than planes.

wtf? As if the only goal for flying is to go as fast as possible. A bird can land on a branch that can hardly support its weight while the wind is blowing said branch.

Reply on news.ycombinator.com