Comment by Cybiote | Hacker News Reader

Comment by Cybiote | Hacker News Reader

Parent Full thread

This is an excellent post smallnamespace. The area of research you're looking for is known as evolvability, reading these wiki/wikibook [↓] links is a good start for the topic.

Evolution seems to have figured out a way to build representations such that nearby steps in random directions are nearby in a semantic (can't think of a better word atm) sense. This should surprise you. Think about an expression tree like (+ (x y)), it's very different from (/ (x y)) despite differing in just 1 position. Or consider that 12 and 32780 differ by only a bit in their binary representation. We humans could really do with figuring out how evolution takes a seeming discrete problem and works out a way to build representations such that efficient search is possible even without explicit derivatives.

It's also interesting that, evolution might have taken this to a meta level with brains. From http://www.fil.ion.ucl.ac.uk/~karl/The%20free-energy%20princ...

> The beauty of neural Darwinism is that it nests distinct selective processes within each other. In other words, it eschews a single unit of selection and exploits the notion of meta-selection (the selection of selective mechanisms; for example, see REF. 89). In this context, (neuronal) value confers evolutionary value (that is, adaptive fitness) by selecting neuronal groups that meditate adaptive stimulus–stimulus associations and stimulus–response links. The capacity of value to do this is assured by natural selection, in the sense that neuronal value systems are themselves subject to selective pressure. This theory, particularly value-dependent learning90, has deep connections with reinforcement learning and related approaches in engineering (see below)

A bit of an aside but: If you're interested in AI, it's really worth looking at what the people studying the intersection of evolution and learning are saying, at animal intelligence and at human brain development. It's surprising to me how many people think the current approach to reinforcement learning is but a hop and a skip to full human like generality.

[-] https://en.wikibooks.org/wiki/Structural_Biochemistry/Stabil...

https://en.wikipedia.org/wiki/Evolvability

Going back to basics to get my thoughts in order:

Synonyms are words with the same meaning. A way of descibing this is that the mapping from the set of all words, to the set of all meanings is not "injective" (aka "one-to-one") - i.e. at least some two words map to the same meaning. There are other ways to describe it, one might say synonymous ways...

Synonyms also occur when two expressions have the same value. e.g.

  2+2 = 2*2 = 4-2

It's also possible with variables e.g.

  x+y = y+x
  x+y+z = (x+y)+z = x+(y+z)
  a(b+c) = ab+ac

All this is redundant, and we tend to like to minimize redundancy, as wasteful inelegance. As in python's There's one way to do it. Though the truth is that there are an enormous number of programs that are functionally identical - synonyms.

Getting to the article's domain, one specific DNA sequence has a number of neighbours that are one mutation away (an Edit-Distance or Levenshtein distance of 1, if you like)... for a long sequence, there are quite a few neighbours. But some of those neighbours are synonyms, and similar for their neighbours, and their neighbours' neighbours. Pretty soon you have an exponential explosion of synonyms. But the crucial point is that you also have an explosion of non-synonyms... an exponential explosion of the surface area of the set of equivalent DNA sequences.

This is where massive parallelization, of having a huge number of organisms, is helpful, many of them being unique points on the surface. Of course, not just for this one piece of DNA, but a different point for each gene.

All this makes it much more likely that a new mutation could generate something new and useful.

It may be that life took so long to get going because it went through many systems that lacked this property... when the most adaptable finally came along (DNA, plausibly preceded by RNA), it completely swamped its ancestors, obliterating them. Then we see other forms that excelled at flexibility/adaptability: multicellularity, sex, birds, mammals and so on.

BTW on the subject of Platonic forms, the surprising thing to me is we have so many clear categories, instead of a huge mush of continuously varying species. They may be local minima... and when a new more flexible form arrives, it again explodes.

We don't like redundancy... but maybe we should.

Thinking more: the tricky property is not only that synonyms are one mutation away, but that functionally very different meanings are one mutation away - enough to make a difference.

For example, in C programs, it's easy to get one mutation synonyms: just mess with whitespace. But difficult to get very different synonyms, which then with one further mutation suddenly gives very different results (I think?). The synonyms don't get you much.

My understanding of the function of DNA coded proteins is pretty much their folding shape (which in turn determines what reactions they "enzymize"). We could say that the mapping between sequence and folding is the mapping from word to meaning. I'm guessing, but it seems to me that a protein could vary much in some areas, without affecting folding much (like whitespace?), some areas would predispose some folds, and others would be critical. It also may be affected by the general environment, and presence of specific other proteins, that might encourage one out of few potential folds.

It maybe that it's also somewhat probabilistic, that proteins are not deterministically folded in one particular way, but produce a distribution of folds. And that as a synonyms may shift the distribution, e.g. more of one base-pair at a certain part may gradually shift the proportions, Provided none of the possibilities are pathological, this seems a way to gradually shift towards alternatives.

The key thing is that when shifted far enough, at a few different folding points, it may suddenly start producing a novel folding. If it happens to be useful, selection would quickly favour stablizing this particular folding (and it's probably possible to move towards a sequence that almost guarantees this fold).

So in some ways it's more an analog system than digital.

I find it difficult to think of an actual design with this property.

There was a good article about this in the April 2004 issue of Scientific American (paywalled, unfortunately; the name of the article is "Evolution Encoded"). It's about how similar gene sequences result in similar proteins, so that mutations don't necessarily result in death.

http://www.scientificamerican.com/article/evolution-encoded/

Reply on news.ycombinator.com