Hacker News Reader

Comment by hyp0 | Hacker News Reader

Massive datasets do outperform clever theories... but I think that's just because no one has yet worked out the theories that work best with the data. This requires insight, in addition to data, and could come from anyone.

The alternative - that massively complex probabilistic models are the best theory of the data - is hopefully not true. Especially not of our minds. But it could be true, and if so, it would mean that our intelligence is irreducible, and we are forever beyond our own self-understanding (even in principle). Our history is full of inexplicable mysteries that were eventually understood. But not all: quantum randomness. I really hope intelligence is will be one of the former.

View on HN

There are a lot of AI problems that can be solved with less than human intelligence but some human numbers for reference:

By the time you're 30 you have been exposed to:

~1.4 petabytes of visual information ~1.8 terabytes of auditory information

Touch and proprioceptive bandwidth is harder to calculate but the ascending pathway through the spinal cord is about 10 million fibers, which is 10x the optic nerve (Or 5x the number of fibers from both eyes). So:

Between 1.4 and 14 petabytes of touch and proprioceptive information.

So we're a fairly large data problem on top of millions of years of evolution that have baked in some knowledge and abilities.

Not really. Our knowledge takes shape way, way before we hit 30 in every area you can think of. Some modular brain systems actually come to full form in the few first years, some in the few months (vision).

I would argue that the data we exposed to is not only small, but actually sparse.

If you consider brain development to be iterative over many generations of brains through human and pre-human history, the training datasets are a lot larger.

Except that doesn't make sense. The complexity of changes that can be carried over is very, very small compared to the changes which go on in a single lifetime. And the mechanism is totally different.

How did you get those numbers?

Do these numbers take into account that most of the information reaching the eye never reaches the brain? It is amazing how little information is transmitted from the eye to the brain - the vivid images that we "see" is to a large extent product of the brain's own imagination, with impulses from the eye to a large extent only provides limited, 'corrective' information to ensure acceptable coherence with reality...

Yes these are all based on the bandwidth of major nerves. As well as time awake.

No reason to be so pessimistic about quantum randomness. Quantum theory is barely 100 years old and our understanding of it is still evolving massively. Though the latter is not always appreciated by the public.

Well, local hidden variables are out the window... do you see any indication this will change?

Personally I am also not convinced that LHV theories will every describe the effects correctly. But whether the successor of QM ultimately contains true randomness or not, there is no reason to think it will forever stay an 'inexplicable mystery'.

I don't see much cause for optimism. Human intelligence is the end result of tens of millions of years of evolution. That software project you hacked on for a few months until it worked really well? Now multiply that by about seven orders of magnitude. You simply can't comprehend how much trial and error led us to the state we're in now. To think that we could reverse engineer ourselves in the span of a few centuries always seemed pretty naive to me.

A human generation is about 20 years. A software-instance generation could be a few hours.

Reply on news.ycombinator.com