The impossibility of intelligence explosion

Discussion of the original article on HN: https://news.ycombinator.com/item?id=15788807

Quote: "An overwhelming amount of evidence points to this simple fact: a single human brain, on its own, is not capable of designing a greater intelligence than itself. This is a purely empirical statement: out of billions of human brains that have come and gone, none has done so. Clearly, the intelligence of a single human, over a single lifetime, cannot design intelligence, or else, over billions of trials, it would have already occurred."

What an incredibly inadequate argument. Obviously, none could have done so in the past. Because the required resources simply weren't available. Breakthroughs are being made right here in this era, with Jeff Hawkins' being one of the latest.

This guy reminds me of Richard Smalley, who, against better judgment, attempted to make irrational arguments against molecular nanotechnology, as envisioned by Drexler.

Also, he can't be a software pioneer. He's too damn young.

"When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong." ~Arthur C. Clarke

There is an estimated 100 billion nuerons in the human brain. We know from information theory that DNA is not nearly long enough to describe all of those connections. This means that most of the complexity of the brain is emergent.

I envision bundles of neurons that begin their existence mostly disconnected... These bundles are described somewhere in DNA (but not their ultimate learned configuration) and so is whatever learning algorithm drives this process.

We will have AGI when we understand the underlying learning algorithm our brain uses... Personally, I believe that algorithm involves fantasizing future scenarios and learning from the fantasies.

"Feynman reported 126, James Watson, co-discoverer of DNA, 124 — which is exactly the same range as legions of mediocre scientists."

Can we stop the Feynman had an IQ of 126 narrative? Many people doubt his IQ was in the 120 range. IQ scores, despite what people thing, are not always accurate. Feynman is an example of such a case. Plus, you'd typically report a range on IQ exams. Given a 5+/- point range that could place him in the 131 category, which still feels too low for someone as brilliant as he was.

>"Feynman received the highest score in the country by a large margin on the notoriously difficult Putnam mathematics competition exam, although he joined the MIT team on short notice and did not prepare for the test." [0]

Someone that does that, most likely does not have a 126 IQ. Given his accomplishments in Physics, it is likely he is more intelligent than any IQ test gave him credit for.

[0] https://www.psychologytoday.com/us/blog/finding-the-next-ein...

And yet general biological intelligence arose from unintelligent forces? Hm, interesting.

I don't know enough about general artificial intelligence, but his claim seems to hinge a lot on what we already know. There are things we may not know yet that will lead to us generating general artificial intelligence. Although I don't think it'd be required, it may turn out that the experience of consciousness isn't something we can mechanically produce but is a property of matter itself. Now, I can't prove that is so but it's a fun little hypothesis I enjoy thinking about. None the less, there's no particular reason to disbelieve why humans couldn't figure out similar knowledge and apply it to creating better artificial intelligence.

Some of his reasoning is just flat out fallacious:

> An overwhelming amount of evidence points to this simple fact: a single human brain, on its own, is not capable of designing a greater intelligence than itself. This is a purely empirical statement: out of billions of human brains that have come and gone, none has done so.

Yet out of all those billions of brains, how many of them have tried developing general artificial intelligence, let alone the "AI" of an if-then-else statement? Probably 0.00001% of those billions of brains. Yes, I made that number up, but it wouldn't be surprising for that number to be minuscule, below a fraction of a fraction of a percent.

Evolution has had billions of years. We've only had computers for less than a century. In that respect, it's not at all surprising that humans haven't mastered what nature unintelligently developed for much longer.

> In particular, there is no such thing as “general” intelligence. On an abstract level, we know this for a fact via the “no free lunch” theorem — stating that no problem-solving algorithm can outperform random chance across all possible problems.

That depends on how you define the word "general". I don't think that scientists and engineers working on intelligence are actually using the word "general artificial intelligence" in the sense that such intelligence could solve literally any problem. Humans are generally intelligent within a set of fixed domains, but that doesn't mean they don't have general intelligence within certain constraints.

Our brains already do it, likelihood is 100%. Nothing would stop transistor based computers from doing it, though they might be very much slower at it.

It seems more likely more fundamental innovations are going to come from academia with a culture of seriousness and decades of work without returns.

On this side there is a strong tendency to be blinded by hype, underestimate problems or wish them away with naive desperation that things will just happen based more on hope than reality.

We saw that with the self driving crowd underestimating the problems and overestimating the existing technology, and now with the AI folks, who are already guilty of perpetuating hype knowing fully well that their extremely limited definition of 'AI' is nothing like what the world understands by AI.

All the privacy shenanigans, crypto disaster, self driving cars and AI leave the reputation of tech community in a very bad place.

I too have some resistance to the notion that we will see AGI at a higher level than humans within 50 years. I don't think the arguments he has make any sense. The technology simply did not exist and we're improving on so many fronts mostly because we're now capable of yielding huge neural networks we couldn't fathom having access to just a few years ago.

I don't think the technology required to build superhuman AGI exists yet at least at a reasonable cost to humanity. I don't think we will reach this level of massively parallel computing required for Superhuman AGI until maybe 2070 and that's only if we bring together all the world's supercomputers together in one giant artificial brain.

To make an accurate projection or statement about something, it is generally necessary that you have a sound understanding of it. This is just my general intelligence speaking here. I see nothing of note that reflects that this person has a fundamental understanding of General Intelligence or has even been in the pursuit of it. Authoring an optimization algorithm suite doesn't mean you have an understanding of General Intelligence. A non-read before even clicking on the link. Two posts today about AGI. Seems the next hype train is arriving right on schedule and its banner will be (AGI).

the meat computer hidden in your head running on ~20W of power does it. why wouldn't an electronic computer be incapable of the same feat is beyond me. make it 20kW or 20MW if 20W sounds unreasonably low.

> There is no evidence that a person with an IQ of 170 is in any way more likely to achieve a greater impact in their field than a person with an IQ of 130.

This feels disingenuous. If other things are bottlenecking human success than intelligence (which isn't even entirely true), computers are generally better at these things, especially in an era of massive low-cost computing resources.

We have an example AGI - the human brain.

We have an example superintelligence - humans working together in groups such as corporations are a superintellgence.

So from that, the invention of a human-level AGI naturally leads to superintelligence - lots of the AGIs cooperating.

The question then becomes of whether we're going to get to AGI. Evidence points to yes. AGI (human intelligence) is a collection of abilities and the machine learning community is steadily making progress on them. Speech, vision, machine translation, question answering, summarization etc. are all being worked on and steady progress, or in many cases - rapid progress, is being made.

Unsupervised learning and reinforcement learning are the frontiers and both have advancements in just the past couple of years (GANs, predictive learning, inverse reinforcement learning, imitation learning, domain randomization).

Unsupervised learning in particular is likely the key to AGI and only recently has significant progress been made on it - predictive learning (as Yann Lecun calls it).

(Personal conjecture - the next couple of years when a larger number of people investigate predictive learning might lead to AGI - in maybe just 2 years).

I don't have the exact quote in front of me, but in 1979, Douglas Hofstadter wrote in his book "Godel, Escher and Bach" something to the effect of: "It may be that some day a computer can beat a human at chess, but when it happens, it will be by the sort of computer that will say, 'no, I'm bored with chess, I would rather talk about poetry'". If Hofstadter can be wrong about anything, anybody can be wrong about anything.