Why Computers Won’t Make Themselves Smarter
newyorker.com
newyorker.com
There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity, but I think it's reasonable to believe that 1 - agi is possible 2 - said agi can participate in human society to improve its own design
An AGI that made a few decades of current-rate scientific progress could switch to a globally distributed quantum computer that’s synchronized in its computation through entanglement.
At which point, it would hit an exponential curve until it ran out of space to grow. (The “singularity” proponents argue this is unbounded.)
An AI that’s 10x smarter than us would be 2^10 smarter than us in a couple years — based on current technology trends. Moreover, it will have overcome the cooperation barrier with a distributed consciousnes, like the original idea for the Borg.
The ability to free itself from a single location or easy disconnection from its assets would be a major step in AI independence.
A reasonable prediction is humans will implement quantum computing in 10-20 years, for special purpose data center operations.
A “super intelligent” AI might reasonably do that in 2-5, at which point the quantum ML gains would allow it to enter exponential growth. We could see an AGI go from “mildly smarter than human researchers” to “incomprehensible” in just a few years.
I’m unsure which step in that chain of reasoning you think is inaccurate.
You just said “to be honest”, implied I was uninformed because I disagree with you, then said something concurring with me: Zuse’s work happened a decade before ENIAC (1936 vs 1946). If quantum is at the equivalent of Zuse’s work, AI could suddenly accelerate it and go exponential.
Go ahead, get scientific.
There is no scientific reason why global quantum communication means that an AI hits exponential growth. (If there is, you sure haven't supplied it! You've just asserted that it is so.)
I understand a number of readers aren’t familiar with QAI research though.
But that’s my point —
> Why should we get scientific? You didn't.
The person responding made a generic negative comment (as did you) due to not knowing the state of the art.
Incorrectly “correcting” people while yourself ignorant is prevented by giving people specific corrections: if you’d taken the time to tell me what you think I’m wrong about, you’d probably realize you don’t actually have a specific thing you think I’m wrong about — you just were surprised by my conclusion.
See here for more detailed arguments:
If you were really smarter than me at QM, you’d be able to point out what was wrong with what I said.
The HN downvotes without correction really speak to the hubris around here.
We can conservatively say that even though there is a limit to amount of computation produced in a given volume, temperature range, and time, this limit is still sky-high compared to what we do now. OTOH the speed of light is already a major limiting factor.
There’s a number of facts that make this not sci-fi:
- China has demonstrated distributed quantum networks.
- The US has demonstrated distributed quantum calculations.
- Anyons were recently experimentally verified.
- We have a number of small quantum ALU.
- There are quantum-effect systems in production, just not fully-entangled systems.
- We have quantum ML algorithms for accelerated training.
Nobody has disputed my baseline estimate: that humans will complete a quantum computer in 10-20 years. (If people just disagree there, they should say so.)
However, once you accept that, it’s trivial to see the following sequence:
- AI is slightly smarter than humans, does quantum work 2-5x faster.
- AI develops quantum circuits in 2-4 years that would have taken 10-20 for humans. (See COVID vaccines for power of tech acceleration.)
- Those quantum systems let the AI replace its current systems with faster local ones.
- The AI uses another 1-2 years at this intermediate stage to extend current quantum networking research to unify its quantum data centers into a single computer, giving full global entanglement.
So in 3-5 years, an AI which starts off mildly smarter could finish nearly-done projects and enter an exponential growth curve before we do.
That progression looks like:
- Scale M AI.
- Scale M/K * 2^K, where K is the portion of M at a single data center.
- Scale 2^M AI, when it can fully entangle.
> Because of that, any level of detail is about as believable as any sci—fi.
That’s not true: most major companies in the space have labs looking to turn fundamental research into products, which is the exact opposite of sci-fi.
I think people just haven’t actually thought about this and are responding with fallacies rather than confront an idea they intuitively reject.
Even though I’m right on the technical issues involved.
The central limit theorem shows that the normal distribution emerges as the limit of the average of independent and identically distributed random variables of any distribution.
There are reasons to believe the opposite, that there's a limit. Not just practical technological limitations (e.g. limits to CPU process, speed, power consumption, etc) but also absolute limits for information processing under certain physical constraints (in the Shannon sense).
with strawmen like these, who even needs a debate?
So the way to have a 300 IQ would be to sandbag the rest of the world. Give them all smartphones for instance (or irrigate their crops with brawndo)
Also,
> I.Q. represents the idea that intelligence can be usefully captured by a single number, this idea being one of the assumptions made by proponents of an intelligence explosion
I have never heard this assumption in a technical discussion in the modern age, and the author provides no source here. I'm overall unconvinced by this article.
Dogs probably have higher emotional intelligence (by human standards) than wolves, and we probably could breed dogs (or have we?) that are smarter at basic puzzle solving activities like opening latches with their paws or whatever.