Realistically what actually ends up happening imo, we get human level AGI and hit a ceiling there. Agents replace large portions of the current service economy greatly increasing automation / efficiency for companies.
People continue to live their lives, as the idea of having a human level AGI personal assistant becomes normalized and then taken for granted.
That is a hard limit on intelligence, but neural networks can't even reach that. What is the actual limit? No one knows. Maybe it's something relatively close to that, modulo physical constraints. Maybe it's right above the maximum human intelligence (and evolution managed to converge to a near optimal architecture). No one knows.
As far as we know, it's impossible to model any part of the universe perfectly, although it usually doesn't matter.
One million Von Neumanns working on that 'problem' is not something I'm looking forward to.
- A simple calculator can beat any human at arithmetic
- A word processor with a hard disk is orders of magnitude better than any human at memorizing text
- No human can approach the ELO of a chess program running on a cheap laptop
- No human polymath has journeyman-level knowledge of a tiny fraction of the fields that LLMs can recite textbook answers from
Why should we expect AGI to be the first cognitive capabilities that does not quickly shoot past human-level ability?
Changed my mind. Think you’re right. At the very least, these models will reach polymath comprehension in every field that exists. And PhD level expertise in every field all at once is by definition superhuman, since people currently are time constrained by limited lifespans.
> People continue to live their lives
Presumably large numbers of those people no longer have jobs, and therefore no income.
> we get human level AGI and hit a ceiling there
Recently I've been wondering if our best chance for a brake on runaway non-hard-takeoff superintelligence would be that the economy would be trashed.
Half-joking. In seriousness, something like a UBI will most likely happen.
It's impossible to run out of work for people who are capable of it. As an example, if you have two people and a piece of paper, just tear up the paper into strips, call them money and start exchanging them. Congrats, you both have income now.
(This is assuming the AGI has solved the problem of food and stuff, otherwise they're going to have to trade for that and may run out of currency.)