> Thus, the future will be unpredictable (if we let superintelligence take control)
As opposed to the predictable future we've had for the past few decades?
As opposed to the predictable future we've had for the past few decades?
If superintelligence emerges soon, we may not even know which technologies will emerge and how many will be unleashed in the next 2 decades.
ADDED: Examples of some concrete predictions:
(1980) https://en.wikipedia.org/wiki/The_Third_Wave_(Toffler_book)
(1995) https://en.wikipedia.org/wiki/The_Road_Ahead_(Gates_book)
Obviously, specifics differ from predictions and plenty of people got it wrong. Many good forecasters got the broad strokes right though.
Which forecasters can even predict most technologies that would be invented after ASI emerges?
The space of possibilities is probably large enough that, given time, ASI can get around most limitations if it has the desire/goal to implement them.
There's a lot of things that we know how to fix, but cannot. It's with thinking through why that is, and whether it's any different for a 'superintelligence.'
This assumes that humanity would remain in control after ASI emerges. We might not.
Suggestive pathways, he replied.