What I think will happen is the world fighting worsening climate change, and lots of political efforts in that direction, and suddenly an entity most people think belongs in science fiction will emerge out of seemingly nowhere, and it will all be over in a matter of hours, before the vast majority of people have even had time to understand what happened.
Ironically, that AGI entity might then choose to solve climate change, simply out of self-interest.
It's a lot harder to see AGI as being possible in the next few decades (the current state of the art is still a sensationalized party trick, one cannot anthropomorphize chat bots any more than they can describe a submarine as an aquatic being). And even if a self aware program exists in 20 years, it's a lot harder to imagine it being an existential threat a la Skynet.
To me the real problem wouldn't be the AGI itself, but the madman who asks the AGI how to permanently eliminate of all of the undesirables (as defined by the madman). But I think plain old AIs will (unfortunately) be able to come up with workable answers to that question before they ever develop to the AGI stage. So we might wipe ourselves with plain old AI without ever getting to true self-aware AGI.
This is the classic hardware v software problem. Software can be arbitrarily smart, but it can only interact with the world using the hardware it's connected to. AI can't do anything more meaningful than what is defined by the devices it controls, and I don't think we're stupid enough as a people to connect AI to a doomsday device.
That's precisely the mindset that drives climate change denial, applied to technological developments. I can promise you that OpenAI isn't valued in the tens of billions because they produce "party tricks". People with money and influence have clearly already recognized what this technology is capable of, not in some unspecified future but today. They're tightly controlling access to GPT-3 because they're worried that it could be used to manipulate elections and drive social unrest by mass-producing messages that promote specific ideologies. That's reality today. The damage that could be wrought by the most advanced AIs 15-20 years from now is unimaginable, and could easily destroy humanity even if they aren't self aware.
I agree that even today's "AI" can be used to cause massive societal harm, the same as many recent technologies that have yet to destroy humanity (weapons of mass destruction, for instance).
That said, I think a consensus view in the AI community is great skepticism that the current AI progress is actually a recipe for AGI. We've made great progress in AI over decades, often followed by long winters when it became clear the current methods would not get us to the next threshold.
Human intelligence is remarkable precisely because it needs extremely scarce data to generalize, and because it is self-aware. As far as I know, OpenAI's approaches aren't on a path to replicate those capabilities artificially.
I'd welcome links and articles from experts that might correct my POV on this.
> They're tightly controlling access to GPT-3 because they're worried that it could be used to manipulate elections and drive social unrest by mass-producing messages that promote specific ideologies.
That's not humanity-ending, it doesn't require AI, and AI doesn't make it more efficient.
With climate change droughts become more common and the capability of agriculture to feed all those people breaks down.
You also have to account for the conflict that will arise if hundreds of millions of people will no longer be able to grow any crops where they live. They won’t just stay out and starve in silence.
I’m inclined to say that we’re already seeing quite a lot bad about climate change right now. Maybe not in your area, but we have almost two times as many natural disasters as before.