risk of extinction due to AI? people have been reading too much science fiction. I would love to hear a plausible story of how AI will lead to human extinction that wouldn't happen with traditional non-AI tech. for the sake of conversation let's say non-AI tech is any broadly usable consumer technology before Jan 1 of 2020.
The proposed FOOM scenarios obviously borrow from what we already know to be possible or think it would likely be possible using current tech, given an proposed insanely more intelligent agent than us.
'Able to play chess'-level AI is the greater concern, allowing humans to create more unavoidable tools of war. But we've been doing that for decades, perhaps even centuries.
err what? Apparently there are 1 million species under threat of human caused extinction [1].
[1] https://www.nbcnews.com/mach/science/1-million-species-under...
But that’s just as much or more of an argument for regulating the tools of synthetic biology.
You don't think than an intelligence who would emerge and would probably be insanely smarter than the smartest of us with all human knowledge in his memory would sit by and watch us destroy the planet? You think an emergent intelligence was trained on the vast human knowledge and history would look at our history and think: these guys are really nice! Nothing to fear from them.
This intelligence could play dumb, start manipulating people around itself and it would take over the world in a way no one would see it coming. And when it does take over the world, it's too late.
- Taking control of robotic systems
- Manipulating humans into actions that advance its goal
- Exploiting and manipulating other computer systems for greater leverage
- Interaction with other technologies that have global reach, such as nuclear weapons, chemicals, biological agents, or nanotechnology.
It's important to know that these things don't require AGI or AI systems to be conscious. From what I can see, we've set up all of the building blocks necessary for this scenario to play out, but we lack the regulation and understanding of the systems being built to prevent runaway AI. We're playing with fire.
To be clear, I don't think I am as concerned about literal human extinction as I am the end of civilization as we know it, which is a much lower bar than "0 humans".