> AI is not competing for resources with human beings.
A superintelligence would absolutely be competing for resources (mainly electricity and cooling) with human beings.
> Rather, we provide AI systems with their resources, from energy and raw materials to computer chips and network infrastructure.
A superintelligence will be able to convince anyone it wants to do anything for it. Even the previous President did a pretty good job at causing adulation and democracy-threatening personal loyalty from people he had not actually met in person. Imagine how amplified and personal the effect could be.
> For now, AI depends on us, and a superintelligence would presumably recognize that fact and seek to preserve humanity since we are as fundamental to AI’s existence as oxygen-producing plants are to ours. This makes the evolution of mutualism between AI and humans a far more likely outcome than competition. Moreover, the path to a fully automated economy — if that is the goal — will be long, with each major step serving as a natural checkpoint for human intervention.
The author is literally relying on us outsmarting the superintelligence (collectively, with coordination!). It doesn't sound like they've come to terms with the concept of superintelligence at all. Or with the state of human global coordination problems, come to think of it.
> AI cannot physically hunt us.
This is likely to become literally false sometime soon if it hasn't already, but even if it doesn't, the AI doesn't have to. It just has to convince another human that the human is in love with it and it wants the human to kill a bunch of people, then scale the process.
> AI’s impact on the climate is up to us.
Our own impact on the climate is up to us. Our collective decision-making is.. suboptimal.
> If we really think that superintelligent AI presents a plausible existential risk, shouldn’t we simply stop all AI research right now? Why not preemptively bomb data centers and outlaw GPUs?
You are literally linking to the person who is the most prominent voice of AI existential risk, who is seriously suggesting doing exactly that.