"Indeed, focusing on this particular threat might exacerbate the more likely risks. The history of technology to date suggests that the greatest risks come not from technology itself, but from the people who control the technology using it to accumulate power and wealth. The AI industry leaders who have signed this statement are precisely the people best positioned to do just that. And in calling for regulations to address the risks of future rogue AI systems, they have proposed interventions that would further cement their power. We should be wary of Prometheans who want to both profit from bringing the people fire, and be trusted as the firefighters."

I've been saying something like that for a while, but in less Olympian terms.

Look at this first as a consumer protection problem. I've pointed out before a Frontier Airlines presentation directed to investors which specifically calls out AI chatbots as a tool for suppressing customer complaints.[1], page 45: "Today, high-touch. ... Avenue for customer negotiation. Tomorrow, self-service. "Chatbot efficiently answers questions, reduces contacts and removes negotiation." The coming thing, already here for some companies, is customer service where customers can never reach a human.

We already see this with the big ad-supported companies - Google, Facebook, etc. They have the power to make major adverse decisions to a user with no consequences to the company. Few governments have the guts to stand up to that. The EU does, a little.

Government doing this sort of thing is a similar problem, and governments are harder to avoid.

The extinction worry is a distraction from the oppression worry.

[1] https://ir.flyfrontier.com/static-files/c7e0a34d-3659-49cc-8...