Remember that AI work with electrons, and we are of atoms. We should focus on where electrons control atoms, and reduce those points. Of particular concern is that AI may be a very strong investor with the right prompts. AI could also figure out how to use any other software. Which can be used to gain access to any marketplace, including the dark ones. Which means AI can use money (electrons) to pay others to modify the world (atoms).

Of course, there is already a problem, as you point out. Humans shouldn't have access to these markets either!

But yeah to specifically prevent electron-on-atom violence we need to limit AI physical degrees-of-freedom. by limiting marketplaces. National/global security, not personal morality, should guide these new regulations.

We need to end all drive-by-wire automobiles, and electronic locks. Too many services are habituated to act on electronic signals without human confirmation - particularly the police. There needs to be an unbroken verbal chain between the person who saw an event and the person doing the law enforcement. Breaks in the human chain should be treated very seriously -they should be treated as firing offenses, at least. There are many other similar changes we're going to need to make.

Some folks aren't gonna like this. Regulations are inherently evil, they say. Maybe the mental model should be more like we're the big bad gorilla in the cage. But now there's a tiger in the cage. Regulation restrains the tiger. Also, some folks aren't gonna like it no matter what change you need. The fact of not liking it doesn't mean we don't need it, and it doesn't mean it won't get done. We have to trust that our leaders don't want to die.

And besides, the world will adapt. It always does. AI isn't optional, there's no putting the genie back in the bottle - and personally I don't want to. But I also don't want to be stupid about the stakes. Getting our whole species killed for lack of foresight would be deeply, deeply embarrassing.