Most of the things people are worried about AI doing are the things corporations are already allowed to do - snoop on everybody, influence governments, oppress workers, lie. AI just makes some of that cheaper.
Politicians that point fingers at other politicians being corrupt or incompetent while they themselves are exactly that use the same strategy.
Power and manipulation. Nothing new under the sun. What’s new though is that we can see in plain sight how corporations control politics. Like literarily this can be documented with git commit history accuracy: thousands upon thousands of people repeating the exact same phrases defending openai and the “revolutionary” product, fear mongering, political lobby, manufactured threats and of course a cure that only they can provide and so on. I would not let people that use such tactics near an email account let alone ai policy making.
A snowball probably isn't harmful unless you do something really dumb.
A snow drift isn't harmful unless you're not cautious.
An avalanche, well that gets harmful pretty damned quick.
These things are all snow, but suddenly at some point scale starts to matter.
I think in a lot of ways we all struggle with the nature of some things changing their nature depending on the context and scale. Like if you kill a frenchman on purpose that's a murder, if you killed him because if he attacked you first it's self defense, if you killed him because he was convicted of a crime that's an execution, if you killed him because he's french that's a hate crime, but if you're at war with France that's killing an enemy combatant, but if he's not in the military that's a civilian casualty, and if you do that a lot it becomes a war crime, and if you kill everyone who's french it's a genocide.
How do we know which slippery slope we are on?
Human made ski runs only will use as much snow as they need because snow is expensive. If ski runs were popular/useful based on their depth then I'm absolutely sure some greedy company would keep piling it up until disaster occurred (mining waste is another great example here).
So how much 'intelligence' is enough? How much capability is too much? How fast is too fast when thinking?
We will never stop improving our capabilities unless some natural law provides that limit. And we absolutely know the base limit to intelligence is the smartest person. And there is little reason to 'peak' intelligence to be limited at the level of humans and their power restricted formats.
You don't want your troops to have to deal with the results of a nuked area. You want to use the psychological terror to dissuade someone to invade you, while you are invading them or others. See Russia's take.
Or you are a regime and want to stay in power. Having them keeps you in power; using them or crossing the suggestion to use them line will cause international retaliation and your removal. (See Iraq.)
Evidently they think regulation is bad only when it puts their profits at risk. As I wrote elsewhere, the tech glitterati asking for regulation of AI remind me of the very important Fortune 500 CEO Mr. Burroughs in the movie "Class:"
Mr. Burroughs: "Government control, Jonathan, is anathema to the free-enterprise system. Any intelligent person knows you cannot interfere with the laws of supply and demand."
Jonathan: "I see your point, sir. That's the reason why I'm not for tariffs."
Mr. Burroughs: "Right. No, wrong! You gotta have tariffs, son. How you gonna compete with the damn foreigners? Gotta have tariffs."
---