perhaps AI will be generally intelligent, but please for the love of god, let's only consider that scenario what it actually happens.
But you're right in that the two problems are inseparable. Wisely implementing sensible and effective AI controls will fall onto politicians and statespeople, from legislatures to constrain things domestically, to Presidents who have to lead abroad in forging meaningful and effective treaties.
This only makes sense if you think that at that point, it will not be a "the genie is out of the bottle" scenario and already too late to put it back in. And yet that's what people constantly say about current LLMs.
Does AI have to be strong to kill everyone? I can imagine scenarios in which it could be fairly shit, but good enough.
> please for the love of god, let's only consider that scenario what it actually happens.
Not a great philosophy to guide planning.
Let’s hear these scenarios that aren’t possible without AI.
An LLM is tied to a bunch of APIs as an event response agent. Maybe in some poduck enterprise support group.
But... there's tons of other LLMs tied to event response agents, in ... less podunk groups.
... and there's some fancy black box org in the CIA that then employs them as some crisis monitor.
and a cascade of false reporting leads to "oh my god there's nukes in flight, and the system has autolaunched nukes in response".
What current AI is good at is faking being intelligent, so that they can be used in many many places to substitute for "real intelligence" humans.
But... that means dangerous cascades in all likelihood.
That isn't skynet, but it is 90% of what the Terminator skynet disaster scenario was: an AI triggered a nuclear exchange.
Reason 1: Profit motive: No one's going to be making a profit off of hooking an AI up to nukes.
Reason 2: For the lulz: This is the kind of thing that happens when you have very broad access to the elements in question. Since access to nukes is very tightly restricted, and their usage controlled by significant parts of governments, it's vanishingly unlikely that either a conscious collective decision will be made to hook up an AI to nukes for the hell of it, or that someone will be able to do so on their own.
And yes, I know that there are lost nuclear weapons, which means both that a) there are some out there not under government control, and b) governments can be careless with them—but frankly, any non-governmental actor known to be holding a nuke would have to be considered a top-level threat no matter what they had making the decisions about whether and how to use it.