The Pentagon inches toward letting AI control weapons
wired.com
wired.com
See Stanislav Petrov - the man who prevented an apocalypse https://en.wikipedia.org/wiki/Stanislav_Petrov
The anti-nuclear weapons crowd conveniently forgets that the alternative to MAD isn't reliably peace.
We're talking about potential existential risk - a wiping out of all of humanity (through severe ecological poisoning after a barrage of cross-globe nuclear strikes).
If my math works out, that's a 10% chance of an occurrence every 1,000 years.
That's more on the fence for me.
On one hand, it's high (and the impact would have been catastrophic!). On the other hand, it prevented NATO / Warsaw Pact mechanized conflict across Europe (which also would have been catastrophic).
Tough call. Especially with the uncertainty that, were you to give up your nuclear weapons, there's no guarantee your counterparties would actually do the same.
In the 60s foundational position papers, there's definitely a strong thread of "We will commit to MAD not because it's the best approach, but because it's the best zero trust approach."
What? You claimed you are comfortable with 0.01% chance, but now you're "on the fence" with a 1% chance?
An analogy: some chance of your child being occasionally bullied in school, or a 0% chance of bullying but a 30% chance they'll get murdered.
You say "tough call" -- and I'm baffled.
It is bizzarre to answer 0.01% only to follow up with "well, I could take 0.9% too"
My question obviously implied "what's the maximum you're willing to tolerate" - else you could have just said 0% -- conveying zero information.
Armageddon is lights out for humanity.
People seem to seriously lack imagination concerning what a scientific and economic major power might do if its existence were threatened by a hated enemy. (Gene drives, anyone?)
And...man, I wish we had that $5.5 trillion back [https://www.brookings.edu/the-hidden-costs-of-our-nuclear-ar...].
Nuclear weapons are bigger explosives. Not different explosives.
I've been to the Hiroshima museum. Like I said, it will change your perspective. I'm not really all that inclined to continue this interchange given your flippance.
But personally, I consider them equally horrific. And as stated previously, if MAD prevents either from happening, the risk seems worth it.
As an AI practitioner, I want us invest in research to manipulate other AI military systems but not use AI ourselves.
Take a page from the HFT book.
- agency: the machine makes decisions on its own.
- defensive vs offensive action
So if Iron dome launches a rocket in defense it's okay, but that rocket mustn't use AI to find its target? I see your general point, but that's going to be a line that's very hard to draw.
As to your specific example, while system x may have 'full agency (autonomous, willful)', if it is purely 'defensive' in nature, it may indeed fall within acceptable zone.
Your metric seems arbitrary. Why (only) two dimensions?
I mentioned it publicly once, and I’m bracing for the day that some established researcher quote tweets it and says “we do not need this kind of thinking in our community, and we have a duty to exclude it” or some such.
I don’t care much though. You seem right, and that’s good enough for me.
I guess I shouldn’t just say “researchers” —- it’s what we need in our citizens of humanity.
https://en.wikipedia.org/wiki/Peace_on_Earth_(novel)
> The evolution of artificial intelligence has allowed major world powers to sign a rather curious treaty: the Moon is divided into national zones (proportional to each nation's Earth real estate) and all weapons development and production must be moved there to be handled by factories. This is supposed to completely demilitarize Earth, achieving the long-sought dream of world peace. A MAD stabilizing factor is apparently preserved by the ability of countries, in case of war, to quickly ship weapons down from the Moon.
In my view AI weapons development demands a judicious, steady, paced, and thoroughly informed approach to keep the whole space from burning up —- figuratively and literally.
It’s not a great situation overall, but you couldn’t uninvent nukes at that point, so it has proven remarkably effective thus far.
Same dynamic with AI/ML —- can’t uninvent. If one aggression-minded power has it, balance will be required for stability.
See: cluster munition and mine bans
Chemical weapons being banned is an example of a mostly inexplicably altruistic action on the part of major powers. Well, inexplicable if you're a cynic.
They are not. It's the most ineffective weapon outside swords and bows.
A sibling comment provided the link discussing it: https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...
Also, surely, none of those are anywhere near as bad as mustard gas. Yes, agent orange gives people cancer, but presumably not everyone (?), and anyway mustard gas immediately burns your lungs. That’s just a different league.
My understanding is that DU tends to be favored for three primary reasons(ish)
1. It's super dense and thus packs a super hard punch in terms of momentum you can slug at a distance for a given caliber barrel.
2. It's incredibly hard and can deflect things that steel can't stand a chance against.
3. That momentum from firing is often high enough that it actually ignites on impact in most circumstances, causing a lot more material damage to whatever you're shooting.
DU might honestly be phased out soon anyways - it's primary strengths are that it's wicked heavy and not stupid soft like lead, but there are some replacements out there for it and a lot of countries have signed on to banning it in warfare - just not the big ones yet.
They are very effective against civilian populations though. Which is likely the real reason behind the ban. Especially when you consider that certain chemical weapons are only selectively banned; they can't be used against a target for toxicity, but it's completely fine to use it to use it in another manner. White phosphorous is most famous example if this, it's a war crime to poison people with it, but a-okay to light them on fire with it, even though that's a horrific way to die.
The only explanation for this hypocrisy is that chemical weapons don't have much value against military targets anymore. So substances are banned unless they are actually useful in other capacities.
I don't have a link, I read a pretty interesting analysis is that the only reason we have such treaties is because those weapons are ineffective against a 1st tier military and are also useless to them. Basically, defense is possible as the needed PPE is relatively cheap and can be used effectively by highly-trained troops, and high-tech conventional weapons used by a trained force are far more effective than gas.
So basically, those treaties were about giving up something advanced militaries would have abandoned anyway.
It's probably this: https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...
1. Send trained 19-30 years old to risk there's life, minimum civilian casualties, but highest chance of combat casualties.
2. Bomb urban area out of existence, with massive civilian casualties, medium chance of combat casualties.
3. Send terminators with medium civilian causalities and 0 combat casualties.
Given the track record of how many unjust wars that have been fought, we need more preventative measures against wars, not more technology to make wars easier and more palatable.
Civilians will suffer the most in wars and in violent conflict times.
Urban warfare is the highest intensity warfare, with highest casualty rates both combat and civilians.
First failed attempt of drone attack with chemical weapon was in 90's in Japan, I believe this if first recorded attempt of terrorist attack using drone.
AI warfare with sci-fi flying, crawling, walking terminators will be more humane then carpet bombing population into oblivion (Soviet–Afghan War and Vietnam War), it will have substantially less combat and civilian casualties, in general it will increase kinetics (weapons effects), precision, accuracy, and decisiveness (RISTA).
You created a false dichotomy - as if those are the only two outcomes and we have to choose between them. When obviously (as your follow up comment shows), there are other scenarios where numerous innocent non-combatants will die.
Every major player has these and will show them when there is a global war.
Treaties do shit.
A slightly unrelated example: There are major laws that you can't kill a journalist in your embassy. But did those stop it from happening?
Can't wait for your finished script of Dr Strangelove 2.
> Can't wait for your finished script of Dr Strangelove 2.
It's easy to mock and parody MAD, arms races, etc., but certain conditions make them necessary and rational, and those conditions can't easily be changed.
If we go down this path, and we survive, earth will be the best approximation of hell in heaven.
Do you realize that only one country nuked civilian cities?
"Some of you might die, but that's a sacrifice I'm willing to make."
In war, weapon speed triumphs safety concerns.
And hypersonic and laser weaponry is only going to decrease that.
We've been in a delegated authority regime since the USS Vincennes / Iranian Air Flight 655.
There are already shifts towards a delegated-preemption regime in EW (see: efforts to equip EW suites with self-adaptive capability). I can't imagine we won't see the same thing with actual weapons.
Sure, put a human in the loop, if there's time. Otherwise...
It's an automated, super fast, self-aiming gatling gun.
I think it will be multiple decades before AI has progressed to the point where I'd trust it to correctly identify enemy soldiers vs non-combatants as well or better than a human being. Fortunately, I think it will be even longer before that's something we need.