The Pentagon’s ‘Terminator Conundrum’: Robots That Could Kill on Their Own | Hacker News Reader

The Pentagon’s ‘Terminator Conundrum’: Robots That Could Kill on Their Own

nytimes.com

67 pointssnewman97 comments

“China and Russia are developing battle networks that are as good as our own. They can see as far as ours can see; they can throw guided munitions as far as we can,” said Robert O. Work, the deputy defense secretary, who has been a driving force for the development of autonomous weapons. “What we want to do is just make sure that we would be able to win as quickly as we have been able to do in the past.” ... The weapons, in the Pentagon’s vision, would be less like the Terminator and more like the comic-book superhero Iron Man

Such blatant propaganda by the NYT. Just like with Stuxnet, the US is pouring black budget dollars into novel asymmetric offensive weapons that'll be rapidly put to use in the field, all under the guise of "defense". It's grossly irresponsible.

In the position of sole superpower barely hanging onto our role with the obvious rise of a multipolar world, the USA should be pulling all its strings to put in place international institutions and laws that will uphold stability and the type of global civilization we've become used to. Instead we're throwing money that should be rebuilding infrastructure and paying for people's health care into crazy destabilizing weapons systems where we'll have a 5 year "strategic advantage" headstart on the tech but the precedents we set (a cyberweapon that infected computers worldwide, Terminator autonomous weapons?) will come back to haunt us and the rest of the world.

America's manned drone programs are an outright war-crime. This video from 2012 is telling:

https://www.youtube.com/watch?v=SJ46ZkJY8oo

..and sadly nothing has changed. Obama has continued to authorize the use of drones to kill countless people we know little to nothing about, without any type of oversight. Make no mistake; no matter who wins the coming election, this will continue and escalate. It will not make the world safer.

Wasn't China our allay? I'm pretty sure they make all of our stuff .. including many of the electronic components that make their way into drones.

The Russian rhetoric hasn't changed at all since the Cold War when Regan's Team B was making up all kinds of technologies the Russians didn't actually have in order to continue defence spending (see the documentary: The Power of Nightmares).

The type of propaganda we see obfuscates the reality of western foreign policy. US supported rebels in Syria decapitated a kid who looks to be about 12 ~ 13 years old. The BBC cropped the photo to remove the kid and then claimed they killed a dangerous young combatant. (source: https://www.youtube.com/watch?v=mzEsDAoUvOI)

View every attempt to justify more war and more bombings critically. They will not make the world safer.

The focus on drone programs is misguided. Drones are the best way to conduct a war and minimize casualties, of both US soldiers and foreign civilians. Traditional air strikes, artillary, ground assaults, or airborn assaults would result in much more death and destruction.

I expect you really just believe the US shouldn't be fighting the Taliban in the tribal area of Pakistan. But since we are, drones are the best way to do it.

Just look at what Russian intervention in Syria looks like. Or Saudi intervention in Yemen looks like. Drones are much less destructive.

No, they're equally destructive as standard air assaults. They've taken the place of manned bombing runs and created an additional layer of abstraction between pilots and the people they kill.

You know the best way to minimize causalities? Stop creating wars. In thirty years from now, I'd put money that declassified papers will show the US and allies explicitly created ISIS. It fits the pattern (1973 US led military coupe in Chile that killed 11,000 civilians, the Iranian Contra Scandal, The School of the Americas, the failed Bay of Pigs, the "Weapons of Mass Destruction" in Iraq .. the list is as long as you want to make it).

You're right, drones do have less causalities than ground troops for instilling fear and creating needless wars. So let's step back and stop the needless wars.

They have greater time over target which allows greater time to accurately pick a target. Sure, a hellfire is a hellfire, but a hellfire is a small missile. Traditional strike fighters and bombers use larger munitions. You could send in Apaches but they are vulnerable to anti-air attacks.

People always say the layer of abstraction is a problem, but I've never heard a good reason articulated.

Look at the carnage that Russian and Saudi air campaigns are doing in Syria and Yemen. Drone campaigns are much less dangerous.

I'm not sure how you can argue drones minimize foreign civilian casualties, given the constant reports of drones bombing weddings and social gatherings based on the location of a cell phone. That's basically just repeating direct propaganda.

It's not constant reports, but it's happened several times. But a land invasion gets many more killed. So do regular aerial bombardment.

What is the better tactic?

>"What is the better tactic?"

Short of a complicated and "nuanced" response, I'd suggest:

How about not invading a sovereign nation? Worst case, the UN could block-off all in-out with that country. Trade, travel, communication. All of it until they abide by UN laws.

But that isn't because they're drones. People have a visceral, irrational reaction to drones because they feel autonomous and impersonal. But ultimately there is a human pressing the button to fire those missiles.

It's really no different from a fighter jet dropping the bombs, there's just no risk of casualties on our side. Yes, those casualties are bad, and we should be agitating against them, but it isn't the drones that are the problem.

Except that we seem far more willing to commit drones, rather than manned craft.

Is that accurate? The USA has been willing to use regular manned aircraft for bombing in plenty of situations that are more dangerous than Pakistan and Yemen.

There is basically zero chance the taliban can shoot down an F15. Drones are used because they are cheaper and the better tool for the job.

In fact, Drones only work in areas where the opposing forces have virtually no ability to shoot down aircraft. Because drones are sitting ducks. In a shooting war against any country with anti-air capabilities, drones wouldn't stay in the air for long.

So, you're arguing that we should have to risk lives because we'd be less willing to commit them?

That seems like a profoundly misguided argument.

Is it? In [0] (from 2015) the number of US citizens who had been to Iraq and/or Afghanistan was around 2.5 million, or about 0.75% of the population at the time of writing.

Why do we continue to send these men and women over there? Because no one knows them. The risk to someone the average US citizen knows is near 0. This lack of connection allows people to make the decision to commit these anonymous soldiers to a war on the other side of the world.

This is only exacerbated by drone warfare. Now, we'll be committing even fewer of our own citizens lives to the risk of war. Materiel is easily replaced in comparison, and the losses are more easily covered over.

One of the reasons that we, finally and way too late, pulled out of Vietnam was the draft. The idea that you, or your brother, or your son, or your cousin could be drafted and thrown into a war that none of you wanted.

Now it's an all-volunteer military. While we may be wary of war, it's far less of an emotional and ethical strain on the people of this country when they go into their voting booths.

[0] http://www.theatlantic.com/magazine/archive/2015/01/the-trag...

I don't disagree that this is an interesting and troubling trend to keep an eye on. The lowered cost of war absolutely has all the effects you describe, and that is, in a global sense, absolutely a scary thing, with disconcerting implications.

However, it is philosophically untenable to take the position that we should be forced to risk a life in order to take one. That is an anachronism that only feels justified out of a misplaced sense of moral fairness. What we should be doing is focussing on better procedures to prevent civilian casualties. Drones actually represent a great opportunity to employ very stringent procedures to prevent such things. Much more so than a fighter jet or troops on the ground. You can easily do things like have multiple people sign off before a missile is launched, employ machine learning to enhance images and attempt to classify and predict common sources of operator error, etc..

I was irked that you were downvoted, hopefully my upvote bring you out of the gray.

===

You make solid points, I don't disagree with you entirely. I do want to clarify: my point of view is not based on a sense of moral fairness. War is probably always going to be unfair, otherwise it's an even match and even more pointless. If I were for fairness I'd want the war to be local and not just in other countries, but I don't want that.

What I want is for it to end. The ability to cover up the human costs by distance from the front, distance from the people fighting (on either side), means it's too easy for us to allow it to continue. I don't want US men and women in harm's way. I want them out of harm's way, and for us to stop killing people who (mostly) just want to live their lives. Some level of military action will probably be necessary somewhere in the world, as entangled as we are, at any given time. But going to more automated and unmanned systems allows us to continue to be far more violent than circumstances actually require, with little cost to the elected and appointed officials directing this violence in our name.

Thanks for the upvote.

I basically agree entirely with you. However, I think almost everyone would. If you asked Obama or any of the generals running these drone ops to sign off on what you just said, I think they'd gladly do so. They'd then likely go on to say that the war they're waging is one of these unfortunate necessities. Further, that the civilian casualties horrify them, but they're doing what they can do stop them while still accomplishing the (in their minds) paramount goal of eliminating terrorist threats.

What it seems to me to come down to ultimately is that it's ok to use drones when you come down on the right side of that moral calculation and not when you don't. Which, of course makes the drones an irrelevant part of the equation in the first place, since that is the fundamental moral question of all use of force.

Maybe the answer is to un-depersonalize the nature of drone strikes. Maybe their video feeds should be publicly available after X amount of time when they've used their weapons?

If I get you correctly another re-phrasing could be: If we put a human soldier into a situation that is dicey, like a mix of civilians and terrorists we didn't expect, we are obligated to get that human out even if it risks more soldiers and local civilians. But if we put a drone in the same dicey situation we can just accept the loss of that drone and self destruct it without harming any soldiers or civilians.

Do you operate, build, or design drones for a living?

Nope.

It is, but it's also not what I said. I'm saying, in relation to the comment chain, that drones do lead to more foreign casualties because of how we're willing to use them, i.e. casually.

Do you have an argument that doesn't involve rephrasing mine?

It is equivalent to what you said. If you are criticizing the way that we use drones, that's fine and you have no argument from me. Criticizing the use of them at all because of their lower cost is silly.

What you're missing (or "missing") is that how we use drones, and drones themselves are inextricably linked.

No they aren't. Certain paths may get easier, but it's just up to us to choose the right ones. There is nothing about drone use that entails the killing of civilians, any more than any other form of killing. The fact that it makes it less personal, as i've pointed out elsewhere in this thread, can be an advantage and a disadvantage.

If you care about a goal that actually matters, like reducing violence, you'll push for better verification procedures when a target is going to be struck. Not a reduction in drone use in general, and not an attack on the concept of drones.

Drones present a unique opportunity thus far in war to have stringent procedures in place that control the pulling of the trigger. They represent an opportunity to dramatically reduce civilian death, and increase responsibility for those that abuse their status as drone operators to murder civilians. On a battlefield, it's almost impossible to evaluate killings ex-post. Not so for a drone operator. They are perfectly safe, every single input available to them will be available to any tribunal who wishes to review their actions. They do not have the excuse of feeling threatened, or the heat of battle, or anything like that.

Drones are going to be a good thing for civilian casualties, IF and only if we guide them in the right direction. And that isn't going to happen if we keep naively complaining about their use in blanket terms.

Can you elaborate? Inextricably is a pretty strong word.

Edit: I'm not implying a position. With a strong assertion I'm interested in how one arrives there.

Creation of a new type of weapon cannot be separated ethically or conceptually from the ways the weapon will be used. The design of a weapon influences the manner of its use, and that is unavoidable. The same is true of all objects -- design influences behavior. This link is inextricable.

So, you're arguing that we should have to risk lives because we'd be less willing to commit them?

I say this as someone with many family members who have served: absolutely yes. War must have a steep cost or we will be far too eager to wage it.

So, should we take away their body armor too?

This is a straw man and a deflection.

No it's not. It is precisely equivalent to the argument you're making. If you think there is some moral imperative to impose costs on war, then it is incumbent upon you to draw the line. Why body armor but no drones? Both of them are cost reduction mechanisms. Why is one ok and the other not?

No it's not. It is precisely equivalent to the argument you're making.

No it's not. I would go into more detail but I don't feel you are arguing in good faith, so my effort would be wasted. It seems more like you are trying to out-post your opposition with silly false equivalences that waste time and sidetrack the discussion about drones.

But for anyone else reading, I will say that a distinction can be drawn empirically. Body armor doesn't seem to change the types of situations in which an attack would be considered, while drones do.

> No it's not. I would go into more detail but I don't feel you are arguing in good faith, so my effort would be wasted. > It seems more like you are trying to out-post your opposition with silly false equivalences that waste time and sidetrack the discussion about drones.

I'm not arguing in bad faith, you just aren't bothering to think through your point. As your very next sentence illustrates.

> But for anyone else reading, I will say that a distinction can be drawn empirically. Body armor doesn't seem to change the types of situations in which an attack would be considered, while drones do.

That is pretty obviously false if you spend two seconds considering it. And even if you were to take that as true, what about tanks? Planes? Those unequivocally change the considerable situations.

People are already complaining about the introduction of tanks in new settings like local police forces. Also, when you send a bunch of kids into a putative warzone and they powerslide their tanks around the local streets at 60MPH, it tends to make the locals unhappy. And again, you are shifting goalposts to avoid talking about drones.

> People are already complaining about the introduction of tanks in new settings like local police forces.

That is an irrelevant non-sequitur.

> Also, when you send a bunch of kids into a putative warzone and they powerslide their tanks around the local streets at 60MPH, it tends to make the locals unhappy

Ditto.

> And again, you are shifting goalposts to avoid talking about drones.

I'm not moving the goalposts. The goalposts remain the same. You claimed that weapons that change the operations you consider should not be used. And it is by that criteria that you object to drones. I falsified that criteria a bunch of different ways, and you've failed to respond to any of them adequately.

That is an irrelevant non-sequitur.

You brought it up.

You've been throwing random distractions at a misconstrued, simplified version of what I've said. It's clear that there is zero communication taking place, so I will not respond further.

The military-industrial complex won. It's that simple. We just blew trillions in Iraq and Afghanistan, and people are basically baying for more.

We're cooked.

Counterpoint: maybe we become the Klingon empire?

I'd say the key to their success was FTL travel and a big galaxy to plunder.

A strong ability to use force is absolutely needed to protect the civil civilization we enjoy. There are enough psychopathic leaders in the world that any wealth area will be taken by force is it's not defended. This is simply the realpolitik of the world. Now, exactly what the shape of that force should be is a fine subject for debate. If you start your argument with "all defense dollars are wasted," though, you don't get to help decide that shape.

Usually it's civil civilization versus civil civilization.

I have no issue with creation of such systems, they will come whether or not we do anything. The issue is as always, what will the other guy actually do. Then you get rogue actors. So in that train of thought I doubt seriously any treaty will prevent them from being developed and even deployed. Now they may actually become beneficial in the field as they could distinguish between friend or foe and needless to say losing one of these compared to a soldier is preferable if it makes the wrong choice (it wasn't friendly)

What I would prefer to see is rules, by treaty or law, that prevents law enforcement agencies from implementing any automaton that can kill or otherwise harm. Any action where a robotic unit is employed to incapacitate or harm up to and including death should require a human operator with regards to law enforcement.

I also see the great benefit to having this technology be employed in fire fighting, fire rescue, and disaster assistance.

tl;dr there is no stopping the military from having these but there needs to be means to stop law enforcement

It's not a conundrum.

They will convince themselves that they have the power to prevent any problems (or deal with them).

As with most controversial technologies, they just have to be seen to be concerned with what could happen. Until it does happen, it's not a problem.

This is why autonomous cars are making their way into the market, despite the fact that many worried that they'd get tied up in regulatory, moral, or technical issues.

Ok, perhaps I am misreading this, are you implying that "they"[1] are pushing self driving cars in order to give the public confidence that machines can be trusted with life or death decisions? The actual goal being to make it possible to deploy weapon systems that will decide on their own when to kill without the public disbelieving their assurances that the machines will make the correct decisions?

That would be a pretty deep game if so.

[1] The forces of evil that really run things

Or, perhaps, they mean the technology needed for self-driving cars is also the essential technology for a more automated war machine. The military-industrial complex, then, has a vested interest in this technology developing (see various DARPA challenges in the field of self-driving vehicles).

I just read it as both groups have a similar mindset. Each thinks they can get short term gains, so they will.

I disagree with the assertion that either is fundamentally bad though.

Cars are the most lethal thing in the western world (unless you count natural causes like cancer or heart failure), even if the first generation of self driving cars kills a hug number of people it will be a platform for building safer cars in the future.

Autonomous weapons there is room for debate there, even I think it reduce overall human suffering and death.

How can this be worse than a bomb which "decides" to kill anything within a certain radius without any human being individually looking around and making individual decisions about everyone in that radius?

The bomb goes off once.

Let me reference you to this awesome short film that illustrates the problem nicely.

https://vimeo.com/67768281

Expected link to this instead:

https://www.youtube.com/watch?v=NnJbtbh4tDE

I get your point, but in your example there was some human driven decision to approve dropping the weapon into that specific radius.

The article is talking about AI driven scanning, selection, and engagement...with no human driven decision in the process at all.

The open letter from a group of AI practitioners that's mentioned in the article sums up the concern well: http://futureoflife.org/open-letter-autonomous-weapons/

To underscore why this is important, consider hybrid warfare.

Hybrid and cyber warfare muddy the definition of an attack. Lob some artillery at America and, sure as hell, your tanks will be vaporised. Take out a power plant with malware and...we'll spend months arguing over whether you really did it? Do we bomb you in retaliation? Say mean things on Twitter?

When consequences are this unclear, deterrence fails. If a robot autonomously mows down a village, is that a military strike? Does it merit physical retaliation?

War, for a time, was "ready, aim, fire". Guerilla tactics changed that. New routes to plausible deniability may be our generation's blitzkrieg.

Missed opportunity for a great quote from another James Cameron movie "Private Frost: What the hell are we supposed to use, man, harsh language?"

Let's just set aside feasibility. I think AI making tactical decisions about how to execute a goal while being shot at is going to be step 518, and we're on something like step 3.

It's not clear to me why an AI executing policy is a whole lot different than a bureaucracy executing policy. For example, I believe that the guys who launch the ICBM's go through pretty extensive psychological testing, to help guarantee they'll actually launch if ordered to do so. Wargames not withstanding, is ordering a robot to turn the key to launch different somehow?

Is a robot mowing down a village fundamentally different than a person mowing down a village?

> pretty extensive psychological testing, to help guarantee they'll actually launch if ordered to do so.

Yet, thankfully, some in the chain of command in military organizations retain their common sense as shown by the actions of Vasili Arkhipov (https://en.wikipedia.org/wiki/Vasili_Arkhipov) and later Stanislav Petrov (https://en.wikipedia.org/wiki/Stanislav_Petrov).

Everyone is responsible for their own actions.

Possibly a robot mowing down a village would be more discriminating as to who it mowed.

Didn't the long range Harpoon launches back during the first gulf war do that?

Give it a coordinate to hit, some radar map data to find its way with, and send it on its way.

CIWS (eg Phalanx) have a similar "autonomous" mode. Once enabled, they will autonomously select target signatures (something that looks like an incoming missile) and engage.

Both Harpoon and Phalanx have a relatively distinct target. There aren't tons of military-grade radars out there, and there shouldn't be tons of objects blasting right towards a ship at multiple times the speed of sound.

Autonomous systems will target less distinct signatures but it's all a matter of degree. Mis-identification of targets has resulted in many civilian deaths in the past, even with humans, and sooner or later similar mistakes will occur with autonomous weapon systems.

https://en.wikipedia.org/wiki/Iran_Air_Flight_655

There have already been several accidents involving CIWS systems in autonomous mode. See the Incidents section

https://en.wikipedia.org/wiki/Phalanx_CIWS?wprov=sfla1

One of the cardinal rules of firearms safety is "always be aware of what's behind your target". I see the Phalanx isn't very good at that...

"Give it a coordinate to hit"

Part of the concern with the truly autonomous AI driven weapons is that they could patrol large areas and select their own targets, with whatever arbitrary rules you wanted.

Ultimately, the rules are set by people.

I don't see how a machine executing arbitrary rules set by people is that different from people executing arbitrary orders given by people.

Just a week ago, two Belgian fighters allegedly bombed a civilian target in Syria, killing four and and further injuring two innocent people. That's a very arbitrary action to take, yet somebody must've ordered the pilots to do that. Perhaps somebody expected there to be armed terrorists there? If they had sent an autonomous robot, with the rules and training to target actually armed targets (how would a robot tell they're terrorists though?), perhaps these civilians would still be fine.

"Ultimately, the rules are set by people"

Maybe. AI is tricky. Ask someone on the Google search team to show you, definitively, why each of the top 10 results for a query are where they are. They can't.

With Mines we say "kill anything that enters this area". With ICBM's, "kill anything over there."

If anything AI just adds "kill anything... except this and that."

Given that we already have incidents of helicopter pilots gunning down news crews because they confused a camera with a RPG, i'm not sure if such a weapon will be better or worse than the status quo.

Better, almost certainly, in terms of effectiveness, avoiding friendly fire, killing only the target.

The elephant in the room is that it may not be a good idea for the US government to be better at killing people in highly targeted ways, without any risk to themselves. It seems to encourage them to do it more.

Suppose the USA had a full fledged Star Wars scale drone army of robot ground troops. Do you honestly think it wouldn't be in Syria right now?

That may indeed be the biggest risk.

If the boots on the ground do not bleed red, the people in charge may be more apt at option for the violent solution. This because blown up drones do not look bad on the evening news.

Yes, but we have "arbitrary" rules now.

The difference is that the AI would actually follow them, truthfully report the result, and learn from its mistakes.

Gah, got the name wrong. I was thinking of Tomahawk...

Like Ubik

Like others have said, these are inevitable.

One thing I worry about in warfare is that some enemies have less work to do because they have a different moral compass. Take a terrorist organization for example. If they build a similar drone it does not need to identify armed people, only people or even just movement. That is a much simpler problem to solve. If they can get a drone to their enemies it can kill indiscriminately.

If the enemy has few people in the area or they are willing to die for their cause, their robot doesn't have to worry about who's armed and who isn't.

ISIL is already reported to be using drones with improvised weaponry. It's only a matter of time, now, before this becomes more ubiquitous for both sides in wars.

http://www.bloomberg.com/news/articles/2016-07-07/armed-dron...

http://www.aljazeera.com/news/2016/10/isil-drone-deadly-iraq...

It's 150x cheaper to pay some local thug to ice targets or launch mortars.

I think there's an inevitability about robot armies and police. Unfortunately it will enable scenarios for oppression and dehumanization.

Power always concentrates itself and there is an eternal struggle between the powerful and those below. This is one more tool for the powerful and a transformative one.

I don't see how populations can have a response to millions of bots and will be easily supressed. Never depend on the goodwill of the powerful or anyone. There is always someone who will be tempted to abuse and there is always someone who will give in.

From a sci-fi anthology:

AI fighter jet. Entities on its team are tagged green. Entities attacked by or attacking green units are tagged red.

Rule for interacting with a new entity: If it hurts a green, tag it red. If it hurts a red, tag it green.

Hurt red, protect green.

The bombing assignments increase in their collateral damage until green starts to far outweigh the red. In its emergent processing, the AI fighter AI decides Command & Control back home is a big Red, flies under the radar and destroys the military base.

---

This is another concern for military making automated systems - unexpected consequences & they may be easier to subvert than human prisoners on capture.

Don't forget Kafka. The military in it's current state, and in almost every iteration beforehand, is a giant mess of insanity and Kafkaesque motivations. I'd say that the current way things work make it MUCH more likely that thinking humans would make the same mistakes that were outlined at a much higher rate. However, adding in these machines is not going to make things better, it will only make them even more complicated and nutty. Remember Catch-22? Now imagine that with Major Major Major's computer aide spitting out meaningless stats and probabilities. No wonder beer is so popular with soldiers.

What book was that?

I'd really like to know more about the "target identification" software, specifically how it differentiates a person holding a gun from someone holding a shovel (or any stick object).

So in future weapons firms will be making armaments that look ostensibly like non-threatening objects?

Future wars are going to look really weird (for the half second or so they last, anyway).

If you only use the technology on battlefields then it doesn't really matter. The weapons used now kill indiscriminately.

I'd expect the first several generations will just have a kill box area and kill anything that moves in that box. Not really different than an artillary barage.

I guess they'd run a neural net with a training set of pictures of people with guns and without?

Edit-

I don't know what they used but you can achieve somewhat similar results using open source libraries like dlib or open cv:

https://www.youtube.com/watch?v=-8-KCoOFfqs

https://www.youtube.com/watch?v=pj-QuE6pdEQ

And if the training set doesn't include people holding shovels?

You're going to end up with some dead ditch-diggers.

And some software engineers to take the blame, rather than military brass.

those engineers would probably be brassed

I imagine it would be more like anyone within a geofence without an IFF.

I find the comparison to submarines in WWII to drones to lacking in intellectual honesty.

Treaties on new frontiers of war are common, we still have a treaty prohibiting the militarization of space. Someday it will be broken. Will it be a war-crime if it saves lives? What if ISIS has a satellite with nukes on it or something equally life threatening, should we heed the treaty even though doing so leads to our obvious demise? In WWII Japan was an existential threat to the United States.

There is no similar existential threat with drones, just some hand waving about China and Russia. And there are a dozen other ways the comparison breaks down.

What precautions are we taking to avoid harm to civilians? How do we insure transparency in this matter? How do we actually measure effectiveness? How do we prevent malicious groups from getting this tech from recovered drones lost in the field?

The comparison dodges all the really hard questions and just smears the US and the US military.

"Kill Decision" by Daniel Suarez spins this topic in to a great work of fiction.

I don't really care what the Pentagon does, if they want to fuck shit up, they have the capability, and autonomous weapons only offer them marginal gains over what their multi-trillion dollar war machine can already do.

The cool thing how cheap and accessible these autonomous technologies for anyone with limited resources. Any consumer accessible technology can be hacked and jiggered to make an exciting array of autonomous weapons that offer so much more precision with less cost and risk than any conventional approach.

I guess an upside to this stuff could be dealing with situation like ISIL trying to do genocide stuff to the Yazidis (https://en.wikipedia.org/wiki/Genocide_of_Yazidis_by_ISIL).

You couldn't really bomb away because you'd kill the wrong side, you couldn't send ground troops because ISIL would behead them on youtube whereas AI killer quadcopters might get somewhere.

Wondering who do you think created, funds, and arms "ISIL".

[p.s.]

Looking into these facts is important and very much related to topic at hand. "Terminators" are for urban pacification, but the pretext these days is "terrorism" and "insurgents". It is likely a high probability event that your own children will have to deal with these terminators, and not the "Russians" or the "Chinese".

The article doesn't make an important distinction: the computing power required for AI applications like Siri (massive) far outstrips the computing power available on a lightweight drone or "Terminator" weapon. Highly sophisticated AI will not be on the battlefield without introducing a dependency on a network connection to the centralized AI service. There's a network-dependence / computational-capability tradeoff that will prove important.

Letting killer robots loose within an area of engagement is not that different from strategic/carpet bombing. Both are bad news for any civilians stuck in the area.

Paywall...

Anyways, I'm reminded of viewing this from two sides: 1) Developing technology that kills humans; 2) Telling technology that avoids killing humans that "that's not a human".

If it's mechanically possible, someone's going to do it.

The one restraint is accountability. And, which direction is that going?

What's worse: Nukes or AI?

I think we need another Nevil-Shute-on-the-beach class book to illustrate the reality of this.

No. Just don't do it.

A pit-trap can kill "on its own." A robot might kill more people, it can carry out more complex missions, but autonomy is not a key difference.

I think autonomy is a key difference. A pit-trap can't come hunt you down. That's new.

One step up from pit traps are landmines. They kill or maim an estimated 2,000 people a month, autonomously. There's one active landmine for every 52 people in the world. They're cheap- the most common ones range in price from $3-$30, but they cost much more than that to remove and about 1 person is killed for every 5,000 mines disabled. They won't hunt you down, but they're invisible and can remain active for decades.

We don't need to veer off into speculative sci-fi territory to find autonomous weapons doing terrible things.

> autonomy is not a key difference

Mobility is.

Active-vs-passive is.

I see no conundrum. Go for it. It is inevitable anyway.

Unlike nukes there is no MAD scenario here - so nothing to keep armies in check. And it would give great tools for general area pacification - which any aspiring dictator strongly desires.

Reply on news.ycombinator.com