Killing Robots and Just War Theory | Hacker News Reader

Killing Robots and Just War Theory

oleg.nu

45 pointsbehindai53 comments

A bit more scary is the political consequences. One of the strongest controls on rogue generals is that a fair chunk of their troops will mutiny if told to destroy their home country for said general's benefit, and logistical support would vanish.

Armies of killer robots start radically changing the calculations. It isn't obvious how but it probably removes a lot of the interest of more ordinary people and makes it a question of what the elites want. If the programmers are on side and the factories are mostly automatic, then the ability of people to push back on their own military becomes complicated.

How is this different from nuclear weapons?

If a general lives in the penthouse at the top of a skyscraper, and maybe has a few thousand people who support him, nukes are not going to help deal with revolutionaries on the bottom floor. They are too indiscriminate and can't be used against close up targets.

Whereas a general with a drone swarm and a few thousand supporters might be able to suppress a city. Fortunately a little far fetched, but what was impossible with nukes is only impractical with drones.

This is well put. There is also the idea that as armies got MI-24s/AH-64s, etc, opposing forces got Stingers/Iglas. Perhaps it will play out in a way where once armies get drones, OpFor gets counters - jamming / hacking / EM / whatever.

The land to be ruled by said rogue general will not be a radioactive wasteland.

Sufficiently advanced robots would leave a lot more of value behind than a nuke which makes deploying them much less costly to the person that wants to actually control the territory.

Seeing AI weapons as a weapon of mass destruction is a trending discourse.

It's a much harder weapon to control via regulation and treaties than nuclear weapons: there is no nuclear material required to hook guns up to computers en masse in factories, let alone improvisationally. Once this cat is really out of the bag, everyone who wants it will have it.

Manufacturing chips is an extremely complicated process and requires thousands of specialists from multiple different disciplines to cooperate. Building a new fab costs billions and the buildings are extremely sensitive for damage.

I think one of the reasons why the west wants to block mainland China from manufacturing chips is to prevent them from building a robot army.

It's true that building the chips is hard, but making it hard to buy general purpose small computers, or digital camera sensors suitable for vision, or rotor blades, or batteries, or antennas, is also going to be very difficult. Proliferation has already happened.

Maybe cheap LIDAR will be something we can keep a lid on? But regular vision is still likely pretty good for lining up a shot or finding a face.

> but making it hard to buy general purpose small computers [...] is also going to be very difficult

Just ban general purpose computers that allow running arbitrary unsigned code. I'm sure lots of governments and corporations would be happy to get on board that one, and many end-users wouldn't notice.

A cure worse than the disease. Also wouldn't stop the military from building up a drone army and then someone within it choosing to go rogue and take it with them. It only ensures that no one else can stop them.

I think the parent post agreed with you and was just being facetious -- 'just ban general purpose computing' isn't very likely to be a popular sentiment on hn!

I hope so as well ;)

In a war situation, likely supply chains will be disrupted, so the regime will have to use whatever it has in the country. There are chips in circulation, but each different chip needs different drivers, has different performance characteristics, etc. E.g. I read that Tesla changed a bunch of chips due to the current shortage and they had to write new drivers for them. That's one chip they had to replace.

Of course you could make your population turn over their old smartphones, remove IR filtering from the camera lenses and solder them to a robot body. But this will slow down manufacturing as it isn't well suited to mass manufacturing processes. Furthermore, it's a limited resource so you might have to deal with attempts by your enemy to exhaust it. Last, you'd be limited in fanciness, as in your robots can only process limited amounts. The best place to source the computation needed for really "smart" robot soldiers would probably be self driving cars as they have the AI processing power needed.

Sadly you can't make a general-purpose chip that only refuses to perform military tasks and otherwise functions normally.

The world made it through the cold war without major disaster, but it is easy to imagine history taking a different path, so I don't think that this is a comforting comparison.

Also, nuclear weapons can't be used 'a little bit' but killer robots can be -- it is much less clear where we draw the line. It seems likely that their use will be normalized/entrenched before we figure that out.

Though I hope that you're right.

>> that a fair chunk of their troops will mutiny if told to destroy their home country

I'd be careful about this. There is a general principle in military communities that troops are not to obey "illegal orders" but time and time again throughout history we have seen that backstop fail. At a base level we all do what we think is best for ourselves. Troops will shoot prisoners. Troops will shoot protestors. They will nuke their own hometowns if they believe that is either necessary or if failure to do so will result in their own demise. And many troops, people, will do what they are told out of pure loyalty to a charismatic leader. The only real safeguard is layer upon layer of checks to ensure no one leader can ever exercise independent control.

It's a fair point that humans don't magically make this go away, but the thrust of the argument above is that robots are likely worse, because they lack any capacity at all for questioning or for remorse, and they also allow great physical detachment from the horrors they can cause. They don't stop to eat or sleep either, and they'll certainly do things that can cause their own destruction.

Humans also may be even likelier to be loyal than moral if a drone with a gun follows them around to keep them on-side.

Humans have a natural aversion to killing, and yes, during war mentioned atrocities do happen, but the soldiers are not perfect killing machines. Often they miss intentionally and just shoot in the general vicinity of the opponent.

See this reddit thread: https://old.reddit.com/r/history/comments/b6k528/percentage_...

Troops can also desert which is especially prevalent in civil war situations. That's why during war, you need propagandists coming up with constant stories about how horrible and bad the other side is, so that you get people to shoot.

The unit that so brutally killed the tianmen protesters was one of the most brutal ones the Chinese military had. That's why they had to wait for days for it to arrive. Had they deployed different units, they might have fraternized with the protesters and joined them in their protest, making the task even harder for the leadership. Try getting even the most brutal unit to shoot at people in their own uniform.

For military parades, you can't afford 5% of people who intentionally don't march in lock step. That's why usually the troops which participate in such parades are isolated and fed propaganda leading up to the parade.

The human component is a constant variable that needs to be accounted for in military leadership.

>> For military parades, you can't afford 5% of people who intentionally don't march in lock step. That's why usually the troops which participate in such parades are isolated and fed propaganda leading up to the parade.

What? I've been on countless parades and don't remember any pre-parade propaganda. I've never even heard of such a think in a western military. Parades are just a regular day at the office. Honor guards and flag parties certainly practice heavily, but the idea that anyone "intentionally" doesn't march properly is ... odd.

Also, "lockstep" should not be used in reference to parades. The lockstep was a punishment once inflicted on convicts in an prisons. If you go back hundreds of year there was a style of marching, but it was nothing like the more recent method used as a punishment.

https://en.wikipedia.org/wiki/Lockstep

Sorry instead of Lockstep I meant the german term of Gleichschritt. At least in the GDR they infused the people who marched at the parades with propaganda for days before the parade. Yes, didn't mean a western country.

> The only real safeguard is layer upon layer of checks

But this is one such check. As I remember it it was at work in the fall of the Soviet Union, and it was worked around at Tiananmen Square (where they brought in troops from the country, not trusting the ones on site to be brutal enough).

It just has to be a war crime to use these.

A domestic army of robots is really just a foreign army on contract. They have zero loyalty to anyone who doesn't directly control them.

This is true. Killer robots push societies in the direction of being more hierarchial, for the reasons you state.

This generalizes to automation of nonlethal means of population control -- crowd control weapons, surveillance and censorship. On the margin it'd hold for partial automation short of AGI -- reducing the needed size of the coalition on top.

This works better if a large enough portion of the soldiers are conscripts (possibly rotated in and out) and not career soldiers. The conscripts know they will go back to their normal life, day job and be part of the civilian society again.

If it's just career soldiers, then it's in their career interest to enlarge the military's influence on society, increase budgets etc.

> troops will mutiny if told to destroy their home country for said general's benefit

I’d generalise this check to all leaders. No man rules alone. Power only exists in its relationship with people.

Robotic armies permit force projection without any people. That changes the political calculus in all sections of society, from military leadership to billionaires to members of a terrorist group.

One good zero-day could give total military and industrial control of multiple superpower nations to a single talented sociopath

"The Second Amendment in Iraq, Combat Robotics, and the Future of Human Liberty" argues that combat robots are unconstitutional for the US to develop, because they render the second amendment useless.

http://vinay.howtolivewiki.com/blog/bigdeal/the-second-amend...

That's one heck of a stretch. The more reasonable interpretation would be that the 2nd simply means combat robots should be legal for citizens to own and operate.

I mean who wouldn't sleep better at night knowing your XJ9000 Eviscerator brand murderbot is patrolling your home for unexpected guests? /s

The only solution to a bad robot with a gun is a good robot with a gun.

It is already pretty useless, because militia is not going to have tanks, bombers, fighter jets and nukes anyway.

It was an alright idea when it was introduced, but is mostly obsolete today.

Vietnam and Iraq are counter examples to this, they didn’t fight with tanks, bombers, fighter jets or nukes.

Ding ding ding! Insurgencies that combine small arms, improvised explosives, clandestine cell social structures, fault-tolerant and encrypted communication methods, and distributed fundraising techniques can and have countered imperial military adventurism. This is not a libertarian fantasy; it happens every day.

The Second Amendment was rendered obsolete once the US had a standing army and militias were no longer needed as the primary means of defending the country. FUD about the creeping authoritarian (probably leftist) menace coming to take your guns is great for scaring up NRA donations and Republican votes, but the only real utility the 2A itself has anymore is in allowing gun owners legal access to firearms for self defense, hunting or other non-militant activities.

Nope. Setting aside resistance to government, there is home defense, which is more practical.

It seems like you are arguing that 1) The second amendment is no longer relevant and that therefore there is no legal justification for US citizens to keep firearms, while simultaneously arguing 2) no-one is “coming to take your guns”

Can you see how people in favor of gun rights might not find your second argument convincing, in light of the first?

It appears you didn't read my comment in its entirety. Nowhere did I claim there was no legal justification for US citizens to keep firearms - rather I listed several such justifications.

I also never claimed no one is "coming to take your guns," although I do believe that is mostly just right-wing propaganda, and was referring to the propaganda value of that threat as being the only useful vestige of the revolutionary principle of the Second Amendment remaining, because the actual disparity in firepower between the populace and the US military renders the practical utility of it irrelevant.

> militia is not going to have tanks, bombers, fighter jets and nukes anyway

One, guerrilla warfare works. Two, the point is to increase the cost of suppression to requiring U.S. tanks and bombers and jets deployed against the homeland. An unarmed population can be more casually suppressed.

If I look at the state of health care, education, and prisons in the US, the population seems quite suppressed to me even without any warfare.

Yes, if we change the meaning of words we can change their meaning.

Fortunately these problems have a military solution.

Ok, so if the robot commits war crimes, who is responsible?

The designer? The soldier who turned it on? The general who deployed a couple thousand to the urban center before they had the “bug”

Realistically, if you win the war, or even if you're a western power that quietly retreats after losing it, there's nobody responsible. And if you lose the war the winner decides who was responsible for your killer robot army. There's Nuremberg precedent for "both".

If you're the designer on the losing side you might get lucky and the winner will 'look to other way' of your crimes in exchange for your future services. Also a precedent from WW2.

Engineers and scientists are generally considered a "resource". The winning power takes them, if needed.

Scientists are generally not considered a threat to power (unlike high-ranked officials). For example, Piotr Kapitsa had his arguments with Lavrentiy Beria and Stalin. Instead of death (delivered for much smaller insubordination) he was gifted a rifle:

"Beria summoned Kapitsa who amazingly refused him: “If you want to speak to me, then come to the Institute.” Beria ate humble pie and took a hunting rifle as a peace offering. But Kapitsa refused to help anymore."

We already have systems like that, whereby the accountability is intentionally so diffuse that no one can realistically be held accountable. Was it the programmers? The people who provided the programmers the specs? The military who set the kill parameters?

Well the answer is everyone on the losing side. How the punishment is carried out would probably be the same as in WW2. Prison time for most, to varying degrees, more severe punishment to main leader, probably limited clemency to the highest ranks so they can work in other countries to pass on their knowledge and skillset.

I speculate

That's I guess the main point of the discussion. The terrible news, that usually the winners judge those who lost. And a huge part of war crimes can be left unnoticed. At the same time using AI-ruled machines gives a huge advantage and increases chances to win. That scares.

If the robot is perfect, then there is no one left to prosecute a war crime. It's only a war crime if someone can accuse and prosecute, thus only losers commit war crimes.

There is no need for robots to have that sort of issue - https://en.wikipedia.org/wiki/Land_mine

>> could not be understood by people and thus programmers are detached from the decisions making by machines.

This isn't really true. A programmer may not understand the exact mechanism of computation but they will have trained it in a particular way using particular training data to do a particular purpose. That purpose is known to the programmer so this fact actually doesn't change anything. It is possible the robot will occasionally act in an unexpected way, but the majority of the time it will follow it's programmed objective, even if following that objective unpredictably.

Reply on news.ycombinator.com