Comment by hammock | Hacker News Reader

Comment by hammock | Hacker News Reader

Autonomous computers likely have already killed humans in Libya two years ago.[1] Four problems with this new reality:

1) Misidentification. When selecting a target, will autonomous weapons be able to distinguish between hostile soldiers and 12-year-olds playing with toy guns?

2) Low-end proliferation. Market pressures could result in the creation and widespread sale of what can be thought of as the autonomous weapon equivalent of the Kalashnikov assault rifle: killer robots that are cheap, effective and almost impossible to contain as they circulate around the globe.

3) High-end proliferation. High-end autonomous weapons are likely to lead to more frequent wars because they will decrease two of the primary forces that have historically prevented and shortened wars: concern for civilians abroad and concern for one's own soldiers.

4) Undermining the laws of war. How can autonomous weapons be held accountable? Who is to blame for a robot that commits war crimes? Who would be put on trial? The weapon? The soldier? The soldier's commanders? The corporation that made the weapon? (from https://www.commondreams.org/views/2021/12/30/humanitys-fina...)

[1]https://www.npr.org/2021/06/01/1002196245/a-u-n-report-sugge...

Are these even serious discussions? Of course the people running these autonomous systems are responsible. It's like dropping a bomb on someone and saying "hey I didn't do it, blame the bomb!"

No one is "running" an autonomous killer. It makes its own decisions. That's the rub.

No it is not making its own decisions. There's always a clear path from programming to execution. The slight variability added by a neural net or whatever else doesn't change that fact.

If a serial killer chooses their victims by using a random number generator, the algorithm isn't going to be tried for murder.

I see No Country for Old Men as partly a study on the conflict between those who believe in free will and those who don't. The villain, Sugar, has shed the illusion of free will, and has thrown himself into the thing he knows motivates him: prejudicial termination of other people who stand in the way of his current clients' goals.

This is in contrast to the backdrop of a community reeling with outrage at and fear of his brutality. They see him as an agent of evil, some sort of amoral if not immoral demon, a spawn of the escalating narco wars. Normal people imbue themselves and others with causative powers. This is exemplified by the exchange with Norma Jean.

"You don't need to kill me. I ain't got any money."

"I'll flip a coin. It's the best I can do. Call it."

"No, I ain't doin' that. It ain't the coin that decides. It's you, it's always been you."

"I got here the same way the coin did. Call it."

It's only first year philosophy students who will consider this a real ethical conundrum.

Or people who still believe in free will.

>the algorithm isn't going to be tried for murder.

Who, then?

Someone built it and deployed it. These aren’t naturally occurring phenomena. If someone sets up an automated turret in a field that just fires at anything that moves, the person who put it there is responsible, not the turret.

Drones did not decide to fly to Libya and shoot “hostiles” all by themselves.

So if one corporation, let's say Skynet, built an autonomous killer robot and sold it to any interested government, who would be responsible?

Is Skynet suddenly responsible for the outcome of every situation the robots are placed in? What about the secondhand robot market? Are the governments who own the bots responsible? Are the governments responsible for errors? If your car's brakes get stuck and you hurt someone, isn't the manufacturer responsible? If a gun you own misfires due to a defect, wouldn't the manufacturer be responsible?

To me it seems that autonomous warbots are going to put the world in a tighter MAD bind. If X has killer robots, Y better have killer robots or strongly align themselves with X.

To break this down further in a sort of visual way:

- Agents involved in a bot-involved death

-- Operator

-- Owner

-- Manufacturer

-- Programmer/Engineer

-- Government representatives

-- Victim

-- Represented populace

What would make sense to me is that the represented pressure their representatives to regulate manufacturers in terms of sales and safety controls. If a sale is found to be by regulation then the manufacturer should be less liable, presuming their programmers and engineers followed safety regulations. If the sale is valid, then the operator should be responsible, otherwise the manufacturer is responsible as well. In the case of malfunction, the manufacturer should be held responsible along with individual engineers and programmers. If the malfunction was due to a gap in regulatory strictures, then the government responsible for the regulation, along with the manufacturer, should be held responsible.

See? Easy.

To hold a soldier/operator criminally responsible for deploying an autonomous weapon that commits war crimes, prosecutors would need to prove both actus reus and mens rea, that is, a guilty act and a guilty mind.

This would be difficult as a matter of law, and possibly unjust as a matter of morality, given that autonomous weapons are inherently unpredictable.

The distance separating the soldier from the independent decisions made by autonomous weapons in rapidly evolving environments may be simply too great.

There’s always criminal negligence.

But I wish governments could just ban AI

Re ipsa loquitur, may suffice.

I mean, look at landmines, the original autonomous killing weapon.

There is no ambiguity who is responsible.

If a team of 10 coders worked on a robot that went on a murdering spree, does every coder go to jail?

Personally? Yes. They brought the thing about. If they did not do their due diligence to endure that outcome eouldn't happen, responsibility, when traced out lay with them.

If you were hoping to trip me up on "how many layers of seperstion until it's not your fault", you got the wrong guy. Unfortunately, I endure maintaining some of the most banal systems imaginable specifically because if I can reasonably forsee an outcome I'm not comfortable being culpable for, I don't work on it.

...You'll slso be surprised how muchoral quandry there is in many systems we take for granted.

It's s curse.

Ah, yes, like the good old “we sent these soldiers there and we will go to prison for their crimes” which never happened.

Soldiers:

- aren’t naturally occurring phenomena

- just fires at anything that moves (though it wasn’t gp’s idea, so you may add “with a rifle” suffix to it)

- did not decide to fly to Libya and shoot “hostiles” all by themselves

Someone is responsible for giving it executive function. The person culpable should be held accountable for the actions they let it make under that circumstance. In the case of AI models, training data should be subpoenaed and cross-referenced with the curators of the data or whoever oversaw the training of the model.

Landmines are autonomous weapons, just much slower than a computerized missile. If you think landmines should be banned, think the same regarding "intelligent" autonomous weapons.

To be clear this is the argument you appear to be making:

If I get a gun, and I shoot it without aiming, and then it hits someone, it doesn't matter that I wasn't aiming, it's my fault.

If I try to making it not my fault by putting the gun in a vice, with a mechanism that randomly pulls the trigger, and then it hits someone, it's my fault.

If I put that vice on a drone, and have the drone fly around randomly, still shooting randomly, and it hits someone, it's my fault.

If I make that drone start following a specific person, firing randomly, and it hits someone, it's my fault.

If I make that drone start following a specific person, shooting specifically at that person, it's my fault.

If that drone accidentally hits someone other than the target it is now ... not my fault?

If that drone gets confused and is following the wrong person, and hits that person it is now ... also not my fault?

I could say the same thing if I sprayed anthrax on someone.

Autonomous weapons have existed for decades, we just call them landmines.

A landmine is not an autonomous weapon. Autonomy suggests being able to select a target and harm only that target. A landmine just blows up when someone steps on it; there is no selection, thus no autonomy.

A landmine is programmed to attack a target of a specific weight when it enters a specific area. The targeting model was programmed by engineers to try and target soldiers (or armoured vehicles), and the locations they were placed were also for that purpose. However, it's possible for the heuristics to fail, and for a mine to blow up a child, because the weights weren't tuned correctly, or to fail to explode when a soldier steps on it.

By your same logic, any AI has no autonomy, because it does X when its code selects X. A guard dog robot attacks will act consistently given identical inputs.

> A landmine is programmed to attack a target of a specific weight when it enters a specific area.

You've literally just said it is programmed. If it is programmed, how can it be autonomous?

Because it kills without consulting the people who put it there.

AI is also programmed; it attacks when the score returned by the threat function exceeds a threshold.

Unless we're talking about AGI, in which case I misunderstood and assumed we were talking about AI that we can currently make.

1) Mines have had this problem for more than a hundred years, and mines from previous wars are still misidentifying targets to this day

2) Mines have also had this problem

3) Not sure this is even grounded in reality - WW2, Vietnam, Afghanistan, and Iraq wars showed no significant "concern for civilians abroad". WW1&2 showed the concern about life of a countries own soldiers was only important in as much as running out of them meant losing.

4) Seriously? If I make a machine that targets tanks, but I do it badly and it confuses them with school buses, the laws of wars would make me responsible. But that's also not a good argument in general, because the "laws of war" have only ever applied to the defeated.

> Who would be put on trial?

If it is done by a powerful country, no one. If it is done by a less powerful country, someone. The same as is the case now.

How is an autonomous weapon any different than a guided missile?

Reply on news.ycombinator.com