Hacker News Reader

Comment by credit_guy | Hacker News Reader

Drones can be, and have been jammed, or even captured [1]. You need now, and will continue to need for the foreseeable future pilots in planes. But teams of piloted fighters and drones? That will be the winning combination. And how can a single human command a flock of 20 aircraft? By giving short orders, and letting AI taking care of the rest. The ethics of killing will never be left to AI, but the technicalities of acquiring and prosecuting the target? You bet.

[1] https://en.wikipedia.org/wiki/Iran%E2%80%93U.S._RQ-170_incid...

View on HN

> The ethics of killing will never be left to AI,

Ha. I know of a device with a very simple AI program that can tell if an enemy is standing on it. If so, the enemy is killed. It’s called a mine. They have been used in every land (and naval) war since they were invented.

I don't think mines quality as "AI", unless you stretch the definition to just mean anything under the sun.

I think they just meant that the military long left the “kill/don’t kill” decision along the wayside despite the propaganda of saying we have someone in control of the kill vector.

The whole discussion was about AI as far as I can recall.

Mines are being compared as something that makes worse decisions but gets put in charge anyway.

I think you’re right and I wish they would have just said that.

It’s a good point.

But AI's can qualify as mines. Sending out AI drones is like deploying mines that kill anything within a certain radius.

AI supposes a certain level of complexity - if not we just use it a buzzword with virtually no signification.

You're missing the point. The point is, we have very stupid systems making kill/no-kill decisions today if you step on top of them or drive by them. Slightly smarter systems making kill/no-kill decisions within a designated envelope should not be surprising.

I'm just saying the ethics of deploying AIs to kill is logically equivalent to deploying mines.

AI _is_ a buzzword with questionable signification.

The level of complexity is arbitrary. Maybe the AI won't blow something up unless it has a heat signature. Maybe it has to detect a pulse. Maybe it won't blow up something that emits a certain beacon. Maybe the kill has to be within a certain timeframe. Maybe the target had to have a certain mass. Which of these considerations qualifies as AI?

That may be a valid technical complaint, but the problem remains—we create devices that are designed to kill humans, autonomously, without human decision inputs.

From a different perspective you might think of guns and bombs as having similar problems—a human might decide to pull the trigger, but the human involved has limited control over who is killed and what is destroyed.

Well let me know when the mines are walking around and deploying themselves. In the meantime they're not AI.

Let me know if I understand your priorities correctly—you care about whether the word “AI” is being used correctly, but you are not interested in discussing whether machines are autonomously killing humans?

I think this has to be a pretty classic example of the way discourse can get derailed on HN, Stack Overflow, or the like—Alice has an interesting point to make, but Bob takes the floor to point out that Alice used a word incorrectly, even though he knows what Alice meant.

The discussion about landmines is actually a good point—the legal language around landmines has evolved somewhat over the years and you might hear them referred to as “LAWs” these days, which stands for “Lethal Autonomous Weapons”. So many of the the same treaties will regulate both autonomous drones and landmines, with the same language.

It's not. Landmines aren't placed randomly. They also do not discriminate in what they kill. They are a simple explosive with a step trigger or proximity sensor. That does not an AI make. It completely deludes the definition of AI to call any switch based mechanism an AI. Words and meanings matter and it cannot mean whatever you want it to mean or it loses all capacity to communicate an idea.

Is an elevator an autonomous vehicle?

If we're going to dive into the rabbit hole of "what is exactly the right word here" I would have to say that an elevator is an automatic vehicle.

> Words and meanings matter and it cannot mean whatever you want it to mean or it loses all capacity to communicate an idea.

Words and meanings are not equally important. Words are only there to communicate meaning. You can communicate meaning without words, and you can even communicate the correct meaning with incorrect words.

There's an interesting discussion to be had here, but "are landmines AI" isn't it.

Fine I'll bite but I still think the conversation is asinine. Landmines are not placed randomly. They are intentionally placed to protect or guard an area. They don't make decisions about targets, they simply react. Of course there's a fundamental difference in strapping warheads to an AI controlled vehicle and allowing it to select and determine its own targets...and dumb switch based landmines. The latter will work as intended: if someone steps on the mine, the mine goes kaboom. There's no risk of the mine uprooting itself and carrying itself into an urban civilian population center.

> There's no risk of the mine uprooting itself and carrying itself into an urban civilian population center.

And yet, landmines kill an estimated 15-20,000 civilians each year. About one in five are children. All because someone decided to defend an area. You say this is “working as intended”, but—how can this be the intended result?

Automatic weapon systems allow you to make decisions about who lives and dies, and they let you escape responsibility by making those deaths occur at a different time/place from when and where you caused those deaths. If you were defending an area with teammates and friends’ lives on the line, you might be a little less careful about how your decisions affect the lives of civilians—especially the civilian deaths are uncertain and distant.

The whole point is that the initial argument was not "AI bad", it was "should we allow weapons to kill without direct human involvement". That is the moral issue with AI weapons. The GC was merely pointing out that we have already done this with things like mines.

There exists an even simpler AI program, wherein an edge is pressed against an object. If done so with sufficient force, the object is cut. This AI is called a knife.

Not a good analogy. The human soldier driving the knife is the agent here, unlike the trigger or tripwire on a landmine.

I know of someone who worked on face-recognition AI landmines for the US. They only explode if they recognize you. I don't think anything like that has actually been used though.

> They only explode if they recognize you.

I’d go back and check with product on the requirements for that one. Sounds like the opposite of what I’d want.

Yeah but presumably your own troops know where your land minds are, so having them not explode if they recognized your face would be kind of stupid.

I think maybe those landmines are supposed to kill specific targets, and avoid collateral damage.

You got a point. I think the controversy lies in situations where an AI has to differentiate between threats and innocent bystanders.

So really what the OP means is the ethics of what to kill and what to let live should not be left to an AI.

There are naval mines which can detect a vessel nearby and launch an autonomous homing torpedo. All with no human in the loop.

https://www.thedrive.com/the-war-zone/25235/the-u-s-is-getti...

These examples of “no human in the loop” are stretched so wide as to lose all meaning. A human designed the mine to destroy anything that meets the programmed requirements (weight on mine or proximity to mine).

You don’t have a case of no human in the loop until that mine is capable of exploding or not exploding and it’s responsible on a case by case basis for deciding which to do.

The naval mine isn't triggered by a simple contact switch. It is capable of launching or not launching the torpedo on a case by case basis depending on pattern matching from multiple sensors.

I guess that's why must countries have agreed to ban their use

https://en.m.wikipedia.org/wiki/Ottawa_Treaty

With some notable exceptions:

United States, China, Russia, Saudi Arabia, Singapore, India, Pakistan, United Arab Emirates, Egypt, Iran, Israel, Vietnam,North Korea, South Korea, Armenia, Azerbaijan, Bahrain, Cuba, F.S. Micronesia, Georgia, Kazakhstan, Kyrgyzstan, Laos, Lebanon, Libya, Mongolia, Morocco, Myanmar,Nepal, Syria, Tonga, Uzbekistan.

Those countries together represent a significant portion of the global population!

https://en.m.wikipedia.org/wiki/List_of_parties_to_the_Ottaw...

They don't have enemy (friend/foe) recognition though, they kill everyone.

So, it's worse, right?

Is a razor wire fence, then, an autonomous knife?

Wouldn't the spear pit, or otherwise camouflaged stake-in-the-ground, pre-date the razor wire fence as an autonomous knife?

Or further still, doesn't the camouflaged trap, or non-attendance-required trap of any sort in general, become the original autonomous tool?

So these "mines" don't explode if a friendly is on them, or a child?

>The ethics of killing will never be left to AI, but the technicalities of acquiring and prosecuting the target?

This is unfortunately completely wrong. Fully autonomous weapons have already deployed for years, and are currently in use on the North/South Korean border. Fully autonomous weapons systems, including drones that use AI to autonomously choose and destroy their targets have been under development for many years.

https://www.nato.int/docu/review/articles/2017/07/28/autonom...

There's a reason experts are sounding the alarm - because the ethics of killing are being shifted to AI right now.

>Kallenborn, an expert in unmanned systems and WMD, describes one type of swarm that he calls an Armed, Fully-Autonomous Drone Swarm, or AFADS. Once unleashed an AFADS will locate, identify, and attack targets without human intervention

https://mwi.usma.edu/swarms-mass-destruction-case-declaring-...

Aren't current drones mostly remote-controlled?

In this case the AI is onboard, right? So much lower surface area for being hacked.

Goodness. Are you suggesting AI-powered flying killing machines that we can't supervise or control remotely (even temporarily due to jamming) are ok?

They’re called cruise missiles.

Touche

> The ethics of killing will never be left to AI, but the technicalities of acquiring and prosecuting the target?

Other have already answered this, but I want to chip in with my own experience working for a submarine and military boat manufacturer + talking to navy officer.

They already have system to acquire and try to distinguish friend of foe. The only reason why the system is not pushing the launch button by itself is because there is a lot of tactic involved in starting an engagement. Especially for submarines who might prefer to remain hidden than launch a torpedo which will uncover its position. Also, those system are not perfect, one navy officer who was on radar duty told me that he activated the alarm on his ship after seeing something on the radar that ended up being a flock of seagull.

If the system was able to manage tactic and be able to distinguish friend of foe reliably, they would give it the power to kill.

> The ethics of killing will never be left to AI

I hope you're right, but what makes you so sure? In a wartime scenario, if a country is short on skilled personnel why wouldn't they release AI combatants if they had the tech? Especially if they're facing a perceived aggressor or defending their territory.

You're hoping against something that is already historical fact. Do you mean that you hope it will no longer be true in the future?

If AI has made a decision to kill without human approval, I haven't heard about it. Do you have references I could look at?

The ethics of killing is really down to the politicians / leaders whether the end job is done by a human or a machine.

Keep in mind that electronically jamming a modern fighter is effectively downing it, irrespective of the human inside. I have read that are beyond the ability of a human to fly directly, and absolutely require fly-by-wire to stay aloft.

As for taking control of an AI drone, you can eliminate that possibility by only allowing abort orders remotely.

Classical drones still require a remote human. This is a different league.

"electronically jamming a modern fighter" isn't going to do anything to its flight controls.

You're correct that flight computers are generally required to keep dynamically unstable aircraft flying -- but that's done at a much lower level than anything that's connected to a radio.

Reply on news.ycombinator.com