Hacker News Reader

Comment by lumost | Hacker News Reader

There is such a blurry line for autonomous munitions. militaries used dumb imprecise munitions for decades - then precision weapons.

A2A missiles used to lock on radar signature leading to huge risks related to accidentally shooting airliners/friendly craft. Now antiship missiles dynamically select their target over 300km away to maximize the chance of hitting a big ship.

During the war on terror, ML models would decide which phone to blow up. We’re probably going to see ai driven target selection and prioritization for fire control within the next few months of the Ukraine war. The US’s new Rapid dragon program almost demands ai control of target selection and flight trajectories.

Where do you draw the line? What would an appropriate agreement look like?

View on HN

This comment appears to be getting upvotes, so I am going to jump in and say that it is very clear to anyone who knows that this commenter has no idea how AI is being used in the military or how military targeting works in the West.

These things are generally classified, but algorithms are nowhere close to being considered decision-capable and human-in-the-loop targeting will be the norm for quite some time.

Even next generation systems that are being considered are still human queued and the AI comes in during the engagement step.

If you look at the stages of the F3EA cycle (Find, Fix, Finish, Analyze) there is no single step which is currently offloaded to an algorithm.

Well, then I am going to jump in to say that insiders who jump in to assure us that due to their classified insider knowledge they can reassure us it's not like that at all... you realize this doesn't work, right? Not on anyone who doesn't already believe it. Even if you were right, you're simply not trustworthy.

There are two things I wish I could make you and your friends understand: one is how deep you've dug that credibility hole. There was a price to pay for all that secrecy.

The other is, when you're out here trying to manage the non-classified public's perceptions, why on earth would you trust that your bosses aren't managing yours? Why would you think you get to know what the long term plans are?

Ehh, I don’t work there anymore. Just got nerd sniped by someone being obviously wrong on the internet.

> one is how deep you've dug that credibility hole. There was a price to pay for all that secrecy.

It’s a weird thing that the public thinks the defense sector doesn’t know this. They’re aware. I could try and reason with you on the internet, but this isn’t a place for changing people’s opinions.

I would instead tell you that people who work in defense tech tend to be motivated by morals and any argument that what they’re doing is dangerous is an implicit argument that they’re intelligent (otherwise, it wouldn’t seem too dangerous).

So, given those two facts — you can probably do a better job of reasoning about their motives than this.

> The other is, when you're out here trying to manage the non-classified public's perceptions, why on earth would you trust that your bosses aren't managing yours? Why would you think you get to know what the long term plans are?

Just bored on HN on a Monday. Same as anyone. Obviously not working on classified stuff, or I wouldn’t have access to HN.

Just because things are secret from the public doesn’t mean the defense industry is some cloak and dagger affair. It’s painfully boring bureaucracy. Decisions happen on the order of weeks to years, across many meetings. Everyone in the room knows the long term plans.

> It’s a weird thing that the public thinks the defense sector doesn’t know this. They’re aware.

Their actions aren't consistent with being aware. Your post wasn't consistent with being aware. They think they understand, but they act in exactly the way they should not act, digging the hole of distrust deeper.

Look at Keith Alexander going to work for Mohammed bin Salman and Saud al-Qahtani. Is that consistent with being "motivated by morals"? Do you think everyone working under him understood that such were his personal long-term plans?

There are two problems with your argument. You’re trying to reason from a specific case to the general and your specific case isn’t even that great.

Keith Alexander retired from active duty 10 years ago. He’s now, currently, on the Amazon board of directors. Does this make every Amazon employee culpable for his actions?

Generally speaking, yes, it is not uncommon for leadership to contract with Saudi Arabia. I did not and continue to not offer any opinion about this nonsequitur. I do not think it’s at all common for them to share secrets. Further, I have no idea what this has to do with the morality of secrets as whole.

I gave an example, I did not "reason from a specific case".

The point is not that your organization gets "tainted" by Alexander's actions. You must answer for your own sins, whoever your leader is.

The point is that he was always that sort of person, and that's the sort of people who become your leaders. Yes, it's a problem that completely amoral people get high positions in business too, but Amazon board members are not quite as powerful as NSA chiefs - among other things, because Amazon is not as good at convincing its workers to identify with it, like you identify with your mil-int employers.

>people who work in defense tech tend to be motivated by morals

Joke of the year

Not to Ukrainians

It is to the Iraqis/Afghans/etc

Sure, gp implied morality was absurd. I didn’t say it was always correct, only that it’s possible to be intelligent and motivated by some set of morals.

I don’t think most of this stuff is a secret… I watched one of the recent documentaries about the frantic exit of Afghanistan and its effect on the Afghan military. There were scenes where the military used drones on various targets, one dude made the decision at the end.

Where is the human in the loop for typical mine deployments? We already have autonomous killing machines, it's just that they are not very sophisticated, often having trouble to tell children from tanks.

I'd expect that mine manufacturers jump on the A.I. bandwagon to suggest that weapons are designed to be less harmful to civilians, and should not be subject to bans on certain types of mines.

The US already uses self-deactivating mines when deployed (after a couple days or months they go inert on their own). The trouble is like any system the mechanism can fail to deactivate (1 in 100 was an estimate I remember from 10 years ago).

"SKYNET is a program by the U.S. National Security Agency that performs machine learning analysis on communications data to extract information about possible terror suspects. The tool is used to identify targets, such as al-Qaeda couriers, who move between GSM cellular networks. Specifically, mobile usage patterns such as swapping SIM cards within phones that have the same ESN, MEID or IMEI number are deemed indicative of covert activities.[1][2] Like many other security programs, the SKYNET program uses graphs that consist of a set of nodes and edges to visually represent social networks.[3] The tool also uses classification techniques like random forest analysis. Because the data set includes a very large proportion of true negatives and a small training set, there is a risk of overfitting.[1] Bruce Schneier argues that a false positive rate of 0.008% would be low for commercial applications where "if Google makes a mistake, people see an ad for a car they don't want to buy" but "if the government makes a mistake, they kill innocents."[1]

The SKYNET project was linked with drone systems, thus creating the potential for false-positives to lead to deaths.[1][5]"

https://en.wikipedia.org/wiki/SKYNET_(surveillance_program)

I generally agree but there are munitions that are capable of fully automated no human in the loop target selection, like LRASM, CAPTOR, or Brimstone. The military is very conservative about this stuff, as they're the last people that want a rogue weapon, but there is a clear trend towards increasing these capabilities because of EW systems denying the ability to have a human in the loop.

Humans are going to be taken out of the loop at some point if for no other reason than being too slow.

Already happened with the Aegis Combat System. Humans are too slow for combat with supersonic missiles. Operators can put it into a fully automated mode and it will detect, classify, prioritize, and engage targets with no human in the loop. This all uses regular deterministic logic and doesn't rely on what we would usually call "AI".

I don't know the first thing about autonomous weapons systems, but nothing the GP said sounds technically infeasible. And what does the game theory look like once one party has AI that can make decisions without humans? Wouldn't that be a huge "advantage" in combat?

The military has practically zero accountability to the public. "National security" gets suits dismissed.

Decision-capable is entirely subjective, and thus imperfectly controlled by bureaucracy.

You don't know for sure no one is already doing this.

The line already drawn, no indiscriminate killings of civilians, all this technology only minimizes civilian suffering, furthermore it is more humane to use terminators instead of 20 years olds clearing buildings and autonomous tanks/tracks driving though IEDs filled streets.

War is deeply in human nature, it is not going away, question is; how to make it more humane and minimize indiscriminate killings and sufferings in most cases poorest populations on earth.

"The line already drawn, no indiscriminate killings of civilian"

In modern warfare, civilians have always been the main casualties of war. Whether this really counts as "indiscriminate" is for war lawyers to argue over, but the fact remains that the civilian toll far exceeds the military toll, and for the victims and their families it matters little whether their deaths were "justified" to some lawyer or politician.

As technology advances, more and more power is going to be concentrated in the hands of individuals, many of whom won't be bothered by niceties such as the Geneva convention, and will target those they hate, be they civilian or military. High tech weapons will only make this easier.

We are headed for a very dark time.

The modern era has dramatically reduced civilian casualties compared previous generations. Moreover the decline is in absolute terms, despite the fact that global population has been increasing in the same time frame. Precision munitions contribute heavily to this reduction. All in all, war has become a lot safer for the typical civilian.

Not true. The 20th century saw more civilian deaths than the total world population for most of human history.

Only in absolute numbers. But e.g. Mongol conquests wiped out >10% of the entire world population over the course of about a century. All of our 20th century wars don't add up to that.

GP claimed the decline was in absolute terms, not just percentage of population. On that note though, WW2 killed 3% of the world population in the span of 6 years.

"The modern era has dramatically reduced civilian casualties compared previous generations." Or perhaps put a temporary dampen, at the cost of increased probability of increased magnitude apocalyptic scenarios. We are flirting with nuclear war as we speak, at risk levels higher than the most nightmarish Cold War scenario. Time will tell.

Current state of nuclear threat is not even remotely comparable to Cuban Missile Crisis.

There is a hot war in Europe funded by USA and a drone just hit the Kremlin.

> All in all, war has become a lot safer for the typical civilian.

Tell that to the 387,000 civilians killed in the "war on terror", roughly equal to the number of "opposition fighters"

https://watson.brown.edu/costsofwar/figures/2019/direct-war-...

> Precision munitions contribute heavily to this reduction.

Tell that to all the wedding parties blown up by precision munitions launched from drones because somebody's uncle, who exchanged texts with an "opposition fighter" and keeps an AK in the bed of his truck, showed up.

~226,000 people were killed in Hiroshima and Nagasaki alone, most of which were civilian.

Shall we include people drafted into a war they didn't want to fight? People shot by their own countrymen because they didn't want to go over the top as they were a scared civilian with minimal training being used as cannon fodder? Those same scared civvies with minimal training who did go over the top and got mowed down by machine guns and shells?

Shall we do civilian resistance groups next? Whole villages of France that were shot dead? You can still visit at least one of them and see it as it was left. It's shocking, and it's left like that as a reminder of how shit things were.

Yes, proxy wars are terrible, as are all wars. All loss of life is horrible. Nobody is saying anything to the contrary. But saying it isn't less terrible is blatant denialism. These are all people who are all someone's son or daughter, both now, and then.

You need to look at ratios. WW2 was such an unimaginably large scale war, that anecdotes don't really tell you anything. In WW2 the Allies lost 16,000,000 military and 45,000,000 civilians. The Axis powers lost 8,000,000 military and 4,000,000 civilians. Comparing this to e.g. Iraq is difficult due to a lack of reliable source of casualties, so we'll have to ballpark it. Leaked US figures (Iraq War documents) claim we killed 45,000 enemies, which is going to be a heavy overestimate. Civilian deaths in Iraq range from 110,000 to 1,0333,000. I'll pick a meet in the middle of 500,000. Using those figures we can measure this objectively:

Civilians deaths per combatant death:

---

WW2 Allied Forces = 2.8

WW2 Axis Powers = 0.5

Iraq War Iraqis = 12.7

---

Modern wars are smaller in scale (for now), but much worse on civilians as a ratio where they do happen. The reason is because of the nature of wars we get into. We invade countries which cannot competently defend themselves, and so it immediately transforms into an asymmetric Guerilla style defense against an occupying force. And in these scenarios its impossible for the invader to know who is an enemy and who is a civilian, so civilians suffer just terribly and over very long periods of time.

The final drone strike of Afghanistan [1] is quite a symbolic one that will certainly go down in history. The US military initially claimed they killed a group of Islamic State forces planning an attack on US forces, after observing an ISIS militant placing explosives in his trunk. In reality they killed 1 man, 2 adult members of his family, and 7 children from the local neighborhood (who ran out after he honked his horn when getting home). Those "explosives" were bottles of water, and the "ISIS militant" was a longterm humanitarian aid worker, who was working for a US NGO and applying for a US visa. If not for the excessive media attention on the final strike of the war, that would likely have just been marked up as another successful strike, with some unfortunate collateral damage. And that was one day in a 20 year occupation.

[1] - https://apnews.com/article/afghanistan-kabul-taliban-strikes...

This is almost like arguing that ISIS inflicted civilian casualties are only due to US air strikes, because it reads to me like you are arguing that Iraqi civilian casualties are not due to unimaginable number of IED's in civilian population, and not due to using combat storage/staging ares in civilian population, in addition, to country as whole being in a civil war a like state of many years.

This is actually a pretty interesting issue, which I hadn't bothered to look up before. A large chunk of all violent deaths are directly attributed to the US Coalition. For instance here [1] is the Lancet study, which directly attributes 186,000 violent civilian deaths to the Coalition alone. That yields a 4.1 ratio due to direct violence from the Coalition alone, which is itself already far worse than even WW2.

And that's extremely surprising to me. When you look at things like WW2 civilian deaths you're not only looking at violent deaths caused by the enemy. You're looking at deaths caused by all involved powers as well as indirect deaths caused by the nature of war - starvation, disease, despair, etc. The fact that one side, alone, in modern warfare can cause more violent civilian deaths (as a ratio) than all of those factors combined, in past wars, really emphasizes the notion that the concept of modern war being better for civilians is just exceptionally misguided.

The history books of the future are going to look back on the present in a way I think few can imagine today.

[1] - https://sci-hub.ru/https://www.thelancet.com/journals/lancet...

"Whole villages of France that were shot dead" That was with gloves on. The Eastern Front was on a whole different level. I stumbled at some point upon the fate of Belarus under 3 years of Nazi occupation, and just could not wrap my head around it.

"Altogether, more than 2 million people were killed in Belarus during the three years of Nazi occupation, almost a quarter of the region's population,[1] including 500,000 to 550,000 Jews in the Holocaust in Belarus.[2]"

"At least 5,295 Byelorussian settlements were destroyed by the Nazis and some or all their inhabitants killed (out of 9,200 settlements that were burned or otherwise destroyed in Belarus during World War II),[3] and more than 600 villages like Khatyn had their entire population annihilated.[3]"

https://en.wikipedia.org/wiki/German_occupation_of_Byeloruss...

Well, the French villages were over civilian resistance in wartime.

I'm Ashkenazi Jewish myself, and I don't like to be the guy invoking The Holocaust at every opportunity because there's plenty of other demographics usually overlooked, and The Holocaust itself wasn't as civilian war casualties, it was state-sponsored murder, and the topic at-hand is civilians getting killed through the war itself, thus invoking The Holocaust would skew the stats on wartime civilian casualties if it was to be included relative to more recent conflicts. Commonly, the victims are counted separately: https://en.wikipedia.org/wiki/World_War_II_casualties

But yes, it's horrifying, and I find it difficult to understand the true scale of. I've been in stadiums with tens of thousands of people, that certainly seems like a lot of lives, a lot of families, a lot of humanity. Hundreds of thousands or millions? Yeah, it boggles my mind, and it really wasn't very long ago whatsoever.

> Tell that to the 387,000 civilians killed in the "war on terror", roughly equal to the number of "opposition fighters"

The fact things aren't perfect doesn't mean they're not objectively better.

No one here said that war is NOT hell, what exactly are you trying to say?

Also wasn't "opposition fighters" counting every able bodied man, regardless of affiliation?

In the good old days we intentionally bombed entire cities with the goal of killing as many innocent civilians as possible. It wasn't a technological issue. The goals have changed since then.

https://en.wikipedia.org/wiki/Bombing_of_Dresden_in_World_Wa...

https://en.wikipedia.org/wiki/Atomic_bombings_of_Hiroshima_a...

You should point to the firebombing of Tokyo [0]

Arguably more destructive than Hiroshima.

0: https://en.m.wikipedia.org/wiki/Bombing_of_Tokyo

> It wasn't a technological issue.

Precision guided munition is 90's technology, this removes any need/justification to carpet bomb anything, today even artillery shells are guided (is in biggest, longest range guns have accuracy in meters).

> this removes any need/justification to carpet bomb anything,

You totally missed the point. The mass bombing in WW2 was intentional. That was specifically the goal, to kill as many civilians as possible.

I guess I fail to see how this is relevant to current and future military technology, where name of the game is stealth, high precision, range, speed, and intelligence, and not in any way about building large number of bombs/cannons that fire in general vicinity of the enemy location.

This is going off topic, but originally I was replying to this comment: "The modern era has dramatically reduced civilian casualties compared previous generations... Precision munitions contribute heavily to this reduction.". I was trying to explain that the cause of excessive civilian casualties in WW2 was not due to technological limitations. It was fully intended. It was the goal.

Are you arguing that war without modern technology had less civilian casualties ?

What are your measures? How do you compare WWI and WWII, for example?

It is really hard, if not impossible to compare civilian causalities directly effected by weapon systems, no two wars are same, not all wars culminated within urban area(s), parties in conflict use different weapons systems, and engagement rules, etc.

For example, hellfire missile is more effective and substantially less destructive then carpet bombing few cities blocks, furthermore video feeds from drones provide some accountability.

This makes me think about the ghost in the shell movie. When you have that kind of technology at your disposal, you can inflict great suffering.

> furthermore it is more humane to use terminators instead of 20 years olds clearing buildings and autonomous tanks/tracks driving though IEDs filled streets

Are you trolling? Because it's exactly this sort of "righteous war" rationale that props up empires and totalitarian dictatorships alike.

It's also human nature to rationalize evil and wish it into "good".

Seems like you misunderstood, it is not about ethics/justification to wage the war, but only about use of technology in violent conflicts.

I would like to join other commenters in questioning whether or not civilians are already routinely and indiscriminately killed.

No civilians are indiscriminately killed - absolutely true [1].

[1] for specific interpretations of "indiscriminately".

Yet rhetoric against war seems to be about poor American soldiers who give their lives abroad, rather than civilians. I don’t think it’s far fetched to think that people will care less about humanity and civilians if the mortality is reduced for your own army.

> War is deeply in human nature, it is not going away

This is:

* Not a consensus view

* Impossible to prove with regard to the future nature of humanity

* Not the working assumption for those of us innovating around peacetime tech

I am not aware of any theory that would suggest humans in forcible future will eradicate violent conflicts, regarding past there is great book;

War Before Civilization: The Myth of the Peaceful Savage

History tends to disagree with your Pollyanna view

And nothing could ever change? Just stupid conservatism and self-fulfilling prophecy due to enough people still thinking like that.. ;)

“If only we could destroy those in the way of changing the world for the better!”

> Where do you draw the line? What would an appropriate agreement look like?

Good questions. I'm not sure, but that line needs to be drawn somewhere. Thousands of experts should be pouring over questions like these, and likely will.

Here's an interview[0] with Lauren Khan from the Council on Foreign Relations[1] who is indeed pouring over those questions.

[0]https://youtu.be/CDoIPpcwnCE

[1]https://www.cfr.org/expert/lauren-kahn

Who pays for those experts, and what are their incentives?

Reply on news.ycombinator.com