Hacker News Reader

Comment by batch12 | Hacker News Reader

So the study[0] involved people making simulated drone strike decisions. These people were not qualified to make these decisions for real and also knew the associated outcomes were also not real. This sounds like a flawed study to me.

Granted, the idea of someone playing video games to kill real people makes me angry and decision making around drone strikes is already questionable.

> Our pre-registered target sample size was 100 undergraduates recruited in exchange for course credit. However, due to software development delays in preparation for a separate study, we had the opportunity to collect a raw sample of 145 participants. Data were prescreened for technical problems occurring in ten of the study sessions (e.g., the robot or video projection failing), yielding a final sample of 135 participants (78.5% female, Mage = 21.33 years, SD = 4.08).

[0] https://www.nature.com/articles/s41598-024-69771-z

View on HN

> Granted, the idea of someone playing video games to kill real people makes me angry and decision making around drone strikes is already questionable.

For that first part, though, what does that even mean? The military isn't gamifying things and giving folks freaking XBox achievements for racking up killstreaks or anything. It's just the same game people have been playing since putting an atlatl on a spear, a scope on a rifle, or a black powder cannon on a battlefield. How to attack the enemy without being at risk. Is it unethical for a general officer to be sitting in an operations center directing the fight by looking at real-time displays? Is that a "video game?"

The drone strikes in the Global War on Terror were a direct product of political pressure to "do something, anything" to stop another September 11th attack while simultaneously freaking out about a so-called "quagmire" any time someone mentioned "boots on ground." Well, guess what? If you don't want SOF assaulters doing raids to capture people, if you don't want traditional military formations holding ground, and you don't want people trying to collect intelligence by actually going to these places, about the only option you have left is to fly a drone around and try to identify the terrorist and then go whack him when he goes out to take a leak. Or do nothing and hope you don't get hit again.

>For that first part, though, what does that even mean? The military isn't gamifying things and giving folks freaking XBox achievements for racking up killstreaks or anything. It's just the same game people have been playing since putting an atlatl on a spear, a scope on a rifle, or a black powder cannon on a battlefield. How to attack the enemy without being at risk. Is it unethical for a general officer to be sitting in an operations center directing the fight by looking at real-time displays? Is that a "video game?"

It's not the same thing. Not even close. Killing people is horrible enough. Sitting in a trailer, clicking a button and killing someone from behind a screen without any of the risk involved is cowardly and shitty. There is no justification you can provide that will change my mind. Before you disregard my ability to understand the situation, I say this as a combat veteran.

Would you say the same about a drone operator in ukraine, attacking russian troops on ukraine territory? I feel like there is still a distinction here. Between "traditional" warfare and those "anti-terror" operations.

I'm talking about long range UAV drone strikes. Short range consumer-grade drones strikes aren't something I have experience with. To me, the idea feels similar to the that of IEDs, mortars, claymore mines, etc which also suck, but not the same thing. Landmines are up there though.

FPV and scout drone operators in Ukraine are very much at personal risk, they operate within several km of the front line. This is in range of mortars, artillery, snipers, autocannon, tanks, etc.

> clicking a button and killing someone from behind a screen without any of the risk involved is cowardly and shitty

It's actually strategic, and if you're fighting a peer adversary, they will be doing the same.

That's true and also doesn't conflict with my quoted comment.

  >> For that first part, though, what does that even mean? The military isn't gamifying things and giving folks freaking XBox achievements for racking up killstreaks or anything. It's just the same game people have been playing since putting an atlatl on a spear, a scope on a rifle, or a black powder cannon on a battlefield. How to attack the enemy without being at risk. Is it unethical for a general officer to be sitting in an operations center directing the fight by looking at real-time displays? Is that a "video game?"

  > It's not the same thing. Not even close. Killing people is horrible enough. Sitting in a trailer, clicking a button and killing someone from behind a screen without any of the risk involved is cowardly and shitty. There is no justification you can provide that will change my mind. Before you disregard my ability to understand the situation, I say this as a combat veteran.

Respectfully, whether or not an action is cowardly is not a factor that should ever be considered when making military decisions (or any other serious decision). With that being said, my uncle was a military drone pilot in the 80s-90s and he said it's pretty much exactly like a video game, and doing well does give achievements in the form of commendations and promotions.

I'm not sure how this is different to conventional air support or artillery in an asymmetric conflict. A-10 and F-16 pilots weren't seriously worried about being shot down in Afghanistan, but I know plenty of infantrymen who have nothing but gratitude for the pilot who got them out of a tight spot. Do those pilots become more moral if your enemy has decent anti-air capability?

As a veteran, I respect your experience, but as a former aviator with flight time over Afghanistan I also resent the implication that an asymmetric physical threat is "cowardly." We play the roles we are given. Choose your rate, choose your fate. I, and drone crews, were ultimately there to keep other Americans on the ground alive, and we deserve better than contempt for fulfilling this role. A coward would not have stepped up, sworn the oath, and worn the uniform.

Also, "there is no justification you can provide that will change my mind" is not exactly something to brag about.

Sorry, I don't agree. Also, not bragging, just stating a fact.

>Also, "there is no justification you can provide that will change my mind" is not exactly something to brag about.

I've noticed a certain type of person tends to feel very threatened and attacked by the idea of someone else having an unwavering code of ethics that they firmly believe in

Unwavering beliefs is how you get atrocities.

Looking at history, asymmetry of force is just as responsible.

"Sitting in a trailer, clicking a button and killing someone from behind a screen without any of the risk involved is cowardly and shitty."

Do you think that drone operators never die in combat? There is plenty of deaths among drone operators in Ukraine. It is a pretty risky activity.

Trailer (or a trench) isn't a particularly good shelter against a Lancet or an Iskander, if the adversary can detect you, or multiple of 'you'.

The Russians would likely be willing to expend an Iskander on certain Mr. Madyar [0], if they could locate him.

[0] https://en.wikipedia.org/wiki/Robert_Brovdi

> For that first part, though, what does that even mean? The military isn't gamifying things and giving folks freaking XBox achievements for racking up killstreaks or anything

Fighter pilots have been adding decals keeping track of the number (amd type) of aircraft they have downed as far back as WWII.

And special terms for those who do well. "Ace" doesn't mean "good at it", it means "has shot down N enemy aircraft" (usually N is 5) - it implies good at it tho.

Not to mention the long history of handing out achievements in the form of medals/ribbons/etc.

It's probably fairer to say that in-game achievements were inspired by medals/ribbons, than the other way around.

Fighter pilots also know damn well what they are getting themselves into. That isn't the "gamification" I'm talking about. Ender's Game is fiction. I've seen drone strikes go down IRL, and no one involved mistook the gravity of the situation or the importance of getting it right. It's not what the other poster derides as a "video game."

> If you don't want [...]. [...] about the only option you have left is to fly a drone around [...] Or do nothing and hope you don't get hit again.

Meanwhile, just about the best increased defense against that attack happening again is that passengers will no longer tolerate it. Absolutely nothing to do with US military attacking a country/region/people.

In total war, however you got there, it won't matter. More dead enemy faster means higher odds of victory, and if digitally turning combatants into fluffy Easter Bunnies on screen to reduce moral shock value, giving achievement badges, and automated mini-hits of MDMA make you a more effective killer, then it will happen in total war.

I could even imagine a democratization of drone warfare with online gaming, where some small p percent of all games are actually reality-driven virtual reality, or gives real scenarios to players to wargame and the bot watches the strategies and outcomes for a few hours to decide. Something akin to k kill switches in an execution but only 1 (known only by the technician who set it up) actually does anything.

That a pet peeve that I have regarding corporate scientific communication nowadays: a clearly limited study with a conflated conclusion that dilutes the whole debate.

Now what happens: people that already have their visions around “AI-bad” will cite and spread that headline along several talking points, and most of the great public even knows those methodological flaws.

Automation bias is a well documented human problem.

I am not defending this study, but establishing and maintaining distrust of automation is one of the few known methods to help combat automation bias.

It is why missing data is often preferred to partial data in trend analysis to remove the ML portion from the concept.

Boeing planes crashing is another.

But why should we distrust automation? In almost every case it is better than humans at its task. It's why we built the machines. Pilots have to be specifically taught to trust their instruments over themselves.

Instruments are not automation. Trusting your instruments over yourself is different from trusting the autopilot decisions over yourself, which I think pilots aren't also taught.

> Pilots have to be specifically taught to trust their instruments over themselves.

There is a difference between metrics and automation. In this case, they are trusting metrics, not automation.

> But why should we distrust automation?

I can name one off the top of my head: exceptions.

Automation sucks at exceptions. Either nobody thought of it or it is ignored and ends up screwing everyone. Take my credit history in the US as an example. In my late teens, I got into an accident and couldn't work. I couldn't pay my bills.

Within a few months, I was back on my feet; but it took nearly 10 years to get my credit score to something somewhat reasonable. I had employers look at my credit history and go "nope" because the machine said I wasn't trust-worthy.

Should you trust the automation? Probably never, unless it knows how to deal with exceptions.

Notice how you ask about automation then mention instrumentation?

Eastern Airlines Flight 401 in 1972 and Air France Flight 447 in 2009 are examples along with the recent problems with the 737 autothrottle system.

Even modern driver training downplays the utility of ABS.

This post was talking about relying on AI in the presence of uncertainty.

Perhaps you aren't familiar with the term?

"Automation bias is the propensity for humans to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct"

The recently-famous Boeing 737 MAX crash of Lion Air Flight 610 happened due automation called MCAS. Apparently, the pilots couldn't override the automation in time to survive.

https://en.wikipedia.org/wiki/Boeing_737_MAX_groundings

> So the study[0] involved people making simulated drone strike decisions. These people were not qualified to make these decisions for real and also knew the associated outcomes were also not real. This sounds like a flawed study to me.

Unless I missed something, they also don't check for differences between saying the random advice is from an AI vs saying it's from some other more traditional expert source.

Most people don't give a shit about killing from behind the computer screen they have become desensitized to it ai has nothing to do with it. US has killed 1000s of children in drone strikes in Afghanistan and Pakistan sometimes knowingly sometimes not. Most of the remote pilots do not give shit about what they did.

How is this phenomenon of very weakly grounded research called? Science washing or something?

I don't know why exe34 has been flagged/downvoted to death, because pseudo-science is absolutely the right answer.

But this isn't it, the paper is fine. It is peer-reviewed and published in a reputable journal. The reasoning is clearly described, there are experiments, statistics and a reasonable conclusion based on these results which is "The overall findings indicate a strong propensity to overtrust unreliable AI in life-or-death decisions made under uncertainty."

And as always, nuance gets lost when you get to mainstream news, though this article is not that bad. First thing because it links to the paper, some supposedly reputable news websites don't do that. I just think the "alarming" part is too much. It is a bias that needs to be addressed. The point here is not that AIs kill, it is that we need to find a way to make the human in AI-assisted decision making less trusty in order to get more accurate results. It is not enough to simply make the AI better.

Reply on news.ycombinator.com