Hacker News Reader

Comment by michaelmior | Hacker News Reader

Even if the decision is made by a human, it really matters how that decision is presented. My understanding of the current state of warfare (which could be way off) is that were possible, targets are still largely selected manually based on verified intelligence.

What if we allow AI to propose targets? Even if testing shows that the AI is highly effective at picking "good" targets, that changes the situation entirely.

Suppose humans make the final decision of which target to strike. How much time is given to make that decision? What information is presented to them? If the answer is that humans make a decision in a short amount of time largely relying on an AI-generated confidence score, are humans really making the decision?

View on HN

Exactly. It’s too easy for these decisions to become a formality. ProPublica’s reporting on Cigna[1] is a great example:

> Before health insurers reject claims for medical reasons, company doctors must review them, according to insurance laws

> A Cigna algorithm flags mismatches between diagnoses and what the company considers acceptable tests and procedures for those ailments

Sounds fine, except

> “We literally click and submit,” one former Cigna doctor said. “It takes all of 10 seconds to do 50 at a time.”

[1]: https://www.propublica.org/article/cigna-pxdx-medical-health...

Have you heard about https://www.theguardian.com/technology/2022/nov/20/part-of-t... ?

And also, how do you feel about already-existing heatseekers? A little different, but the killchain still leans on an automated system for targeting and engagement.

Thanks for the link!

As far as existing heat-seekers, I would assume that they are generally used in cases where the only target which will likely be locked on to is the desired target. However, this is coming from a place of total ignorance of what the process of target acquisition might look like and what conditions they are used in.

Heatseekers can be very effective, but as with any automated system, it isn't infallible.

Friendly-fire even happens by human beings using small arms, just like strikes on civilian schools and hospitals happen because the intelligence told a human being that it looked like it could be a "terrorist training ground".

I'd be interested to see what the failure rate of an AI looked like in what actions it would have taken based on available data versus what actions human took over a several year sample. I have a feeling that the AI will either look terrible, or the human beings will look terrible, or they'll look pretty equal with strange fringe cases where the AI is better than the human, and vice versa. Judgement and authorization are interesting.

Something else you might be interested in (which you may already know about) is PAL systems for nuclear weapons: https://en.wikipedia.org/wiki/Permissive_action_link

You'll likely be interested in the "two-man rule": https://en.wikipedia.org/wiki/Two-man_rule

I guess my point was given my limited knowledge, it doesn't seem as though heat-seekers are necessarily any less fallible than humans. I'm not suggesting that "no worse than a human" should be the goal, but I'd say that's the bare minimum.

Precisely. All of these things can fail, with or without human involvement, and humans can fail just as easily. Whilst these are all absolutely horrible contraptions that shouldn't be necessary in relative modernity, it's important to look at stats, but also to "sanity-check" an authorization with concepts like a two-man rule.

Whilst AI may indeed be superior to human beings in x areas now or in the future, human ethics, intuition et al are also very important and likely to never be replaced. In the same breath, fuckups will always happen by the very nature of every system and every human being imperfect.

identify a target

decide to attack the target

implement the attack

determine the result of the attack

If humans are required for the decision, then humans have an inherent hook to prevent more attacks. Otherwise you run into scenarios like https://www.youtube.com/watch?v=pyMNIFZTQkg where humans are incapacitated and AI just continue on like nothing has happened. It makes the failure modes more sane. It's not a catch-all but a good agreeable step to take.

Reply on news.ycombinator.com