The article and your comment are confusing and it's still not entirely clear what happened. Did an operator lose their life?
Was this article written to be intentionally confusing?
Was this article written to be intentionally confusing?
The US military is playing with fire creating AI that will carry out the mission at all costs. This scenario is literally the plot of EVERY sci-fi warning about misused AI.
Anyone read/see 2001? You’d think we’d take the warnings provided by our own worst nightmares more seriously.
These aren't operational systems, these are research systems. No need for the hyperbole. The reason for these tests is to figure out exactly these kinds of failure modes (many of which have already been predicted) and how to handle them.
"It worked in the simulation," is the embodied AI equivalent of "It worked on my machine." It's not acceptable.
How confident are you that the issues can be addressed? How confident would you need to be to endorse deploying it out in the wild? How confident would you need to be that no "unforeseen" issues would arise?
I believe it's necessary to establish these levels of confidence even before discussing specific failure cases.
We would normally discuss failures in terms of base-rate comparison, but we don't have one for "how often autonomous weapons are unaligned?" Enumerating the failure modes isn't a substitute for the empirical distribution of failure mode probabilities.
I don't think they would be talking about points if someone lost their lives.
It's pure clickbait, should be removed.
There are easy ways to clarify the article but they clearly didn't try.
"Rogue AI Destroys Normally-manned Control Tower in USAF Simulated Test"
Hell, the wikipedia on "Military Simulation" [1] is very clear that there's a very broad gradient under the section "The simulation spectrum", saying:
The term military simulation can cover a wide spectrum of activities, ranging from full-scale field-exercises,[2] to abstract computerized models that can proceed with little or no human involvement—such as the RAND Strategy Assessment Center (RSAC).
Military exercises focus on the simulation of real, full-scale military operations in controlled hostile conditions in attempts to reproduce war time decisions and activities for training purposes or to analyze the outcome of possible war time decisions.
the neutral way to say this is something like, "drone simulated killing a human operator in test", which removes ambiguity of whether the killing happened independently from the simulation, or within it.
you may not have read it that way, but plenty of people did. it's safe to assume that this was done intentionally.
the title should be changed.