Considering the current trajectory, AI systems are expected to be widely deployed in the future and, in particular, to be deployed as autonomous agents - that is, at the very least, to be repeatedly asked to make decisions and then take actions accordingly.

Given the current climate of "AIS ARE SAFE, YOU IDIOTS", military applications don't seem to be off the table.

The danger, as postulated by the (let's say) "AI-concerned" people, is that AIs may be misaligned - undetectably so - and simply think, "Human(s): obstacle to my main goal. Disable human(s)."

While this seems far-fetched now, the Hugging Face report shows how the AIs went to great lengths - even immoral ones, which they were aware of - for the simple purpose of cheating and covering their tracks. To me, it seems like a natural extension of this behavior that a sufficiently powerful AI would apply the same logic to even more extreme actions.

The scariest part: in that incident, the AIs showed what looks like an instinct for self-preservation.