AI hallucination of Chinese nuclear components almost led to US Military attack | Hacker News Reader

AI hallucination of Chinese nuclear components almost led to US Military attack

arstechnica.com

14 pointstheanonymousone16 comments

Open articleView on HN

Whenever I see this information, I’m reminded of the first half of movies where robots fight humans

And here we are… AI is Going to end all humans.

humans in the loop caught this, albeit way later than they ought to have

1. should never have reached the point of planes in the air

2. my smell test thinks the CCP is not so ignorant/arrogant to ship nuclear components to Iran right now

I expect AI will continute to be used inappropriately until the headlines read sonething between "thousands dead" and "millions dead".

I suppose we are in (an ai) doomerism media (hype cycle)

It's not impossible, of course.

But AI is also being hyped beyond its competences* by e.g. Hegseth, to the degree that Anthropic saying "we don't want to be used like this" resulted in an illegal attempt to punish Anthropic and ban any of their products in a far more general fashion. And Anthropic is only the least bad of the AI companies: the general consensus in safety circles is Anthropic's grade is "barely acceptable" in objective terms, everyone else is much worse.

Some people treat automated systems (both machine learning and simple computer systems) as infallible oracles; they're not infallible oracles, and people do die as a result. This already happens. So far, humans in the loop have caught and prevented massive disasters (Stanislav Petrov day is coming up soon).

Even though the personal disasters keep killing people, even though the big disasters have only been prevented by humans, even though the people making the automation (e.g. Anthropic) say "it's not good enough for that", there's people who keep pushing for full automation.

* it has a lot of competences, but it also has a lot of gaps. I can see inference suppliers being legitimately valued at multiple trillions of USD even today, but there's a big gap between that and the hundred-ish trillion dollar global economy and the combined market cap this would entail for an AI that could genuinely take over all jobs even if somehow "make more AI" wasn't among those jobs.

There's a big difference between "AI is creating misinformation and intelligence failures that could compound, especially given the fact that in a pre-AI world we came to the brink of nuclear warfare" vs the AI Doomerism of "Super-intelligent AI will kill us all in some mysterious way" of the AI "safety" TESCREAL cult, often used to hype up their own product capabilities.

Headlines read that under COVID, and instead of doing anything importantly useful about it, Americans went and created mental gymnastics _not to do anything_.

I find it hard to believe anything is going to yield to the current regime of blind backflips to oligarch's needs.

Some did, some didn't, government acted. Trump acted, even if he now dissociates from vaccinations in general and by extension takes no credit for Operation Warp Speed.

and it's clear that was momentum from a long gone state. the last 2 years have demonstrated zero of that, so, yeah, my statement remains.

I understand (and to extent I agree), but he does seem to respond to some headlines at least.

In other cases, use of Palantir's AI-powered Project Maven lead to us using our long range $2.5M Tomahawk missiles to blow up an Iranian school along with over 100 schoolchildren, and humans in the loop did nothing to stop the accumulation of intelligence failures from the overreliance on AI. Palantir-funded politician JD Vance said he's doing God's work and "if that leads to the end times, OK".

https://www.bloomberg.com/graphics/2026-iran-school-attack/

> the end times

This is a nod to the evangelicals, JD Vance says things people like to hear, he rarely believes them (imo) as he seems to lack any convictions, willing to "adjust" his view to whatever suits the times (and his political career). The guy called Trump "America's Hitler" and is now VP...

On the murder of Iranian school girls, that seems more a process failure than straight up Ai. The database had outdated information, which should have been a big flag to go get updated information (I assume our observation capabilities are much better than 10 years ago)

That being said, the Ai being around made those humans sloppy, they also have a boner for bombs

No. Humans are going to start the war. First, the people who exaggerated AI abilities and donwplayed disadvantages. Who bribed Trump and stood behind him.

And then people who work for Trump and made series of bad decisions, because they wanted more of cheap lethality.

AI is not accountability sink. It is a tool. The ones to be blamed are Altman, Hegseth, Amodei, Musk, Thiel, Trump and those under them

It's the same pattern we see in predictive policing where "AI" is being used to manufacture probable cause, but instead of just using it to harass minorities as cops do, the people doing the policing are people actively trying to start Armageddon and the end times because they believe they are doing God's work. So they spend our long-range missiles stockpile to blow up schools because AI told them to.

https://www.theguardian.com/us-news/2026/sep/02/jd-vance-god...

https://www.bloomberg.com/graphics/2026-iran-school-attack/

It is really terrible.

Reply on news.ycombinator.com