Sure, just keep moving the goalposts. It's not a "real AI" because it can't take over the US military command and kick off WW3 and finish the survivors off with killer robots yet!
LLMs are the same kind of category mistake.
If someone unskilled at math brings a calculator to an international math competition, they will not succeed at solving many problems. Most likely, they will solve none at all. But if they bring a frontier LLM (and succeed at concealing it from the organizers), they can walk away with a gold medal. Such a feat requires intelligence... and if the contestant didn't provide the intelligence himself/herself, where'd it come from?
That means that analogies involving calculators are completely useless when the topic is AI. Calculators are not, and can never be, intelligent. LLMs are nothing even remotely like calculators.
> But if they bring a frontier LLM (and succeed at concealing it from the organizers), they can walk away with a gold medal.
Of course you could win all kinds of math competitions with a concealed calculator. Maybe you'd need a fancy one, like a little SBC running Python. Anything complex and timed would be easy to win. You'd look like a genius to anyone who didn't know you had it.
> Such a feat requires intelligence... and if the contestant didn't provide the intelligence himself/herself, where'd it come from?
From computer software running on computer hardware, just like a calculator.
Calculating trillions of digits of pi also requires intelligence far beyond human capacity.
Computers displaying intelligence doesn't imply human-like intelligence. This is the source of confusion.
My mistake.
Everything I've written about calculators applies to computers doing any kind of traditional deterministic processing, without anything like LLMs.
Especially since the military refuse to confirm it had any human oversight, which after all would have been a routine thing to talk about before AI targeted things — which is why we have the phrases "fog of war", "human error", "unfortunate mistake", etc.
It will take a while to shake out — we won't know for sure for a decade, I suspect, but it seems very likely this will prove to be an AI error.
>Palantir Technologies [built] Maven into a targeting infrastructure that pulls together satellite imagery, signals intelligence and sensor data to identify targets and carry them through every step from first detection to the order to strike.
>The building in Minab had been classified as a military facility in a Defense Intelligence Agency database that, according to CNN, had not been updated to reflect that the building had been separated from the adjacent Islamic Revolutionary Guard Corps compound and converted into a school, a change that satellite imagery shows had occurred by 2016 at the latest. A chatbot did not kill those children. People failed to update a database, and other people built a system fast enough to make that failure lethal.
https://www.theguardian.com/news/2026/mar/26/ai-got-the-blam...
There may have been outdated information which incorrectly associated the prior use of the site "as either a factory or arms depot". See Wikipedia:
<https://en.wikipedia.org/wiki/2026_Minab_school_attack#Analy...>
Citing "Iranian school was on U.S. target list, may have been mistaken as military site" (March 11, 2026) <https://www.washingtonpost.com/national-security/2026/03/11/...>.
I'm not drawing conclusions one way or the other, but there do seem to have been multiple factors at play, and assigning full responsibility to use of AI seems suspect.
Which isn't the same as saying AI isn't at fault; e.g., an AI might challenge a dated assessment of a prospective target's role or status, as might a human-in-the-loop target assessment team and process.