And they want this crap to be able to automatically execute workflows without human intervention.
> What AIP does not do is walk through how it plans to deal with the various pernicious problems of LLMs and what the consequences might be in a military context. AIP does not appear to offer solutions to those problems beyond “frameworks” and “guardrails” it promises will make the use of military AI “ethical” and “legal.”
Utterly, utterly ridiculous. But it's OK because they said it's "ethical."
The computing industry has gone completely off the rails with LLMs; we spent a month or two debating hypothetical sci-fi questions until everyone got bored of them, and now nobody is thinking about ethics or security at all for any of their products. It would be embarrassing if it wasn't so horrifying.
What we've learned from this is that "how does the AI escape the box?" is a fun thought experiment, but "is this technology inherently vulnerable to extremely basic malicious input attacks?" is a boring question that's not fun to debate and might have implications for the stuff we're actually building, so we're just not going to have that conversation. Or if we do have that conversation, we're definitely not going to delay anything we're building until we have an answer to that question. We're just going to say it's an ongoing issue that will be fixed sometime in the future, probably.
It's not reasonable to ask us to solve that problem before we let an LLM control a military drone. /s
But every single journalist or writer who covers this stuff should be pointing out that so far, the industry has basically zero idea of how to actually secure an LLM in production. And that's not even getting into "what happens when your Chat AI hallucinates an answer to a question that's wrong and then orders a drone strike off of that hallucination?" Figuring out how to build an LLM agent that can't be hacked is the baseline, and the industry hasn't even passed that point yet.