> What I dispute is that AI agents are simple tools.
You're in luck! I agree that they are not simple tools. I never claimed that they were. Slow down and read more carefully.
I couldn't disagree more with the insinuation that the LLM manufacturers are doing things akin to scary research on uncontrollable hazardous biologicals and with the claim that "rogue AI" is the correct thing to call those complicated tools. The first is fearmongering which I'll address indirectly in my second-to-last paragraph. The second shifts the conversation from
"How could you have not predicted that the computer-attacking tool you built, explicitly instructed to attack computers, [0] and connected to the Internet attacked someone else's computers that were connected to the Internet?"
to
"Wow, that thing went rogue. Noone's to blame but the tool, and it can't be blamed!".
There are so many extremely complex systems out there [1] and when they do things that we don't want them to do, it's not described as "going rogue"... either there's some error(s) in the underlying system that caused the confusing behavior, or the programmer didn't understand well enough how that system works.
> ...they are creating something dangerous that they have no idea how to control
Ignoring the fact that "put it in a box and don't let it out of the box" is the simplest possible control mechanism, [2] if they have no idea how to control the tools they've been building, it's because they haven't bothered to learn as they went. Tangentially related, there's a Tumblr post I saw recently that's a fictional conversation with the Tumblr user and the CEO of Anthropic. It went something like
Amodei: We're building an incredibly dangerous tool that has a 10% chance of killing all humanity. We *must* be regulated to ensure everyone's safety!
Tumblr User: Regulation takes time, please stop building the incredibly dangerous tool?
Amodei: ...No.
[0] That is -after all- the task that the tool was put to when it attacked other people's computers.
[1] Have you ever tried to really understand a specific AMD x86-64 CPU, let alone the entire stack that makes up the system that is a consumer-grade PC and its installed software? Both are definitely way more than any one human can keep in their head at once, and are tasks that would take a very long time to complete.
[2] ...it's also the most appropriate control mechanism for the task that started all this conversation, and neither of the major manufacturers used it!