I mentally replace “AI agent” with “intern” when I read this stuff and it helps clarify the root cause. People are connecting directly into prod and making changes live? It’s not (solely) the fault of the actor, but of the whole process that makes it possible for the event to happen. It may be the case that there’s a break-glass situation where a specific person needs prod access to fix an urgent thing. In that case, there needs to be an approved plan like “I’m connecting to this DB, making this query to find the affected row, then running this one to fix it”.

If it makes you shudder to imagine allowing an intern to do a thing, you should shudder harder to imagine letting an AI — an intern who can type really fast — do it.

I work in AI. I love using AI. I don’t want to go back to not using AI. But darned if I’m letting anyone, human or AI, just waltz into a prod environment and make random changes.

Does the computer running the agent have production DB credentials on it anywhere? If it does, the AI has access to the production DB.

This is part of why I'm bearish on the new hotness of "don't write tools, just write a Markdown skill and let the LLM write its own bash commands". It does work, for the most part, at the cost of it being entirely capable of changing its environment and executing arbitrary commands. Approvals exist, sure, but I've never seen anyone manually approve a command past like the 3rd permission dialog.

It didn't have access to any db. In short: It went looking in the codebase for a credential to manage the staging environment, found a testing credential unrelated to anything it was doing, that the devs didn't know had permissions to administer anything, and then used that to delete the wrong db.