Hacker News Reader

Comment by jkubicek | Hacker News Reader

I’m going to defend Claude. If you give a robot the ability to delete production it’s going to delete production. This is 100% the user’s fault.

This problem has not yet been solved and will never be solved.

View on HN

> If you give a robot the ability to delete production it’s going to delete production

If you give an intern the ability to delete production, it's going to delete production. But to be honest you can as well replace "intern" or "robot" by human in general. Deletion in production should have safety layers that anyone cannot accidentally do it, specially without the ability of rolling back.

I think the difference, though maybe I'm incorrect, is that when we have interns on our codebase they get restricted permissions. Can't push to prod, need pull requests with approvals and reviews, etc. Certainly can't delete backups. Whoever setup the robot's permissions did it wrong. Which is interesting because early on there were people complaining that these AIs refused to push to main, but now this stuff keeps happening.

> Whoever setup the robot's permissions did it wrong.

It doesn't have permissions of it's own. The way he's using it, it has his permissions.

Also, in order to be able to do deployments like that you need pretty wide permissions. Deleting a database is one of them, if you're renaming things for example. That stuff should typically not happen in prd though

That was my first guess but I wasn't sure. I've seen AIs as authors on things. So yeah that's even worse. You don't give the intern your credentials.

I had a senior tech lead delete production doing a late night quick fix. Especially in panic mode where sometimes processes are ignored, things are going to go wrong. Don't need interns for that, nor llms.

Which is why I actually said replace intern or robot by "human" in general in my comment.

That’s a broken analogy. An intern and a llm have completely different failure modes. An intern has some understanding of their limits and the llm just doesn’t. The thing, that looks remarkably human, will make mistakes in ways no human would. That’s where the danger lies: we see the human-like thing be better at things difficult for humans and assume them to be better across the board. That is not the case.

"Anything that can go wrong, will go wrong. Including deleting production."

I'm also waiting for the day we see a "Claude sold my production database on the darkweb" post haha.

You can't fix stupid.

Reply on news.ycombinator.com