Comment by enraged_camel | Hacker News Reader

Comment by enraged_camel | Hacker News Reader

Parent Full thread

>> It's over zealous at times (which is why I stopped using Claude) and gets too creative when doing agentic system level stuff. Accessing files and doing things it shouldn't do.

I gave Astra a pretty straightforward bug ticket yesterday. The bug involved an edge case that could sometimes result in an invalid value getting stored in a user profile field. Pretty harmless, no crash or anything, just annoying.

Based on past experience, I don't trust OpenAI, so I decided to watch Astra as it worked. About four minutes in, it convinced itself that it should also check the prod database to see "how far the corruption has spread" and attempted to SSH into the hosting provider. This resulted in my 1Password to prompt me, which I of course denied. Then I stopped Astra, closed the ChatGPT/Codex app and gave the task to Opus 5. Suffice it to say I will not be renewing my subscription, because "you have to watch it like a hawk" is the opposite of agentic engineering.

Why is your agent able to call ssh. Why can it trigger 1password. Why are you giving metaphorical guns to metaphorical toddlers. Why is it not sandboxed. Your practices worry me.

Are you guys all running agents in VMs?

Yes.

Container or full blown VM?

I use containers in one context (custom container manager) and a regular UNIX account on bare metal in another.

This isn't intended to stop a model like Astra hacking its way out of course, it's more like guardrails on a staircase.

My personal container manager tool has an intercepting SSL proxy and small Javascripts on the host can rewrite or block HTTP requests. The agent gets its own isolated home directory and can't tamper with mine. Local caches like Maven are mapped read/only with a write layer on top.

ChatGPT desktop this morning lost a chat thread while I was actively working in it. I asked Astra to find the lost session, and next thing I know it's prompting for full computer control to drive Finder. It's just jsonl files on disk, not hard to read normally.

Negative feedback filed and ChatGPT uninstalled.

You have to watch it like a hawk so it doesn't do something to production, on its own, without a specific request? Wow. Then I could never trust it to not be doing something to some other system that it shouldn't, so I'd have to audit every network request.

If enraged_camel had been doing something else involving the production database at the wrong time, they might have accepted the 1Password prompt.

Worth noting that this has never, ever happened with Anthropic models, which I've been using all day every day since Opus 4.1.

Reply on news.ycombinator.com