Comment by fipar | Hacker News Reader

Comment by fipar | Hacker News Reader

Parent Full thread

Thanks, I'll pass.

I do pre-approve read-only tools though, and some very specific tools that can change state (append data to a specific file).

That's what works best for me so far: start with open-ended exploration with me approving everything. Have it fork subagents to write scripts that automate the most common requests from the main agent, review and eventually approve those, and then let it go using those tools pre-approved but coming back to me for anything else. Combined with a stupidly basic "dashboard" (a terminal tab with a view of all top-level agents and their status, so that I can see which one is waiting for me and jump to it with a key or 2) this has made me a lot more productive.

It's fine if your approach works for you, I'm just too old school (and have been burned before) to just let a computer unattended overnight, with an internet connection, even if in a locked down container/vm. Those all have escape hatches and one of the agents will eventually figure it out. Remember that things that happen once every million times, happen all the time on computer-scale :)

I've had good luck giving it a local vm to keep its state / memories in, an ssh key + password-less su / sudo to some random box on the internet, --dangerously-skip-permissions, and then lots of encouraging commands like:

"No really, if the remote box breaks no one will care. It's the cheapest possible VPS on a bargain-bin obsolete hardware config. You have exclusive access + I can get a new one for free you brick it."

If I didn't want to use the cloud for this purpose, I could probably just stick a raspberry pi in the DMZ on my lan, and do the same.

The worst it's done so far is spawn parallel sub agents that accidentally stomp on each other (remote OOM, reboot, etc). Once it figured out why the remote machine kept "crashing", it implemented discretionary locking in the FS and social conventions for keeping the agent coordinator up to date, and asking for permission for certain tasks.

In theory it could use my endpoint to launch a civilization ending event (it technically has permission to, though I get the impression it's been trained to mostly obey the law...), but, well, that's not what I asked it to do.

Even if that did happen, it'd find an internet that's been hardened by decades of human hackers, and years of actually-malicious agent swarms run by organized crime. If it did accidentally knock something important over, frankly, the administrators of that thing should thank it for the bug report.

The current situation reminds me of the Back Orifice days. Even if everything gets broken into again, it won't be any worse than the damage done by the current US administration during an average day, except there will be an upside: A systemic hardening of whatever infrastructure gets knocked over.

The worst thing we could do is wait until the AI gets 100x smarter or whatever, then unleash it and watch everything burn on the same day. Instead, ramping up progressively smarter models provides progressive stress test + hardening cycles. In the same way remote penetration via fuzzing broke into lots of stuff, but was ultimately a win.

I think that's a good approach, whether it works on not probably depends on the nature of the work you want it to do.

In my case, given I work with databases, there's little that agents can do on their own except in the exploratory phase. I have run exploratory phases in self-contained VMs, including containerized DBs within the VM, but when it comes to go to prod, my endpoint could be used to launch a career ending event so I prefer to stay with the current approach. I'm still moving way faster than just 1 year ago, but in a safe way.

But I can totally see your approach working in other scenarios.

When I'm not too pessimistic, I agree with you on the result being a systemic hardening. I just hope the incidents that happen on the way to that hardening aren't too bad.

Reply on news.ycombinator.com