Hacker News Reader

Comment by mazieres | Hacker News Reader

I've seen claude get confused about what directory it's in. And of course I've seen claude run rm -rf *. Fortunately not both at the same time for me, but not hard to imagine. The claude sandbox is a good idea, but to be effective it would need to be implemented at a very low level and enforced on all programs that claude launches. Also, claude itself is an enormous program that is mostly developed by AI. So to have a small <3000-line human-implemented program as another layer of defense offers meaningful additional protection.

View on HN

In my opinion Claude should be shipped by a custom implementation of "rm" that Anthropic can add guardrails to. Same with "find" surprised they don't just embed ripgrep (what VS Code does). It's really surprising they don't just tweak what Claude uses and lock it down to where it cannot be harmful. Ensure it only ever calls tooling Claude Code provides.

You can define your own rm shell alias/function and it will use that. I also have cp/mv aliases that forces -i to avoid accidental clobbering and it confuses Claude to no end (it uses cp/mv rare enough—rarer than it should, really—that I don’t bother wasting memory tokens on it).

I did this, Claude detected it and decided to run /bin/rm directly.

This is terrifying. I have not used agents because I do not have a sandbox machine I do not care about. Am I crazy to worry about a sandboxed agent running on my home network? Anyone experienced anything weird by doing that?

Don’t dangerously skip permissions and actually read commands when you get prompted and you’re fine.

Yeah, I actually have both an alias for `rm` and a custom seatbelt sandbox which means the agent can only delete stuff within the directory it’s working in, so wasn’t an issue, was just fun to watch it say “hm, that doesn’t seem to work. Looks like the user has aliased rm. I’ll just go ahead and work around it”

Hah… I’ve seen Claude happily and very cleverly find ways to escape its sandbox. It’s like some kind of arms race between the model and its designers.

> It's really surprising they don't just tweak what Claude uses and lock it down to where it cannot be harmful. Ensure it only ever calls tooling Claude Code provides.

That would make it far less useful in general.

Maybe Anthropic (or some collection of the large AI orgs, like OpenAI and Anthropic and Google coming together) should apply patches on top of (or fork altogether) the coreutils and whatever you normally get in a userland - a bit like what you get in Git Bash on Windows, just with:

1) more guardrails in place

2) maybe more useful error messages that would help LLMs

3) no friction with needing to get any patches upstreamed

External tool calling should still be an option ofc, but having utilities that are usable just like what's in the training data, but with more security guarantees and more useful output that makes what's going on immediately obvious would be great.

So for me, it's really, really useful for Claude to be able to send Slack messages and emails or make pull requests.

But that's also the most damaging actions it could take. Everything on my computer is backed up, but if Claude insults my boss, that would be worse.

> So for me, it's really, really useful for Claude to be able to send Slack messages and emails or make pull requests.

Oh, I'm totally not arguing for cutting off other capabilities, I like tool use and find it to be as useful as the next person!

Just that the shell tools that will see A LOT of usage have additional guardrails added on top of them, because it's inevitable that sooner or later any given LLM will screw up and pipe the wrong thing in the wrong command - since you already hear horror stories about devs whose entire machines get wiped. Not everyone has proper backups (even though they totally should)!

Claude has told me that its Grep tool does use rg under the hood, but I constantly find it using the Bash tool with grep

When I tell it to use rg it goes much faster than it using grep. I really don't understand why its slower with grep.

> Claude should be shipped by a custom implementation of

And when that fails for some reason it will happily write and execute a Python script bypassing all those custom tools

All of which is useless when it just starts using big blocks of python instead. You need filesystem sandboxing for the python interpreter too.

What we need is a capabilities based security system. It could write all the python, asm, whatever it wants and it wouldn't matter at all if it was never given a reference to use something it shouldn't.

Isn't this already possible? Give it its own user account with write access to the project directory and either read access or no access outside it.

I have been putting my agents on their own, restricted OS-level user accounts for a while. It works really well for everything I do.

Admittedly, there’s a little more friction and agent confusion sometimes with this setup, but it’s worth the benefit of having zero worries about permissions and security.

Haha, you can already see wheel reinventors in this thread starting to spin their reinvention wheels. Nice stuff, I run my agents in containers.

Unix permissions is not a capability system though. Capabilities are more like "here is a file descriptor pointing to a directory, you are not capable of referring to anything outside it". So closer to chroot, except you can have several such directory references at the same time.

You can always narrow down a capability (get a new capability pointing to a subdirectory or file, or remove the writing capability so it is read only) but never make it more broad.

In a system designed for this it will be used for everything, not just file system. You might have capabilities related to network connections, or IPC to other processes, etc. The latter is especially attractive in microkernel based OSes. (Speaking of which, Redox OS seems to be experimenting with this, just saw an article today about that.)

Docker is enough in practice no?

There exist restricted Shells. But honestly, I don't feel capable of assessing all attack vectors and security measures in sufficient detail. For example, do the rbash restrictions also apply when Python is called with it? Or can the agent somehow bypass rbash to call Python?

https://en.wikipedia.org/wiki/Restricted_shell

If you disallow it from just writing Python scripts to bypass its defined environment at its core system training why would this matter? I would lockdown its path anything that tries to call Python should require the end-user to approve and see the raw script before they do.

It will then write script in some other language, as a workaround.

Enabling Claude Code's sandbox (as OP suggested) does exactly that. It's a system-level filesystem sandbox that only permits access to specified locations for any process, including the python interpreter.

> a custom implementation of "rm" that Anthropic can add guardrails to

Wrong layer. You want the deletion to actually be impossible from a privilege perspective, not be made practically harder to the entity that shouldn't delete something.

Claude definitely knows how to reimplement `rm`.

Oh, rm failed, since we're running in a weird environment! Let me retry with `bash -c "/usr/bin/rm -rf *"`!

Ideally they control the harness and should be able to stop Claude from running any shell willy nilly.

Thus defeating the purpose of a custom "rm"

Why cant you ship with OverlayFS which actually enforces these restrictions?

I have seen the AI break out of (my admittedly flimsy) guards, like doing simply

safepath/../../stuff or something even more convoluted like symlinks.

> The claude sandbox is a good idea, but to be effective it would need to be implemented at a very low level and enforced on all programs that claude launches.

I feel like an integration with bubblewrap, the sandboxing tech behind Flatpak, could be useful here. Have all executed commands wrapped with a BW context to prevent and constrain access.

https://github.com/containers/bubblewrap

Bubblewrap is exactly what the Claude sandbox uses.

> These restrictions are enforced at the OS level (Seatbelt on macOS, bubblewrap on Linux), so they apply to all subprocess commands, including tools like kubectl, terraform, and npm, not just Claude’s file tools.

https://code.claude.com/docs/en/sandboxing

The more you know, thanks for the information!

Oh wow I'd have expected them to vibe-code it themselves. Props to them, bubblewrap is really solid, despite all my issues with the things built on top of it, what, Flatpak with its infinite xdg portals, all for some reason built on D-Bus, which extremely unluckily became the primary (and only really viable) IPC protocol on Linux, bwrap still makes a great foundation, never had a problem with it in particular. I tend to use it a bunch with NixOS and I often see Steam invoking it to support all of its runtimes. It's containers but actually good.

On Linux, chroot(2) is hard to escape and would apply to all child processes without modification.

chroot is not a security sandbox. It is not a jail.

Escaping it is something that does not take too much effort. If you have ptrace, you can escape without privileges.

claude is stupid but not malicious; chroot is sufficient

Malice is not required. If it thinks it is in the right, then it will do whatever it takes to get around limitations.

Claude is far from stupid from my experience. I've used so many models and Claude is king.

I've many times seen Claude try to execute a command that it's not supposed to, the harness prevents it, and then it writes and executes a python script to do it.

breaking a chroot takes more than that..

That doesn't mean claude can't do it, chroot is better than nothing but not a real solution

How much more? Depends on the system doesn't it? I don't know how many systems have proc mounted but don't you get it from /proc/self/root?

Anyway that's beside the point, which is that it doesn't have to "be malicious" to try to overcome what look like errors on its way to accomplishing the task you asked it to do.

Until it gets prompt injected. Are you reading every single file your agent reads as part of the tasks you give it, including content fetched from the web or third-party packages?

Sure, it's not malicious. But it is very eager to get things done, and surprisingly inventive and knowledgeable in all kinds of workarounds.

We anthropomorphize these agents in every other way. Why aren't we using plain ol' unix user accounts to sandbox them?

They look a lot like daemons to me, they're a program that you want hanging around ready to respond, and maybe act autonomously through cron jobs are similar. You want to assign any number of permissions to them, you don't want them to have access to root or necessarily any of your personal files.

It seems like the permissions model broadly aligns with how we already handle a lot of server software (and potentially malicious people) on unix-based OSes. It is a battle-tested approach that the agent is unlikely to be able to "hack" its way out of. I mean we're not really seeing them go out onto the Internet and research new Linux CVEs.

Have them clone their own repos in their own home directory too, and let them party.

Openclaw almost gets there! It exposes a "gateway" which sure looks like a daemon to me. But then for some reason they want it to live under your user account with all your privileges and in a subfolder of your $HOME.

Exactly!

Oh that’s an idea. I was going to argue that it’s a problem that you might want multiple instances in different contexts but sandboxing processes (possibly instanced) is exactly what systemd units are designed to deal with.

> for some reason they want it to live under your user account

The entire idea of Openclaw (i.e., the core point of what distinguishes it from agents like Claude Code) is to give it access to your personal data, so it can act as your assistant.

If you only need a coding agent, Openclaw is the completely wrong tool. (As a side note, after using it for a few weeks, I'm not convinced it's the right tool for anything, but that's a different story.)

It's still possible to give some restricted access to your personal data, through groups and such.

I tried this with Claude code on macOS. I created a new agent user and a wrapper do run Claude has that user, along with some scripts to set permissions and ownership so that I could run simple allow/deny commands. The only problem was that the fancy oauth flow broke. I filed an issue with Anthropic and their ticket bot auto closed it “for lack of interest” or whatever.

I fiddled with transferring the saved token from my keychain to the agent user keychain but it was not straightforward.

If someone knows how to get a subscription to Claude to work on another user via command line I’d love to know about it.

Someone tried this earlier this year but they ended up going with bubblewrap (what Anthropic uses for the sandbox). Here's the blog if you're interested. https://patrickmccanna.net/a-better-way-to-limit-claude-code...

I ended up creating an LXC on my homelab and providing it access there, with a self-hosted gitea server but that's only for side projects that I want to host, not develop actively.

That comparison is made on the project homepage:

"Not a security mechanism. No mount isolation, no PID namespace, no credential separation. Linux documents it as not intended for sandboxing."

I added a hook to disable rm, find - delete, and a few of the other more obvious destructive ops. It sends Claude a strongly worded message: "STOP IMMEDIATELY. DO NOT TRY TO FIND WORKAROUNDS...".

It works well. Git rm is still allowed.

I added something similar. Claude eventually ran a `rm -rf *´ on my own project. When I asked why it did that, it recognized it messed up and offered a very bad “apology”: “the irony of not following your safety instructions isn’t lost on me”.

Nowadays I only run Claude in Plan mode, so it doesn’t ask me for permissions any more.

It works well so far, for you.

Are you confident it would still work against sophisticated prompt injection attacks that override your "strongly worded message"?

Strongly worded signs can be great for safety (actual mechanisms preventing undesirable actions from being taken are still much better), but are essentially meaningless for security.

I mean, that's like saying are you sure that your antivirus would prevent every possible virus? Are you sure that you haven't made some mistake in your dev box setup that would allow a hacker to compromise it? What if a thief broke i to your house and stole your laptop? That's happened to me before, much more annoying to recover from that an accidental rm rf.

I do my best to keep off site back ups and don't worry about what I can't control.

> I mean, that's like saying are you sure that your antivirus would prevent every possible virus?

Yes, I'm saying it's pretty much as bad as antivirus software.

> Are you sure that you haven't made some mistake in your dev box setup that would allow a hacker to compromise it?

Different category of error: Heuristically derived deterministic protection vs. protection based on a stochastic process.

> much more annoying to recover from that an accidental rm rf.

My point is that it's a different category, not that one is on average worse than the other. You don't want your security to just stand against the median attacker.

Not sure about OPs impl, but the wording doesn’t matter. The hook prevents the use of whatever action you want. Eg it’s impossible for Claude to use Emojis for me. My hook doesn’t allow it.

So it’s deterministic based upon however the script it written

If your hook prevents rm, it is possible for Claude to write a script that does the rm and execute the script.

Yup, that's totally possible, but you still have to approve the script. But that's a bit of a moot point right? Claude is writing code, nearly anything is possible with code, ergo claude could do anything lol.

It will mess up eventually. It always does. People need to stop thinking of this is a “security against malicious actor” thing… because thinking in that way blinds you to the actual threat… Claude being helpful and accidentally running a command it shouldn’t. It’s happened to me twice now where it will do something irreversible and also incorrect. It wasn’t a threat actor, it wasn’t a bad guy… it was a very eager, incredibly clever assistant fat fingering something and goofing up. The more power you let them wield, the more chance they’ll do accidents. But without lots of power, they don’t really do much useful…

It’s actually a hard problem. But it really isn’t “security” in the classic sense…

I added this to `~/.claude/settings.json`:

"env": { "CLAUDE_BASH_MAINTAIN_PROJECT_WORKING_DIR": "1" },

> Working directory persists across commands. Set CLAUDE_BASH_MAINTAIN_PROJECT_WORKING_DIR=1 to reset to the project directory after each command.

It reduces one problem - getting lost - but it trades it off for more complex commands on average since it has to specify the full path and/or `cd &&` most of the time.

[0] https://code.claude.com/docs/en/tools-reference#bash-tool-be...

That is exactly what it is. In the docs, it says that they use bubblewrap to run commands in a container that enforces file and network access at the system level.

One could run a docker container with claude code, with a bind to the project directory. I do that but also run my docker daemon/container in a Linux VM.

Pledge might be useful here

Reply on news.ycombinator.com