I mean, ideally enforce W ^ X, and enforce stricter permissions on executables where the agent has flipped the W to X. My broader point is we should be using operating system primitives instead of regex and wishful thinking.
You can't give shell access and then revoke special parts of it. This is not controversial.
All your criticisms apply to all LLM harnesses - and everyone who genuinely cares about these things is using security on top of the harness, because OBVIOUSLY.
Your complaints are misguided.
This doesn't require cooperation from the shell. If writable paths are non-executable, and executable paths are non-writable, then executing `cp $(which git) my-new-favourite-binary` doesn't grant you permission to run it.
The part which does require cooperation (from outside of the shell, i.e. from the harness) is the ability to upgrade writable paths to executable, at the cost of writability and whatever other restrictions you place on the binaries generated by your agent.
This kind of W^X invariant is widely deployed in every desktop operating system, and the type of W-X upgrade hook I described is used by every JIT engine.
I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables?
> You are confusing CPU-level security with filesystem-level security.
> I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables?
W^X applies to file systems as well as page tables, and this is called out in paragraph 4 of the Wikipedia article you didn't read. Respectfully, I'm leaving this conversation at this point because it doesn't seem productive.
My question stands - if you accept that LLM harnesses are designed to allow the building and testing of executables - how do you imagine you can prevent the building and running of executables?
Respectfully.
Separation of duties and ephemerality are well-known security mechanisms that many harnesses jump right over in the pursuit of easier UX.
Mingling permissions + trusting vibe-coded security boundaries in the harness itself (developed by folks whose appetite for risk would make even an 00s front-end developer blush) is rolling a handful of dice all at once.
But fundamentally, this is a disagreement between two risk appetites who will never agree: {it works most of the time} vs {it's guaranteed to never fail}
I'm not sure why you disbelieve agents can do "guaranteed to never fail". Proving correctness is one of their strengths. Proofs in general, in fact, much much better than the typical human coder.
Because I must warn you - it's extremely easy to ask a coding agent to prove code correctness before committing his work.
But that doesn't mean they don't have >0% failure modes, often in very non-human, unexpected, even dangerous ways.
Whether that's "sloppy"?
Eh, it's just the nature of the tool, like a perfect screwdriver that randomly strips 1% of screws.
> it's extremely easy to ask a coding agent to prove code correctness
That statement is categorically wrong.
If you're talking about a formal proof for non-trivial, real-world code, no.
If you're talking about a comprehensive set of tests... well, I'd challenge you to elucidate how you see creating a comprehensive set of tests as trivial.
And if the answer is more LLM... then I think we're at a fundamental disagreement about what satisfies "guarantee" or "provable".
I invite you to examine the test suite on the project I just published. Was it trivial? No, I'd say the whole project did take me a couple days.
https://github.com/qarl/arcade-js
You have a nice day.