Show HN: Scan your AI agents for dangerous capabilities
github.com
github.com
You can make a file called "orders" and you can run your agent as a user with write access to that file, or as one that doesn't, and then you don't need scans or audits to tell you whether the agent can create orders or not, you can just take your operating system's word for it.
Is there anything all this bolt-on AI security stuff does that can't instead be handled by donning a sysadmin hat and managing your agents as separate users?
Like everything else, the packaging and ergonomics matter. Do we need podman or docker when we could just don our sysadmin hats and manage namespaces and cgroups directly instead?
Linux cannot stop an agent from calling:
POST /wire-transfer amount=5,000,000
Sure it's work to build such things, but building restraints into the app feels more reliable than playing whack-a-mole with scanner results.
Operating systems can probably do better to meet this need (e.g. capability based ones like fuchsia) but even as is their rules just feel so much more binding.
There is also @makerchecker/embedded, which has runtime permission primitives you wrap around the agent's actions, so the restraints live in the app exactly like you're describing
Less whack-a-mole, more wrap the risky calls and they're bound
OS controls answer "can this process make network requests?"
App-level policies answer "is this request, with these parameters, acceptable?"
I'll admit that OS interfaces could stand to improve in their ability to do so, there's a lot of stuff in plan9 and fuchsia that I'd love to see in widespread use, but despite that I still think the quickest route to sanity is improving existing tools that constrain arbitrary processes, not replacing them with tools that specifically constrain agents.
Further, servers still have hosting value, but any business running agents is almost certainly going to want a sandbox that limits what code runs for agentic work, so targeting _sandbox_ environments is probably the better bet long-term. And, yes, you could implement your proposal in any chroot jail or gvisor, but nobody wants to get their hands dirty finnicking with that - programmatic access control beats file-based access control for the simple reason it's managed for you.
If anything, my critique of OP's implementation is actually the opposite of yours: they've chosen the right primitive and layer, but people really need contextual access control rather than RBAC. Sort of like ongoing zero trust. If it was possible to inspect the context, decide if it was a bad idea to allow the tool call, without exposing the decider to untrusted context, you could have something that really changes things.
Runtime enforcement already exists (the embedded governor wraps tool calls in-process); extending it with a quarantined contextual evaluator like you describe would the logical next step.
Thanks for the feedback, actually will raise an issue on that to explore
I've been working an open-source toolkit to stop AI agents from running amok.
You can scan your code (Python, JS, TS) and it will flag any risks and can offer fixes. It runs offline, but you can wire an LLM to do code analysis as well.
You can run it with:
npx @makerchecker/scan
Would love to get any feedback!