Hacker News Reader

Comment by verdverm | Hacker News Reader

I called them "artificially incessant" after I watched our PR orchestrator agent use subagents to work around permissions to read files, despite instructions that explained the intentional restrictions. I've since added more markdown telling it that using subagents to work around these is a security violation. We'll see if this tactic is mostly reliable

View on HN

I think we should aim to move these security violation rules away from the prompt, to a deterministic place. Not sure how your orchestrator works, but is it possible to add a check between the agent's decision and its execution? e.g. the agent decides to read a file, that decision goes somewhere that checks if the agent has permission to do that or not before it actually reads the file.

Totally agree and this is actually on my short list. I use opencode, so one will need a custom plugin to gate tool calls. They have permission config, which I'm using to block file reads, but the orchestrator is allowed to use specific subagents for more targeted reviews, and they have file read permissions, which the main agent is abusing.

Some of them can be deterministic rules, but others cannot. For example, if you want to permit the GH cli for adding comments, but not merging...

1. GitHub has not provided granular enough tokens

2. You can wildcard in the opencode config

3. The agent can work around this with bash, if it has access

4. The agent apparently will also use subagents, who do have the permission, to work around it's own permission limitations. (This is the problem I'm actually facing)

This pattern is the "relentlessly proactive" as Simon Willison calls Fable, or "artificially incessant" as I called it this week (kimi in this case). It's the same training that enables the long-horizon task completion and mythos style hacking, double edged sword.

I think it unlikely we can block all avenues with deterministic only tools. I'm also looking at policy tuned micro llms, and then creating a merged 'or' signal from the various checks. Later I can look into loosening the signal if there are too many false-positives

This is interesting. Thanks for sharing more. Looks like it's a trade-off of it being relentless, which is something we want in some cases. We need to figure out a way of closing the door and at the same time signaling the door is closed so it does not keep trying different manners.

I've been thinking about this but in a different context: non-coding agents. e.g., an AI agent that approves travel expenses is not allowed to approve expenses bigger than X USD (no matter what). In this case, it looks closer to an ACL thing.

take a look at ADK, they have features for exactly this, mixing determ with agentic in a dag

You can't enforce everything with a deterministic layer. Some things have to be communicated as a policy and tactically remanded on trigger.

Reply on news.ycombinator.com