Hacker News Reader

Comment by a_c | Hacker News Reader

Every subagent send the same ~30k system prompts. If you are using fable/opus, that's easily 30% of a 5-hour window for 7 subagent, before doing any work

View on HN

The shared prompts are all cached so it's a cache read which is like 10x cheaper than a regular prefill

I'm gonna jump in here and say that the way Anthropic's KV cache works (prefix matching) doesnt guarantee this at all if the subagents are all put into flight at close to the same time. There's nothing in the agent SDK that allows you to instance more than one agent simultaneously, and the docs don't say anything special about KV cache use by agents in this scenario.

You could probably engineer your own harness or workflow to warm the cache by running a single warmup agent, but this is not at all how the tools behave out of the box. It's mostly a cost optimizaiton, so Anthropic really don't have any incentive to do it proactively, and if they did, surely they would want to tell their customers how much they were saving?

Well yeah Claude caching is frustrating. If implemented correctly subagents are really cheap.

I recently did a few tests. And always the same prompt has been cached properly.

Cache is usually not shared between agents - they can have different base prompts, tools, and be an entirely different model.

Compared to the primary agent, maybe. But it's highly unlikely that all the agents have different tools and system prompts than each other, and those account for the bulk of the context per the post.

Depends on if they are launched serially or in parallel then.

The system prompt and available tools would likely only change for different agent types. So how they're launched probably doesn't matter.

I say this as I don't actually know how Clade Code does this, since it's not open source, but I fail to see why two agents doing the same thing launched at different times would have different tools and system prompts.

I think you are right and most people here are overthinking it.

If you make a bunch of identical API calls quickly, before the initial request completes its cache write, you will have cache misses.

If you make a single API call to warm the cache, then make a bunch of subsequent identical requests, you will have cache hits.

The optimization for this sort of thing is to deduplicate in-flight requests, but i've seen no evidence that is being done; the better design pattern would be an api or tool call that explicitly launches multiple agents after a shared prompt processing stage

There's no single call that can launch multiple event loops at the same time, so there really isn't a notion of serial or parallel; it's always going to be a simple question of how the first model calls made by each hit the backend. I highly doubt that a bunch of agents created within milliseconds of each other are going to get cache hits. It's more likely they will spin up in parallel. I strongly suspect if Anthropic were trying to deduplicate their inflight requests for cache efficiency, they'd at least want to tell you about it.

If it's always the same prompt, can't they have it pre-cached globally for all?

I'm pretty sure the system instructions are a function of your environment and not the same universally. That said, there should be a finite number of branches so still cacheable.

System specific stuff is probably quite limited, it can be a short dynamic segment at the end of the system prompt, perhaps.

The system behaviour is totally up to anthropic's discretion. Its current behaviour is verifiable. In claude code, spawn a subagent with

1. Agent("Test")

2. look at your token usage

3. Repeat a few times

I didn't check again as I type this message but am somewhat sure subagent doesn't cache system prompt as of maybe last week

Reply on news.ycombinator.com