Hacker News Reader

Doesn't that mean these 33k tokens can be cached, since they don't depend on the input? The model can just start generation on the 33k+1th token.

View on HN

right, but the cache retention time is very short for Anthropics LLMs. 5 minutes or 1 hour (with additional costs). So you have to prompt basically non stop to not get a cache eviction.

Anthropic even changed this silently: https://www.reddit.com/r/ClaudeAI/comments/1sk3m12/followup_...

A const prompt across all of Anthropic's subscribers could draw from a global cache rather than per-user?

Although saying that out loud makes me question it - each per-user chat and growing cache would need eventually to own its own ~contiguous memory block.

If there are no changes to the system message, yes this is possible and also likely done by Anthropic. When there are additional local MCPs, reusability will be lower.

I think that Anthropic will bill you in any case :D

Yes, but all subsequent tokens do depend on them, so they are still being charged for

Reply on news.ycombinator.com