$38k AWS Bedrock bill caused by a simple prompt caching miss
This was not a leaked key. This was not crypto mining. This was not an infinite loop. This was not one ridiculous request.
It was a normal local coding-agent workflow:
Droid -> OpenAI-compatible API -> LiteLLM -> AWS Bedrock -> Claude Opus 4.6
I assumed prompt caching was working because every layer in the chain made that assumption feel reasonable:
- Claude supports prompt caching - Bedrock supports prompt caching for Claude - LiteLLM supports Bedrock - Droid can talk to an OpenAI-compatible endpoint
But the bill told a different story.
The gross Opus usage was $37,901.73. AWS credits covered about $8,026.54, leaving roughly $29,875.19 net.
The expensive line item was not output. It was repeated uncached input:
- uncached input tokens: ~6.47B tokens, ~$35.6k - cache read input tokens: ~1.67B tokens, ~$918 - cache write input tokens: ~101M tokens, ~$698 - output tokens: ~25M tokens, ~$698
So yes, there was some cache activity. But nowhere near enough for a high-frequency agent workflow.
The thing that makes me angry is that all of this was allowed to fail silently.
“Prompt caching is supported” is not the same as “your actual agent stack is using prompt caching correctly.”
“Budget alerts are configured” is not the same as “spend will stop.”
“Credits are applied” is not the same as “you will notice the bad cost structure early.”
These are soft signals pretending to be safety boundaries.
For LLM agents, that is not good enough.
A coding agent can run while you sleep. It can carry huge context. It can repeatedly send repo state, tool schemas, instructions, history, and file contents. If caching is misconfigured or partially effective, the failure mode is not a small inefficiency. It is a runaway cloud bill.
Why is there no simple hard cap at the Bedrock/API level?
Why can I not say:
- this IAM principal may spend at most $200/month - this model may not be called more than N times/day - this workflow may not send more than N uncached input tokens/hour - stop serving requests once the budget is crossed
Cloud providers have had decades to learn that “email me after the money is gone” is not a safety mechanism.
I am not saying AWS, Anthropic, LiteLLM, or Droid alone caused this. I made the mistake of putting a metered premium model into a daily agent workflow before putting hard guardrails around it.
But that is exactly the point: the platform lets a very normal integration mistake turn into a car-sized invoice.
My takeaway:
Prompt caching is not a checkbox. Budget alerts are not a kill switch. Credits are not protection. And metered AI backends need hard spend limits before agents become normal infrastructure.
Has anyone here built reliable guardrails for this? IAM deny rules? API gateways? token-budget proxies? per-workflow kill switches?
Because right now the default feels absurdly dangerous.