The $2/6 pricing seems to only apply for context under 200K.
Above that (max context is 500K) pricing doubles to $4/12.
Above that (max context is 500K) pricing doubles to $4/12.
Anthropic have a fixed price regardless of context usage.
These per-token pricing schemes aren't directly comparable though since these models all use different numbers of tokens, even for input (Anthropic's recent tokenizer change generates 30% more tokens for exact same input), as well as for reasoning, and context/token usage also varies wildly by harness with Claude Code using 3x the context/tokens of Pi.
Does anyone know why they would charge more for higher context usage (other than they can)?
AI serving cost is apparently mostly hardware depreciation rather than operating cost (electricity etc), and if your large context request is occupying VRAM for some fraction of a second then you are paying for the depreciation that occurs in that time!
e. H100 costs $20-40K to buy, with a lifetime of maybe 3 years, and will only consume maybe $2K in electricity if run 24x7 for those 3 years.
To use an analogy: imagine your friend is the author of an unfinished book. They die with 19 chapters written, and on their death bed ask you to write the 20th chapter. Assuming you're up to the task, you can only do this well if you take the time to absorb the entirety of what's been written so far.
This is how LLMs work. Context caching is an optimization on top of this, but it has its limits.
No longer feels as inexpensive. Will likely just include this in the rolodex of <200k context tasks, like being one of my review agents.
I wish my company gave me more options than just using Claude to test these things out
As always it depends on what you are using them for, and how you are using them.