Opencode said they are working on matching the old prices using their own inference.
Right now, they give 4100 credits for Luna and 63 000 for Deepseek on their prepaid plan (both are 2x)
Right now, they give 4100 credits for Luna and 63 000 for Deepseek on their prepaid plan (both are 2x)
Do they just set a super low caching time and hope that drops effective cache rates low enough? Do all other providers somehow overcharge by that much? Are they just going to sell it as a loss leader?
This, I think. Cached inputs have an opportunity cost (keeping the KV cache until use) but a hit is basically free. “Basically” - if the cache is offloaded to system RAM or NVMe there’s some scheduling overhead.
From a consumer viewpoint a more interesting metric than the raw costs is
cached cost * hitrate + input cost * (1 - hitrate)
I don't have a clue on what the real cost to inference providers comes out to, but it seems really weird that there would be such a big gap, in what should be a pretty competitive market.
This can somewhat be the case, depending on your config. I updated mine to make DeepSeek high priority because I was having a lot of cache misses and reliability issues with the default (cheapest (at face value)) providers, and cost was actually higher overall than anticipated. Was smooth sailing from then; might have to tweak things again now pricing has changed though.
Not sure if this was an OpenRouter issue or with the other inference providers.