Hacker News Reader

> I have a feeling that the price of 1 million tokens transmitted over the internet is more expensive than cache hit.

And this kinda makes sense. What is cheaper few KB of disk space or internet bandwidth?

100%, but this means we are going to move to stateful APIs on the AI provider's end (like OpenAI already does with Codex and Responses API) to make this work.

Reply on news.ycombinator.com