Hacker News Reader

For OAI and Anthropic at least you can set a spend limit per response. Also tokens are well-defined.

I'm not worried about the volatility in the definition, i'm worried that I give it 1 token today and receive 2 token output, tomorrow I receive 40. If i'm doing this a hundred thousand times a day it is difficult to price this in for users downstream or in the extreme cases be able to absorb that at all short of going into a failmode with degraded access until someone goes and buys more tokens or gets the bill. The alternative is just pass the buck and bill your non-technical customers with a "tokens" line iteim every month.

No, those doing this 100k times a day have very good data on this, good estimators and modeling. And the API has various knobs to change and evals will give you actionable data.

> If i'm doing this a hundred thousand times a day it is difficult to price this in

When you’re doing this 100K times per day you get an extremely good idea of what it costs. You also have all the tools to see when something starts changing quickly.

This change is for Claude Code the harness. If you’re using the API at scale and paying full price then you get exactly what you put into the request.

Reply on news.ycombinator.com