Hacker News Reader

Comment by simianwords | Hacker News Reader

We have people suggesting that ai is so costly to run that all labs are secretly subsidising tokens and we can expect a reprice soon.

Then we have these articles that say tokens will get so cheap that labs won’t know how to make profit.

Who is correct?

View on HN

They're not secretly subsidizing, they're openly subsidizing.

Token pricing was a small minority of customers up until this year, when all the labs started trying to force customers onto token-based billing. Within the last week, Anthropic repriced my team's plan from a temporary "50% extra tokens" to 25%: https://support.claude.com/en/articles/15910845-claude-code-...

The fact that all this is ongoing within such a short timeframe should make you suspicious of any analysis that claims to be observing "statistical trends" like they've discovered a new Moore's Law out of 6 months of pricing data from 2 companies.

your repricing has nothing to do with subsidising which means selling at a loss. Within this year, the real prices have gone down more than 10x on average which is way more than the teeny 25% you are fighting for.

I find it difficult to believe the inference only providers (Baseten, Fireworks, Digitalocean, etc) are all selling tokens at a loss.

Asking Claude for a rough estimate based on publicly available throughput and cost data for open weight models on modern GPUs suggests serverless, pay-as-you-go inference is profitable on owned GPUs with reasonable utilization (30-50%).

The cost is decreasing quickly, mostly because the labs stopped competing on quality. At the same time, the costs are enormous, and all labs are very openly subsidizing usage hoping to get enough scale to be profitable (while 1 of them has suspicious unity numbers and can be hiding negative marginal income).

Also, it's impossible that they become cheaper than specialized software. Or even as cheap as them. It's still possible that they become cheap enough that it doesn't matter.

how is this nonsense still so persistent? There's piles and piles of evidence that inference has massive gross margins at api pricing. what are you actually talking about?

IKR?? The problem with the discourse is that there are still people who believe in this version of conspiracy theory. You can now see the reason for why I posted the original comment. There are people who believe in both extremes - strange world

The labs themselves when they openly say that they’re subsidizing tokens, I’d imagine.

where? I mean the API costs

Reply on news.ycombinator.com