Hacker News Reader

Comment by runtime_terror | Hacker News Reader

> I'm tempted to assume that inference costs are closer to subscription pricing than to API pricing

So just going on vibes?

While some people don't like his content, Ed Zitron shows a lot of evidence for your assumption being very wrong.

These companies are bleeding cash at ungodly rates. It's likely their API pricing is still subsidized if you look at their overall financial picture.

Related, there's a good reason those API prices keep going up a lot every new version and it's not just because the models are better.

View on HN

Selling inference for more than inference costs is not incompatible with bleeding cash at ungodly rates. They do in fact pay ungodly amounts of cash for other things, like training, marketing, etc. Heck, you can bleed cash while being profitable (in the accounting sense)

Also, API prices going up a lot every new version is more an OpenAI thing, and even there it's a recent trend: GPT 5.0 was a big price drop compared to 4.1, and 4.1 was cheaper than 4o, which itself got a price cut at some point and is cheaper than 4. Meanwhile Anthropic's API pricing stayed stable for many versions, then got slashed to a third with the 4.2 release and have stayed at that level since.

But explain to me how these companies will recoup these costs outside of increasing inference pricing?

Their business model is selling inference but the training and other costs have to be accounted for somehow. Unless I'm missing something obvious, inference costs must go up drastically if these companies are going to survive beyond the subsidy stage.

Sell more. The hope is that there is a huge addressable market that includes huge per-worker demand in almost all white collar work and lots of inference in people's private lives

If that doesn't work, then yes, then prices will have to go up

Both anecdotally for myself and from what I'm reading in the news, it seems just as likely that AI usage has already largely peaked.

There was a lot of hype and exploration of capabilities, but models aren't evolving fast enough to keep that going, so I'm settling down into a familiarity with what an LLM can and can't do that means I am using them less overall that I was 6 months ago when I was throwing everything under the sun at it just to see what happened.

Without either new model breakthroughs or dramatically _lower_ costs, I will be very surprised if the ultimate market doesn't end up within an order of magnitude of where it is today.

> AI usage has already largely peaked.

I think this is minimally likely. While as individuals on the bleeding edge, we're perhaps using these tools less and less, and our echo chamber reinforces that, the penetration of AI into the normal corporate workplace is still very low - emails rewritten with ChatGPT, meeting notes summaries generated by default, etc. There are a million use cases for LLMs which are not yet built out. The tokenmaxxers will begin using AI less, but the penetration into the mass market will continue at a huge velocity.

I agree that more uses will be found and that maybe we're not at the peak. But it also seems very clear a few players have been actively working to inflate usage numbers by margins that might take a while to replace with legitimate uses

Exactly. Like how Meta has a "blow our money on LLMs" leaderboard. Seems like a few companies are attempting to inflate hype enough so all the investors can exit without losing their heads.

Reminds me of the crypto hype but where the hype agents are some of the largest companies in the world.

Yeah from my understanding they'll need to create a few trillion dollars more demand to break into profitability if we look at all the debt/obligations/contracts

Obviously they need more paying users. The entire game in tech is taking advantage of (comparatively)low marginal costs to pay off capex once you corner the market

I do think that's at least part of the strategy. The problem is that we've never seen a single product category so hyped in history, literally trillions of dollars invested. To recoup that, some not so trivial miracles will need to happen.

I think that within 5-10 years most white collar workers around the world will be paying for AI assistants. There are 1.2-1.3 billion such people to sell ai to, so getting more users doesnt really seem like a miracle to me. I do think convincing everyone to use expensive proprietary models instead of open ones hosted cheaply by third parties will be a minor miracle for the AI labs. Definitely not out of the question though.

Reply on news.ycombinator.com