Comment by altcognito | Hacker News Reader

Comment by altcognito | Hacker News Reader

Parent Full thread

It is funny that so many comments vascilate between "It is so expensive these companies can't make money and will go bankrupt in seconds" and "Inference is so cheap that these companies can't make money and will go bankrupt in seconds".

I never take them seriously, I just assume they are coming from countries that don't understand how capitalism works or are operating out of bad faith. The underlying reality of the market is always changing and needs are always changing. Some AI companies will fail, that is a given. Remember alta-vista? Yahoo? Did search go away? How about Microsoft phones? Nokia? Motorola?

OpenAI and Anthropic are not in the inference business. That is a commodity. They need to sell products and solutions.

They’re not contradictory positions. Inference is too expensive now to make money because the industry is immature and hasn’t yet optimized for financial success while customers don’t care much about price because they’re more concerned about not missing out.

Inference will be too cheap long term to make money because it is being commoditized and customers will start to care about results and not just be wowed by impressive technology.

And of course this technology will continue to exist but that is irrelevant to the business. OpenAI investors don’t care if LLMs exist in 10 years, they care if their investment in OpenAI has made money.

How much conclusive evidence people need to stop parroting that inference is expensive? Literally this article is another example showing it's cheap and just got massively cheaper.

The article is about how a month ago a new approach allowed inference costs to be cut substantially. Anthropic currently spend over $5bn per month on compute and have over $400bn in committed spend over the next 5 years. Inference is, by any measure, expensive, it’s just now getting less expensive.

Relative to traditional software margins, the type of margins we are all used to, inference is obscene.

Reply on news.ycombinator.com