Hacker News Reader

Comment by petesergeant | Hacker News Reader

> demand for tokens/AI inference capacity exceeds CURRENT supply

That's not at all obvious to me; costs as a consumer are going down, rather than up. Can someone steel-man this guy's argument for me?

View on HN

"Demand" for inference seems infinite because you can always think harder or insert AI for free into some workflow where the customer didn't request it (e.g. Google search). It's like a gas that expands to fill the container. Then during the crash people are going to discover that some fraction of that demand wasn't needed and they'll just turn it off.

It would be a good sign if paid demand exceeds supply but I don't know if we can measure that.

Cost to consumers is being subsidized by investors trying to grab market share - penetration pricing.

I don’t think that’s true, given the wide range of inference providers on OpenRouter. These are mostly undifferentiated and commoditized, with no chance of gaining market share at the exclusion of others.

Same issue, you're not seeing what it actually costs, yet. All of those are based on foundation models that have very high training costs - tens to hundreds of millions of dollars. Even smaller models like Mistral apparently cost several million just for compute. The user-facing economics and inference costs are only part of the story.

There is only so many requests people want to process with LLMs.

Even though I'm a touch sceptical as to how good LLMs are at coding, imagine developing world demand for LLMs via a cheap Android handset for $1/month.

There's a lot of use case for the information retrieval side, especially for people whose educational level and mindset means well-crafted search queries were never a thing for them.

But there's also the other side of the trade. How much are suppliers willing to pay for upranking? It's a dirty business model, Enshittification 3.0+, but when did that stop anyone?

Reply on news.ycombinator.com