Hacker News Reader

Comment by NorwegianDude | Hacker News Reader

A common extreme misconception is that inference is expensive and that providers are loosing a lot of money. Inference is extremely lucrative and profitable.

View on HN

Inference is the phase where they make money. But the question is whether they can be profitable overall as training continues to balloon.

I think the case for this is pretty strong actually. Last year my company was maybe willing to pay $100 a month to Anthropic (per developer). Today we're all on the $300 plan without any hesitation. If Fable ever becomes available as the default model, I imagine my company would be willing to pay in the $500-$1000 range per month per developer.

Okay, but that still has a limit, right? Do training costs have a limit? Everyone is in the frothy stage of this technology wave and they continue to buy more, but training the next model requires exponential increases in model sizes to get the same sorts of model performance increases, which suggests exponential cost increases, too (even ignoring temporary cost factors such as RAM price increases). You say your company will double or triple what they are paying today; how far are they willing to go? At some point they are going to have to cut developers to fund it (e.g., cut half the developers and give the survivors each an AI assistant with $180k in token budget, captured from the salary savings), but that also presupposes the productivity gains are there to support it.

First of all, yes cutting developers to fund AI spend budgets, is the entire operating idea behind these AI companies; and most companies would love doing that. I'm not saying this is a good thing, my heading is on the chopping block like everyone else's.

But isn't this like a Jevon's paradox thing, also? If I'm able to become vastly more productive, and that value produces more sellable output for my company, there's no reason to cut anywhere to fund it. This is the same reason a company like Microsoft can hire 80 000 developers, it's because each dev pays for themselves in value (on average). I guess the same can be true for AI spend?

Reply on news.ycombinator.com