Hacker News Reader

Comment by jeanmichelselli | Hacker News Reader

Clearly, the prices and limitations will increase with time if they don't find an efficient way to perform training and inference of LLMs. It would be great to see those issues being seriously addressed and, eventually, being fixed for good. THAT would definitely make an important and practical difference. Not only economically but also scientifically.

View on HN

I always wonder where the people are who continually claim that this whole thing is already profitable are when prices are increased or services cut.

I’ve not heard anyone say that the whole thing is profitable. Companies like Anthropic have said that inference is profitable, but I assume that’s only on a steady state basis, which nobody in the industry has ever reached at this point. As far as I know, everyone is still burning cash with data center buildouts and overall training costs.

I think the people that are saying that mean that the inference is massively profitable. From what I've seen, the margins are between 50 and 90% on an API call. It's the training that's very, very expensive!

As far as I understand, for both OpenAI and Anthropic, currently the majority of their GPUs are being used for training, with only a smaller portion being used for actual customer serving.

Inference requires training. That's a disingenuous point.

It's similar to people talking about drug companies charging hundreds of dollars for a medication that costs dollars to manufacture. Nevermind that it took them 15 years of research and trials to discover it and get it through trials and the approval process. Nevermind again all the drugs that failed at one of those steps.

Reply on news.ycombinator.com