Hacker News Reader

Comment by HDThoreaun | Hacker News Reader

I really can’t stand when writers point to the difference in price per token on the api and subscription and use that as evidence that inference loses money. This author even says it’s implausible that the api charges 4x marginal cost when I think it’s very likely even higher than that. The entire rest of the post sits on this faulty assumption. Fixed costs don’t matter when marginal revenue is profitable and growing rapidly. The ai labs only have 2 questions. Can they prevent users from switching to open source models? Can they scale the number of users on enterprise plans the way they did for coding but in a more general way for all knowledge jobs?

View on HN

Then what are the real costs?

Wrote this a while back. https://martinalderson.com/posts/no-it-doesnt-cost-anthropic...

OpenRouter is the best guide to real costs.

Thanks, that’s exactly what I was looking for!

And much more informative than the speculation and guessing in the article.

agreed, this doesnt even account for prompt caching or the fact that anthropic has substantial proprietary efficiencies on their inference stack specific to their models and scale.

> Can they scale the number of users on enterprise plans the way they did for coding but in a more general way for all knowledge jobs?

Do these knowledge jobs have a significant corpus of not only knowledge but discussion and problem solving, all conveniently labelled for the AI to train on? Probably not. Coding has stack overflow, what does, say, advertising use?

I agree this is a hard problem for the labs. I would be hesitant about “probably not” though. There is just as much marketing copy floating around as there is coding training data. I struggle a bit in this question because I’ve only ever worked as a software engineer, so I can’t exactly make claims about all the work other jobs do. But, one example is I was talking to a doctor friend of mine the other day. He was talking about how he had to take his recertification exam recently and put the questions into chatGPT and thought it gave answers that were generally more thoughtful and correct than his own. Does that mean doctors are done? Of course not, but he’s now pushing hard for more ai tool use in his practice.

> Coding has stack overflow, what does, say, advertising use?

Advertising has centuries of print ads, 100 years of radio advertising, 70 years of TV commercials, etc. And modern AI does not necessarily need labeling.

My suspicion is coding agents didnt just train on source code but also on human discussions from message boards. There is no equivalent source for this data in advertising... That I know of. Any large scale question and answers boards pertaining to advertising, that have pretty much covered every common query of the past twenty years? If that exists, then I'm wrong.

Reply on news.ycombinator.com