Comment by reticulates | Hacker News Reader

Comment by reticulates | Hacker News Reader

Parent Full thread

re: substituting with cheaper models. I think some people here are oblivious to how much of Anthropic and OpenAI’s usage is artificial.

Take this one example of a user with 15 Codex subscriptions ($3k) generating $60k in API-equivalent usage per month: https://hraness.com/writing/my-girlfriend-asked-me-why-i-hav...

“there’s a once in a lifetime discount happening at the OpenAI Intelligence Depot, and I brought 15 shopping carts.”

We have to understand current usage with this behavior in mind, there are tens of thousands of people just like this author who are intentionally generating as much usage as possible on their subsidized plans because they feel compelled by some need to get free intelligence.

The only reason these expensive models are generating so much usage is because they are so heavily subsidized. Pragmatic users will shift to cheaper models which will hurt per token revenue, yes, but huge volumes of usage is going to just disappear because there isn’t the demand when it isn’t being subsidized. Less revenue per token and less tokens.

https://tokscale.ai/leaderboard a small sample of just 2k users have generated over $100m of API-equivalent usage while paying closer to just $1m.

https://aicharts.io/gpt-subsidy

It goes a layer up, too. Harvey (legal AI) recently tweeted that a single user query cost that firm 26k USD. This, while they have thousands of users and reportedly whole law firms paying zero. That difference is being subsidized by billions of VC dollars.

A bit of a collective action problem, but if everyone leverages this services as hard as they can without cost concern, better capital decisions will eventually be made. Like the Federal Reserve draining the M2 money supply, AI users must drain the capital supply of subsidized tokens. This pulls forward the future of "Does this tech have value at actual unsubsidized costs?" If it does, tremendous, if it doesn't, also a reasonable outcome to the grand experiment. The current pain comes from the valley of uncertainty we find ourselves in at the moment.

> That difference is being subsidized by billions of VC dollars.

"History never repeats itself, but it rhymes." -- Twain

Doordash and Pizza Arbitrage - https://news.ycombinator.com/item?id=23216852 - May 2020 (514 comments)

> I cut this deal with my neighborhood Italian restaurant! I texted the owner about being miffed they hadn’t told me they were on DoorDash. He replied. They aren’t. We compared pricing, and found the prices advertised are way off from what the restaurant charges. So I placed a $5,000 order to the neighbourhood homeless shelter. DoorDash paid him over $20,000, and I get free pasta for the rest of the year. (My neighbours have also partaken.) Glad to know it’s scaling. SoftBank has assembled a unique concentration of stupidity for itself.

Yes, there are good examples of this working, anti-competitive as it may be.

What if, in the DoorDash example, the pizza shop was itself venture backed and selling pizza at a loss to win customers and using this 20k revenue as a basis to raise money?

What if the same VCs were backing both and gearing up for an IPO?

Once circular funding scales, VCs become the Fed.

Why do everyone assume they are subsidized? When we seemingly have no idea what it costs? Maybe average subscription is breaking even and token spend is pretty much pure profit?

Case in point, claude code seems hell bent on increasing usage at all cost. Which makes sense in the growing phase (get people hooked) but it does not make sense given the hardware shortage. So, which is it?

Anthropic admitted last year to losing money on inference, it had negative margins. The margins have improved and are now positive but there’s still a significant cost. If plans aren’t being subsidized it would mean that the margin on inference is ~99%+ which would mean OpenAI and Anthropic should be wildly profitable but both are still losing money. So, it’s mathematically impossible that they’re not subsidizing plans.

The most widely accepted estimates (though I disagree with them) are that Anthropic’s margin on API inference is ~70% from which people extrapolate what their token usage would cost via the API and compare that to what their plan costs.

https://newsletter.semianalysis.com/p/anthropic-3q26-profit-...

(edit: better link https://newsletter.semianalysis.com/p/anthropic-growth-and-b...)

re: increasing usage with resets, it’s because they’ve overblown usage and need to show that usage is growing ahead of the IPO. They’re increasing usage on fixed price plans without increasing the cost, the only plausible explanation is they have unused capacity. If they were capacity constrained then the last thing they would do is give away more usage for free.

The last I saw with Claude was that the plans are subsidized somewhere around 10x. The usage of Claude and similar products (for people not paying the actual API token costs) would be lot less if they were paying 10x more per account/seat.

A lot of people are using Claude and ChatGPT for all kinds of minor things at work, and they probably wouldn't be if they were paying the true cost of the product. And this is all the while their work product is suffering because AI is not a great fit for a lot of use cases.

>Anthropic admitted last year to losing money on inference, it had negative margins.

No they did not. Dario has repeatedly stated if they stopped training new models, they would be very profitable.

When?

“The majority of the cost is inference, not necessarily the training of the model.”

https://youtu.be/7xij6SoCClI

> Why do everyone assume they are subsidized?

It's not an assumption when the companies hosting the models publicly announce that they're subsidizing costs.

Depends on whether the costs they have are inflated ten-fold due to hardware shortages.

Ignoring current state of shortages, the interesting part (for me at least) is how much a SOTA token costs on current hardware if you exclude the costs for paying for current TSMC shortage and without factoring in the cost of new datacenters being rushed and renting hardware from your competitors (who are also supply limited).

The "actual" cost is of course also relevant and interesting but it is another thing entirely.

Have they actually announced that they're charging less than the marginal cost of inference, or that the revenue they're taking in doesn't make up for the cost of training each newer and better model?

The former does not pass the smell test when there are random providers selling tokens for competitive open weight models.

You can compare the subscription cost to the API per token cost for similar queries.

That tells you that subscription is cheaper than tokens. Which ought to be a given.

But it does not tell you what and how much either is subsidized...

Given the cost of openweight frontier models like Kimi I’m fairly sure the token prices actually make some money.

theyre not being subsidized, and no ones using tokens just because.

Reply on news.ycombinator.com