Hacker News Reader

Comment by santiagobasulto | Hacker News Reader

But doesn't $15M/day of inference cost imply "demand" from users? If this is the case, it's just a matter of time until costs can be reduced.

View on HN

> If this is the case, it's just a matter of time until costs can be reduced.

Is it, though? We cannot predict technological advancement, and the times of ̶M̶u̶r̶p̶h̶y̶'̶s̶ ̶L̶a̶w̶ Moore's Law* for computational power are long gone. There is simply no guarantee that the costs will go down enough.

* thanks lucianbr!

Moore's Law.

The times for Murphy's Law for computational power are just beginning.

I think there is plenty of room to make AI inference much more energy efficient. For example, there are companies testing creating custom silicon to run the model. Once that technology matures and we have some "good enough" models for normal use, inference cost for non-bleeding-edge models can come way down.

I don't expect bleeding-edge models to become any cheaper, but previous generation models can potentially be really cheap.

The actual revenue was quoted at $2.1m .. total. Ever.

It would require multiple order of magnitude cost reductions to make that worthwhile. Maybe another few decades of Moore's law, if we have that left.

This was the Moviepass model of selling $10 bills for $9.

Much worse, really; it was selling $10 bills for half a cent.

The Moviepass thing, I think if you were kinda gullible you could maybe buy into it eventually working on scale. This could never work on scale.

Disneyworld has lines longer than the park can manage for decades, do you expect it to just be a matter of time until park management finally figures out how to queue people efficiently enough, or do you think the solution will be once again raising costs for the customer.

I’ll eat lots of free samples at Costco of foods I’d never pay money for

A box that says "Free T-Shirts" can create near-infinite demand.

Paying near-infinity dollars for T-Shirts people want for $0 isn't a profitable business model.

Demand side price sensitivity impacts potential supply side margins.

> But doesn't $15M/day of inference cost imply "demand" from users? If this is the case, it's just a matter of time until costs can be reduced.

If you build a website that gives $100 for free to each one of your users, you’ll quickly have "demand" but that’s not "a matter of time until costs can be reduced".

The "matter of time" is getting more and more expensive, not cheaper, at least for next 2 years

Not sure what you're referring to. If you're talking about inference cost for frontier models, that's going up because researchers keep pushing those frontiers, often without considering cost. And while they're subsidized (to gain market share), users have no reason NOT to use the crazy expensive frontier models.

Once the market consolidates, and users get used to the idea of using models that are "good enough" because frontier models are too expensive, there's no reason AI cannot be profitable.

There's not much profit in inference, it's heavily commoditized. There is an illusion of potential profitability because the closed-weight models are currently a step ahead of the open-weight models. However, if you ignore the closed-weight models, then the open-weight models are also getting better every year. In the limit, the open-weight models will end up just as good as the closed-weight models.

AI is an inverse gold rush, the people who are getting rich off it are the people using it. The shovel-sellers are screwed.

> Not sure what you're referring to. If you're talking about inference cost for frontier models, that's going up because researchers keep pushing those frontiers, often without considering cost. And while they're subsidized (to gain market share), users have no reason NOT to use the crazy expensive frontier models.

The price of GPUs and the price of RAM to put in the servers.

Yes but it was capped by their restriction on sign ups/registrations. That could've easily been in the hundreds of millions if the app had public signups.

Idk if Instagram would exist if they were spending hundreds of millions a day.

Why would demand imply costs will be reduced? If you're making an economies of scale argument, there's plenty scale right now, and costs don't seem to be trending down.

Costs would have to be reduced about 2,000 times just to break even, assuming that inference was the only cost, which of course it was not.

It could be over provisioned or that cost is supposed to be minimum cost with some minimum capacity which was never reached.

Costs were reduced to $0. Can't get better then that for a product that OpenAI had no clue how to monetize

Ha, that's a good one!

Many things are only in demand because they are free.

And if nobody is willing to pay for it, it hardly matters how low you bring down cost, because it’s always a net negative.

Just look at the general state of the internet over the past two decades. Do you think it would work for Sora to insert ads into the slop?

Make a loss on every sale but make it up in volume!

Reply on news.ycombinator.com