Hacker News Reader

Comment by janalsncm | Hacker News Reader

There is a difference between valuable and profitable. I think anyone who wants to say there isn’t a bubble needs to solve two problems:

1) Inference is too damn expensive.

2) The models/products aren’t reliable enough.

I also personally think talking to industry folks isn’t a silver bullet. No one knows how to solve #2. No one. We can improve by either waiting for better chips or using bigger models, which has diminishing returns and makes #1 worse.

Maybe OpenAI’s next product should be selling dollars for 99 cents. They just need a billion dollars of SoftBank money, and they can do 100 billion in sales before they need to reraise. And if SoftBank agrees to buy at $1.01 the business can keep going even longer!

View on HN

I think AI will be useful to industries/companies where #2 is unimportant: Where the quantity of product is far more important than its quality. Disturbingly, this describes the market for a lot of industries/companies.

It seems like every 6 months they come out with a new model that's as good as the previous generation but with a fraction of the inference cost. Inference cost for a given level of quality has been dropping fast. E.g. GPT-4.1 nano outperforms the old GPT-4 model but costs 300x less.

Right now, the API cost for asking a single question costs about a fiftieth of a cent on their cheap 4.1-nano model, up to about 2 cents on their o3 model. This is pretty affordable.

On the other end of the spectrum, if you're maxing out the context window on o3 or 4.1, it'll cost you around $0.50 to $2. Pricy, but on 4.1, this is like inputting several novels worth of text (maybe around 2000 pages).

That would be great news for OpenAI if they could somehow prevent people from buying their own computers. Because as inference costs come down for OpenAI, their customers also get access to better and better commodity hardware (and also better models to run on it). And commodity models become more and more capable all the time, even if they’re not the best.

It counters the claim that "inference is too damn expensive". You can argue that cheap inference is actually bad for OpenAI because it makes it easier to run models on commodity hardware, but then you're arguing against your point that inference was too expensive for OpenAI to be sustainable. Which is it? Is inference too expensive or too cheap?

For now, OpenAI's models are closed source, so if you find their models offer the best value for your use case, you don't have the option of running it on your own hardware. If a competitor releases better products for cheaper, OpenAI will fail, just like any other company would.

It’s both. For an increasing number of tasks, it’s too cheap. People will be able to do simple things on their own hardware. Things OpenAI would love to charge high margins on. There’s probably an 80/20 rule on the horizon.

And for others, it’s too expensive. The frontier is constantly being pushed, so they can’t stop improving or they will fall behind. Google at least makes their own chips so they can control their costs somewhat.

Honestly, it's the training costs that will kill them. AFAIK, training costs have not come down anywhere near as much as inference costs.

And the models don't last long. So you have a rapidly depreciating capital asset that you need to provide your services, not really a recipe for a sustainable business (certainly not with the fat software margins tech companies are used to).

So you suspect their cheap models have few customers because people prefer to run open source models on their own computers, and their high-end models have either very thin margins over the inference cost or few customers because the costs are too high.

Do you have any evidence for any of this?

If you're looking at it from the perspective of business sustainability (i.e. what the article is about) "we keep lowering our prices" doesn't sound so great. The question is whether GPT-4.1 nano costs OpenAI 300x less than GPT-4 to run or not. If it costs exactly that much less, that still means demand needs to grow by more than 300x just to keep revenue constant. And if it does, then total inference cost correspondingly goes up again.

I think that's convoluted logic trying to show that a business lowering its costs is actually bad for the business. You can make this assumptions about any business. It's not a good assumption. If a car company discovered a way to lower its production costs and started selling their cars cheaper, would you argue that maybe this is actually bad for the car company because maybe they lowered their price so much that it cancelled out all the benefit? You're just hoping they're really bad at pricing their cars.

The different GPT models are for different use cases. The existence of the nano model does not imply everyone who used the formerly cutting-edge GPT-4 will switch over to the cheaper nano model. Most of them will switch to the smarter models like 4o, 4.1, or o3. Nano allows the creation of new tools where the other models are either too pricey or too slow to respond to be viable.

It's funny that you bring up cars as a point of comparison, because constantly releasing new models while dropping prices is exactly what made a bunch of Chinese EV companies go bankrupt in recent years.

It's not that they were stupid and priced their cars badly, it's that they didn't have much choice. Developing a new car model has significant upfront costs that require a correspondingly large number of cars to be sold for the company as a whole to turn a profit, each individual sale being profitable is not enough. But in an environment with lots of competitors constantly releasing new models for cheaper, any single company had little choice but to also release a new model and lower their prices in order to sell any cars at all, eating a loss on the previous model. And eventually some companies couldn't take those losses any more and folded.

OpenAI is definitely losing money overall, presumably because training new large language models is expensive. But can they stop doing that to turn a profit? If OpenAI announced that the funding for their next frontier training run fell through and they're laying off research staff to focus on inference, and then competitors come out with better, cheaper models, how long will OpenAI be able to stay relevant? I guess they hope everyone else gives up first and they won't have to find out.

This is all true. Competition reduces a company's profits.

OpenAI reducing their costs isn't what hurts them. They're better off with lower costs. Their competitors getting lower cost models is what hurts them.

Absolutely some AI companies will be out-competed and fail. I don't know which one will end up on top.

But even if almost all the AI companies fail, it doesn't stop the industry as a whole from being very valuable. Whichever AI companies are left will just take over the market share of those that failed. And this eases the pressure on the surviving companies.

> The models/products aren’t reliable enough.

People aren't reliable enough.

Nature isn't reliable enough.

For most uses, all that is needed is a system to handle cases where it is not reliable enough.

Then suddenly it becomes reliable enough.

There are different kinds of reliable. There is the reliability of certain failure for instance. These things are unreliable in a way that the other things that you listed are not.

Nature is reliable in that we have rules (physics) and practices (engineering) to base our process upon.

People aren't reliable, for a specific value of reliable.

We expect of technology(machines, software, AI, whatever) to be: deterministically reliable (knowing their failure modes), and significantly more reliable than humans at what they do (because that's what we rely upon, why we use them to replace humans at what they, in ways humans can't: harder, faster, stronger).

A computer can never be held accountable. Therefore, a computer must never make a management decision.

Reply on news.ycombinator.com