Hacker News Reader

Comment by itsoktocry | Hacker News Reader

>I don't see things ending well for OpenAI.

I mean, what exactly do you see happening? The have a product people love and practically incalculable upside potential. They may or may not end up the winners, but I see no scenario in which it "doesn't end well". It's already "well", even if the company went defunct tomorrow.

>that clause with that timeline seems a bit risky.

I'm 99% certain that OpenAI drove the terms of this investment round, they weren't out there hat in hand begging. Debt is just another way to finance a company, cant really say it's better or worse.

View on HN

I don't think it really matters how much people love their product if every person using it costs them money. I'm sure people would love a company that sold US$10 bills for 25c, but it's not exactly a sustainable venture.

Will people love ChatGPT et al just as much if OpenAI have to charge what it costs them to buy and run all the GPUs? Maybe, but it's absolutely not certain.

If they "went defunct" tomorrow then the people who just invested US$6bn and lost every penny probably would not agree with your assessment that it "ended well".

Model training is what costs so much. I would expect OpenAI makes a profit on inference services.

Running models locally brings my beefy rig to the knees for about half a minute for each querry for smaller models. Answering querries has to be expensive too?

The hardware required is the same, just in different amounts.

It’s less (gross) expensive for inference, since it takes less time, but the cost of that time (per second) is the same as training.

Obviously, that's my point.

We can do the math. GPT-4o can emit about 70 tokens a second. API pricing is $10/million for output tokens and $2.5/million for input tokens.

Assuming a workload where inputs tokens are 10:1 with output tokens, and that I can generate continuous load (constantly generating tokens). I'll end up paying $210/day in API fees, or $76,650 in a year.

Let's assume the hardware required to service this load is a rack of 8 H100s (probably not accurate, but likely in the ballpark.). That cost $240k.

So the hardware would pay for itself in 3 years. It probably has a service life of about double that.

Of course we have to consider energy too. Each H100 is 700watts, meaning our rack is 5.6 kilowatts, so we're looking at about 49 megawatt-hours to operate for the year. Let's assume they pay wholesale electricity prices of $50/mwh (not unreasonable), and you're looking at a ~$2,500 annual energy bill.

So there's no reason to think that inference alone isn't a profitable business.

That doesn't sounds like brilliant margins, to be honest. You've left out the entire "running a business" costs, plus the model training costs. They need to pay their staff, offices, and especially lawyers (for all the lawsuits over the scraped content used to train the models).

It's not unusual for a startup to not be profitable, and they're obviously not as the company doesn't make a profit, but I'm not sure why isolating one aspect of their business and declaring it profitable would justify the idea that this company is inevitably a good investment "even if the company went defunct tomorrow".

Perhaps you meant "win" in the sense of "being influential" or something, but I'm pretty sure the people who invested billions of dollars use definitions that involve more concrete returns on their investment.

Oh they are 100% losing money hand over fist if you include training costs and the eye-watering salaries they pay some of their employees.

I was responding to someone upthread suggesting that they were running even inference at a loss.

You're missing the fact that requests are batched. It's 70 tokens per second for you, but also for 10s-100s of other paying customers at the same time.

All these efficiencies just increase OpenAI's margin on inference. Of course it's not "one cluster per customer" and of course a customer can't saturate a cluster by themselves, my illustration was only to point out that the economics work.

Inference alone totally can be. Just look at banana.dev, runpod, lambda labs, or replicate.

The issue is OpenAI is not just selling inference.

Though I wouldn’t be surprised if there were some hidden costs that are hard for us to account for due to the sheer amount of traffic they must be getting on an hourly basis.

Oh actually banana.dev shutdown. Maybe it’s not as profitable.

70 somethings per second, is slow. So that means it does take a very significant amount of resources, considering it's running on the same or better hardware. To sustain 70 things per second for thousands of users, it gets expensive really quickly.

My point is that at current API pricing the users are paying enough to cover inference costs.

> The have a product people love and practically incalculable upside potential

I'm willing to bet that if you swapped out GPT with Claude, Gemini or Llama under the hood 95% of their users wouldn't even notice. LLMs are fast becoming a commodity. The differentiating factor is simply how many latest NVIDIA GPUs the company owns.

And even otherwise, people loving a product isn't what makes a company successful. People loved WeWork as well. Ultimately what matters is the quarterly financial statement. OpenAI is burning an incredible amount of money on training newer models and serving every query, and that's not changing anytime soon.

> I'm willing to bet that if you swapped out GPT with Claude, Gemini or Llama under the hood 95% of their users wouldn't even notice

You can say exactly the same about Google and Bing (or any other search engines), yet Google search is still dominant. Execution, market perception, brand recognition, momentum are also important factors, not to mention talent and funding.

Not everyone who wants to invest, can invest in this round. You may bet the investors are wrong, but they put money where their mouth is. Microsoft participate, even though they already invested $13b.

Thing is, when I go onto Google, I know I'm using Google. When my employees use the internal functions chatbot at my company (we're small but it's an enterprise use case), they don't know whether it's OpenAI or Claude under the hood. Nor do they care honestly.

From an API point of view (I.e. developer/corporate usage), I am quite sure that OpenAI is lower usage than Gemini now, and Anthropic API revenue is like 60% of OpenAI based on recent reporting. OpenAI is definitely not dominant in this area.

The aspect of corporate usage where OpenAI seems to be ahead is in direct enterprise subscriptions to ChatGPT.

Yeah because no company that attracted funding from lots of top VCs has ever failed...

How is A16Z's massive crypto fund doing again? And what about Softbank's other big bets?

I use LLMs for coding and I would instantly notice. It’s GPT4 or Claude, Gemini a close third, llama and rest are far away. The harder the question, the better OpenAI performs

> they have a product people love and practically incalculable upside potential.

They are not the only ones with the product. They don't have a moat. They are marginally better than competing models, at best. There is no moat into LLMs unless you happen to find the secret formula for super intelligence, hope nobody else finds it, and lock all of your R&D in the Moria mines so they don't go working and building it elsewhere.

There is no moat here. I can't believe so many intelligent people, even on HN cannot grasp it.

the default answer is to love ChatGPT but be unable to use it because of the prohibition on competition. Who wants to chat with something that learns to imitate you and you can’t learn to imitate it back? Seems like everyone using ChatGPT is sleepwalking into economic devastation…

also, for my use case I eventually found it’s faster and easier and less frustrating to write code myself (empowering) and not get sucked into repeatedly reminding AI of my intent and asking it to fix bugs and divergences (disempowering)

Plus, you can find alternatives now for the random edge cases where you do actually want to chat with an out of date version of the docs, which don’t train on user input.

I recommend we all “hard pass” on OpenAI, Anthropic, Google, basically anyone who’s got prohibitions on competition while simultaneously training their competing intelligence on your input. Eventually these things are going to wreck our knowledge work economy and it seems like a form of economic self-harm to knowingly contribute to outsource our knowledge work to externals…

We already freely distribute entire revision histories with helpful notes explaining everything in the git history.

The git repo and review history for any large project is probably more helpful for training a model than anything, including people using the model to write code.

It sounds like you are happy to not use LLMs. I’m the opposite way; code is a means to an end. If an LLM can help smooth the road to the end result, I’m happy to take the smooth road.

Refusing to learn the new tool won’t keep it from getting made. I really don’t think that code writers are going to influence it that much. The training data is already out there.

I seems like not ending well is the vast majority of outcomes. They dont have a profitable product or business today.

It seems that me most likely outcome is that they have one replaceable product against many and few options to get return commensurate with valuation.

My guess is that investors are are making a calculated bet. 90% chance the company become irrelevant, 10% chance it has a major breakthrough and somehow throws up a moat to prevent everyone else from doing the same.

That said, I have no clue what confidential information they are showing to investors. For all we know, they are being shown super human intelligence behind closed doors.

> That said, I have no clue what confidential information they are showing to investors. For all we know, they are being shown super human intelligence behind closed doors.

If that were the case, I wonder why Apple passed on this investment.

If that was the case then the valuation would have a couple extra 0s at the end of it. Right now this "super human intelligence" is still figuring out how to count the number of Rs in strawberry, and failing.

> being shown super human intelligence behind closed doors

This seems to be the "crypto is about to replace fiat for buying day to day goods/services" statement of this hype cycle. I've been hearing it at least since gpt-2 that the secret next iteration will change everything. That was actually probably most true with 2 given how much of a step function improvement 3 + chatGPT were.

> The have a product people love and practically incalculable upside potential.

...yet they struggle to find productive applications, shamefully hide their training data and can't substantiate their claims of superhuman capability. You could have said the same thing about Bitcoin and been technically correct, but society as a whole moved in a different direction. It's really not that big of a stretch to imagine a world where LLM capability plateaus and OpenAI's value goes down the toilet.

There is simply no evidence for the sort of scaling Sam Altman insists is possible. No preliminary research has confirmed it is around the corner, and in fact tends to suggest the opposite of what OpenAI claims is possible. It's not nuclear fusion or commercial supersonic flight - it's a pipe-dream from start to finish.

They took money from ARKK and SoftBank this round which suggests that there were a lot of funds passing.

SoftBank wrote a ticket of $500m which is only 2x the min ticket size of $250m.

Reply on news.ycombinator.com