Comment by kd913 | Hacker News Reader

Comment by kd913 | Hacker News Reader

The biggest discussion I have been on having this is the implications on Deepseek for say the RoI H100. Will a sudden spike in available GPUs and reduction in demand (from efficient GPU usage) dramatically shock the cost per hour to rent a GPU. This I think is the critical value for measuring the investment value for Blackwell now.

The price for a H100 per hour has gone from the peak of $8.42 to about $1.80.

A H100 consumes 700W, lets say $0.10 per kwh?

A H100 costs around $30000.

Given deepseek, can the price of this drop further given a much larger supply of available GPUs can now be proven to be unlocked (Mi300x, H200s, H800s etc...).

Now that LLMs have effectively become commodity, with a significant price floor, is this new value ahead of what is profitable for the card.

Given the new Blackwell is $70000, is there sufficient applications that enable customers to get a RoI on the new card?

Am curious about this as I think I am currently ignorant of the types of applications that businesses can use to outweigh the costs. I predict that the cost per hour of the GPU dropping such that it isn't such a no-brainer investment compared to previously. Especially if it is now possible to unlock potential from much older platforms running at lower electricity rates.

Why is there this implicit assumption that more efficient training/inference will reduce GPU demand? It seems more likely - based on historical precedent in the computing industry - that demand will expand to fill the available hardware.

We can do more inference and more training on fewer GPUs. That doesn’t mean we need to stop buying GPUs. Unless people think we’re already doing the most training/inference we’ll ever need to do…

“640KB ought to be enough for anybody.”

Historically most compute went to run games in peoples homes, because companies didn't see a need to run that much analytics. I don't see why that wouldn't happen now as well, there is a limit to how much value you can get out of this, since they aren't AGI yet.

This just seems like a very bold statement to make in the first two years of LLMs. There are so many workflows where they are either not yet embedded at all, or only involved in a limited capacity. It doesn’t take much imagination to see the areas for growth. And that’s before even considering the growth in adoption. I think it’s a safe bet that LLM usage will proliferate in terms of both number of users, and number of inferences per user. And I wouldn’t be surprised if that growth is exponential on both those dimensions.

> This just seems like a very bold statement to make in the first two years of LLMs

GPT-3 is 5 years old, this tech has been looking for a problem to solve for a really long time now. Many billions has already been burned trying to find a viable business model for these, and so far nothing has been found that warrants anything even close to multi trillion dollar valuations.

Even when the product is free people don't use ChatGPT that much, making things cheaper will just reduce the demand for compute then.

Everyone uses chatgpt now. You too. Hundreds of time per day.

It's just not called chatgpt. Instead it is at the top of every Google search you do. Same technology.

It has basically replaced search for most people. A massive industry turned over in 5 years by a totally new technology.

Funny how the tech took over so completely it blends into the background to the point where you think it doesn't exist.

Using it doesn't mean people like it. It's forced on us. See recent stories about Microsoft.

> It has basically replaced search for most people.

Not because it's better than search was, though.

They lost the spam battle, and internally lost the "ads should be distinct" battle, and now search sucks. It'll happen to the AI models soon enough; I fully expect to be able to buy responses for questions like "what's the best 27" monitor?" via Google AdWords.

I think training demand is what you might predict would plummet.

Inference demand might increase but you could easily believe that there’s substantial inelasticity currently.

Over the long run maybe, but for the next 2 years the market will struggle to find a use for all this possible extra gpus. There is no real consumer demand for AI products and lots of backlash whenever implemented eg: that Coca Cola ad. It's going to be a big hit to demand in the short to medium term as the hyperscalers cut back/reasses.

Seems like your reasoning for how the next 2 years will go is a little slanted. And everyone in this thread is neglecting any demand issues stemming from market cycles.

There's no consumer demand for AI?

In a thread full of people who have no idea what they're talking about either from the ML side or the finance side, this is the worst take here.

OpenAI alone reports hundreds of millions of MAU. That's before we talk about all of the other players. Before we talk about the immense demand in media like Hollywood and games.

Heck there's an entire new entertainment industry forming with things like character ai having more than 20M MAU. Midjourney has about the same.

Definitely. An industry in its infancy that already has hundreds of millions of MAU across of it shows that there's zero demand because of some ad no one has seen.

Could even argue that the price should go up, since the amount of with one GPU can do and its potential ROI just increased

Reply on news.ycombinator.com