Hacker News Reader

Comment by pfisherman | Hacker News Reader

> The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value.

I do not quite follow. GPU compute is mostly spent in inference, as training is a one time cost. And these chain of thought style models work by scaling up inference time compute, no?

So proliferation of these types of models would portend in increase in demand for GPUs?

View on HN

As far as I understand the model needs way less active parameters, reducing GPU cost in inference.

If you don't need so many gpu calcs regardless of how you get there, maybe nvidia loses money from less demand (or stock price), or there are more wasted power companies in the middle of no where (extremely likely), and maybe these dozen doofus almost trillion dollar ai companies also out on a few 100 billion of spending.

So it's not the end of the world. Look at the efficiency of databases from the mid 1970s to now. We have figured out so many optimizations and efficiencies and better compression and so forth. We are just figuring out what parts of these systems are needed.

Hyperscalers need to justify their current GPU investments with pay2go and provisioned throughput LLM usage revenue. If models get more efficient too quickly and therefore GPUs less loaded by end users, short of a strong example of Jevon's paradox they might not reach their revenue targets for the next years.

They bought them at "you need a lot of these" prices, but now there is the possibility they are going to rent them at "I dont need this so much" rates.

Reply on news.ycombinator.com