Hacker News Reader

Comment by mrinterweb | Hacker News Reader

Investing in AI infrastructure seems very risky to me. With how short of a lifecycle computing hardware generally has, does it make sense to blow billions on hardware that will need to be replaced in a few years? AI companies have been burning billions on hardware. Will the costs of that hardware be covered by profits before the hardware lifecycle is up? That's a big gamble.

We used to talk about AI as having no moat (easy for other players to accomplish similar AI achievements). China has made that clear. Some open weight LLMs are fine and can run on laptops. It seems the new moat is model parameter size and VRAM requirements. I would bet on innovation in hardware or disruptive algorithms changing that game so better LLMs can run on more personal computers. Remember how bitcoin miners all used GPUs then ASICs came along and made that no longer profitable?

There are many ways the AI industry can be disrupted which makes it that much more volatile.

View on HN

Considering we're two years in and everyone's still... talking about chatbots pretty much. Yea, I'd say that's a hell of a gamble.

Of course the hardware can probably be repurposed towards crypto mining afterwards, so hey!

The chatbot focus may be a news/accessibility bias. Take a look at deepmind's research blog to see applications in more areas.

There's definitely some cool stuff coming out of it!

Is it going to be as world upending as some folk were making it out to be? I'm not sold on that yet.

Once you can run a GPT5 level LLM locally on a device, it’s over. All this mighty infrastructure is no longer any more impressive than a top of the line 2013 Mac Pro in 2025. I think we’re 10 years away from that.

Have you tried the reasoning mode of Gemini Pro 2.5? https://aistudio.google.com/

It gives me the chills, thinking about when it has 1000x cheaper ~GPU compute.

This a hosted model with closed weights, though.

Yes. Not the point.

10 years from now, the capabilities of gpt5 will be as relevant to AI as Atari is to modern gaming

Unless GPT 5 is more like the PlayStation 5 and not the Atari 2600.

So people will be paying money for the nostalgia of ChatGPT after it dies? That tracks.

Walmart will sell cheap Chinese emulations to hop on the nostalgia train too.

I doubt it. Newer state of the art models might be a little better, but not enough to justify paying $1000/month for the average person or employee.

If you can get a GPT5 level AI, locally and privately, for just the cost of electricity, why would you bother with anything else? If it can’t do something, you’d just outsource that one prompt to a cloud based AI.

The vast majority of your prompts will be passing through a local LLM first in 2035, and you might rarely need to touch an agent API. So what does that mean for the AI industry?

Consumer devices are already available that offer 128gb specifically labeled for AI use. I think server side AI will still exist for IoT devices, but I agree, 10 years seems pretty reasonable timelie to buy a GTX 5080-sized card that will have 1TB of memory, with the ability to pair it with another one for 2TB. For local, non-distributed use, GPUs are already more than capable of doing 20+ tokens/s, we're mostly waiting on 512gb devices to drop in price, and "free" LLMs to get better.

Are we constrained by RAM production?

RAM Price per GB Projected to decline at 15% per annum.

That's quite a few years before you'll get double the RAM.

For mobile I'm guessing power constraints matter too.

My guess was that nvidia is limiting memory size on consumer cards to avoid cannibalizing their commercial/industrial sales. I see no reason why a 5060 or 5070 can't come with 64/128/512gb memory outside of intentional decisions to not support those memory sizes; I don't need a 5090 as I don't need more than ~20-40 tokens/s as a 1-4 user household system

While I agree with the general sentiment on throwaway compute infra, the generated know-how with large scale experiments is not thrown away. I think a lot hinges on the scaling laws and whether you will hit the jackpot at a certain scale before everyone else. This is hard to guesstimate so someone has to do it in the spirit of empiricism. This might sound a lot like gambling or exploring depending on your sentiment. So, I think it is more justified to criticize the scale and the risks than the spirit of these investments.

While scaling laws are only empirical curve fitting and extrapolation, none of them predict a discontinuous "jackpot effect."

AFAIK, certain abilities such as understanding arithmetic manifest at discrete scale points even though there is a continuous build up of potential. There is also the more remote possibility of a discrete scale that AI takes over its own training or at least starts to contribute substantially. A lot of real world leverage and arbitrage depends on such discrete surprises that may not be visible during the continuous incremental evolution. I think this principle holds computationally as much as it does biologically.

>Investing in AI infrastructure seems very risky to me. With how short of a lifecycle computing hardware generally has, does it make sense to blow billions on hardware that will need to be replaced in a few years?

That also depends on the budget. I work in the public sector, so there's usually no money, but now they've just discovered that they have a 25 million budget, which needs to be spend this year on hardware (for AI). If we do not spend the money within this year, the budget expires and then we cannot spend it at all

25 million is not billions, but that could still be a higher budget than any other public institution agency in the entire state here has

You're assuming that gold is going towards infrastructure. I don't think a lot of it is. I think it's a money grab while the getting's good.

>I would bet on innovation in hardware or disruptive algorithms changing that game so better LLMs can run on more personal computers.

This is classic jevon's paradox. As efficiency increases so does demand.

Very high risk/Insane payoff (maybe).

Pretty much it, albeit I think very few appreciate how high's the risk of leaving with nothing in their hands.

Valuation models probably begin to break down when the potential payoff is control of our solar system as we know it.

OTOH it seems like mere single billions can buy control over the US if you're brazen enough, so perhaps these stock market suckers are overpaying with their trillions.

I wonder how expensive it would be to buy control of PRC in a similarly creative (but very different) way.

Each datacenter iteration that increases the size and power consumption is a stepping stone towards (possibly) recursive self-improvement in AI tech. I think that's the way to frame the absolutely massive bets being made by (only a few) companies. There aren't many companies or nation states in the world that can marshal the resources and talent needed to keep competing on this path.

It's a race to the the most powerful and transformative technology in history.

Or it might all collapse like a house of cards. But worth a shot.

> But worth a shot.

Worth betting the entire US economy on it?

Only a significant portion of it.

No. Nothing is but is that what's happening?

Huge bets were made on the Internet technology in the US and it paid off even with the dot com bubble bursting.

If we don't stay on top of technology then we will simply be passed by China.

People also said the Internet was going revolutionize everything in the 90s and it did. AI is already pretty impressive and we are just in the infancy of the technology.

The thing is, China seems to be able to keep up with much less invested. I don't know how much it took for them to create DeepSeek and their other power efficient models, but it sure doesn't take much to run them. Meanwhile American AI seems to have a limitless hunger for energy.

China is doing what they have done best for decades - lean on western tech, and improve it's efficiency. They're still clearly behind on SOA developments in AI.

But they are obviously rocketing ahead in energy capacity, which will be the real bottleneck as data centers continue to scale.

Then we should become more efficient. I do get your point that the amount of money available as investment could be responsible for inefficient growth (because the focus is only on the goal).

A solution could be competition locally by a more efficient implementations. There might also be push back by local populations if it causes energy prices to rise.

>AI is already pretty impressive and we are just in the infancy of the technology.

What do you mean by infancy? AI research exists for at least 50 years, longer than web development.

You’re just being difficult. It’s obviously changed trajectories massively since transformers came along.

Reply on news.ycombinator.com