Hacker News Reader

Comment by amelius | Hacker News Reader

The real GPU bubble will be when AI companies figure out they can better make their own ASICs and ditch all their GPUs onto the market.

View on HN

In data center operations, GPUs have some specific lifetime. Because datacenter GPUs are currently so expensive and hard to get, they don't get dropped on the market at some point (even if a better replacement has arrived), but used as long as possible.

Even if the AI companies decide to use their own ASICs, they will rather slowly, but continuously introduce them, while removing GPUs that have reached their end of life.

Yes, short term this is right. But at some point PyTorch will have a model.toVHDL() method, and we'll have a PCBWAY-style website for tapeout of the circuit. Nvidia's future looks less bright than they think and their GPU market will certainly pop.

Doesn't that assume that VHDL is trivial? I feel like there are tons of performance tradeoffs or hardware designers wouldn't have jobs

No it does not assume that. Some very smart people will write that model.toVHDL() function. And keep in mind that a DL model is only a very small subset of what you can use VHDL for, and most models will have a very similar implementation in hardware from a conceptual point of view.

And don't take it too literally, VHDL could be replaced by other hardware design languages, maybe even at lower abstraction levels.

Not trying to take it literally, but aren't there costs vs performance tradeoffs? Like the py.toHDL would have like (maxSize,maxCost,minThroughput) as free and that would determine energy usage?

And a GPU is already pretty optimized for inference, no? Like isn't it a bunch of FP mults? I don't think HDLs do well with that, either.

I can't imagine that model lifetimes will ever justify using model-specific ASICS for public serving (maybe something like serving fixed certified AI models in a vehicle or robot) over more generic GPUs/NPUs until after the AI bubble pops.

Be aware that currently the hardware costs and electric bill are two huge problems of modern LLMs.

If such AI models will deliver on their qualitative promises, and just the huge cost is the burden to overcome, custom ASIC might be a part of the solution.

If, on the other hand, AI models will still be unsuitable for many applications because of their qualitative issues, it is a much harder and different problem to solve - in this case, the AI bubble will plausibly burst.

Reply on news.ycombinator.com