Hacker News Reader

Comment by beejiu | Hacker News Reader

It interests me that the $200-600 billion number seems to be all-derived from GPUs. Are LLMs/AIs totally dependent on GPUs? I read last week (https://news.ycombinator.com/item?id=40787349) that there is research ongoing to run LLMs on FPGAs at greater energy efficiency.

I'm reminded of Bitcoin/crpyto, which in its early history was all operated on GPUs. And, then, almost overnight, the whole thing was run on ASICs.

Is there an intrinsic reason something similar couldn't happen with LLMs? If so, the idea of a bubble seems even more concerning.

View on HN

There is a fairly new ASIC named "Sohu" that is purpose-built for transformers. They have some bold claims that are impressive if true.

I found a short discussion[2] you may find useful.

[1]: https://www.etched.com/

[2]: https://www.lesswrong.com/posts/qhpB9NjcCHjdNDsMG/new-fast-t...

This is just a PowerPoint slide at this point.

These numbers are just the hole from GPUs that have already been bought/ordered. Today's GPUs will inevitably be replaced by something, whether it be better GPUs, NPUs/TPUs, ASICs, or FPGAs. As chips get cheaper in the future the hole will grow at a slower rate but it only grows.

>Is there an intrinsic reason something similar couldn't happen with LLMs?

Only if we can increase the efficiency of LLMs by 2-3 orders of magnitude, there are only some in lab examples of this and nothing really being publicly shown.

Even then the models are still going to require rather large amounts of memory, and any performance increases that could boost model efficiency would very likely increase performance on GPU hardware to the point we could get continuous learning models from multimodal input like video data and other sensors.

There is a reason why we don't use ASICs and instead use GPUs.

While people may say something is a Transformer that's more of a general description. It's not a specific algorithm; there are countless transformers and people are making progress on finding new ones.

Bitcoin runs a specific algorithm that never changes. That's for an ASIC. AI/ML runs a large class of models. GPUs are already finely tuned for his case.

Nvidia and AMD bought out the big FPGA makers.

AMD bought Xilinx, but Intel recently spun off Altera.

Opps, yeah was thinking of Intel's Altera buy, not Nvidia, which I guess is now undone.

Reply on news.ycombinator.com