Hacker News Reader

Comment by 1-6 | Hacker News Reader

With today’s AI, we’re taking a big hammer to the problem using unoptimized but vastly flexible machines like GPUs. Once the code settles down, expect ASICs to run much of the show and lower energy consumption.

View on HN

You're maybe not wrong, but I'm pretty sure I was reading this same comment 10 years ago, when we were just calling it deep learning.

Yeah and it actually works now. It's not just weird dog faces hallucinated with Starry Night.

This is really unlikely, because the operations are pretty basic math and the GPUs do these about as efficiently as possible. Surely those GPUs will be improved further, but a custom ASIC probably won‘t help much with training. And NVIDIA has been producing GPUs that are specifically targeted at deep learning and they control both the software library and the hardware. I don‘t quite see how they couldn‘t optimize whatever they want to optimize.

For inference, there is the option to produce ASIC's with the weights hard coded into the chip. They'll be super fast and power efficient, with the huge downside they aren't reprogrammable.

The chips won't be that much more efficient on a like for like basis but the models will be much smaller so the chips can be smaller or run much larger batches

Being that us meatbags operate on a pretty limited power budget I'm guessing there is still a lot of algorithmic optimization to be done, and those algorithms will need hardware to run on.

> Once the code settles down, expect ASICs to run much of the show and lower energy consumption.

If history is anything to go by energy consumption will remain the same or increase, we’ll just have more output

The H100 at this point includes just the bare minimum hardware to be considered a GPU. As long as AI is primarily matrix multiplication, GPUs are going to be near ideal.

It's still spending most of its space on generic floating point units. And there's no hardware support for compressed weights as far as I know.

It is on Groq's LPU units iirc.

The models will simply fill the unused compute with additional training or more expensive models.

There are already some custom chips out there such as groq's[1]. And isn't Google using a lot of TPUs for this sort of thing now? Microsoft also said to be working on custom AI chips.

[1]https://groq.com/

> expect ASICs to run much of the show and lower energy consumption.

further increasing the efficiency of the process will only increase total consumption even further.

the entirety of the AI boom has been a perfect demonstration of Jevons Paradox in action. Previously these intractable problems simply went unsolved. Now it is possible to solve them with a plausible degree of efficiency, and that increases the demand for solved problems hugely because the price per solved problem has declined. If you push energy even lower it will only fuel even further demand for more solved problems because wow, it’s cheap now!

Agreed, using neuron layers as vectors seems like a waste. Moving to a graph-like compute model seems like it'd be more efficient.

... and then the energy saved by using more efficient hardware and software will be used to power more hardware to increase total AI power. There is no "enough" AI yet. There may never be.

Jevon's paradox basically. AI use cases that did not have RoI will open up as it becomes more efficient. The enough is always as much as it can.

Reply on news.ycombinator.com