Nvidia Announces DGX GH200 AI Supercomputer | Hacker News Reader

Nvidia Announces DGX GH200 AI Supercomputer

nvidianews.nvidia.com

40 pointsbfoks14 comments

The picture almost looks like a 2U server, but it's actually 24 full server racks.

https://www.anandtech.com/show/18877/nvidia-grace-hopper-has...

The amount of RAM, even if it's just DIMM, made my jaw drop. I wish I could fast forward time for 10 years to see how integrated and affordable this card has become, or if it would even be possible to integrate something like this into a smartphone in 20 years. Just imagine how people would be interacting with their phones then, interacting with them as if they were humans.

No other technology gets me as excited as storage and compute, I guess this is how car people get excited if a new Maserati or something else "with numbers" is launched.

Most of that memory would be LPDDR5X.

https://semiconductor.samsung.com/dram/lpddr/lpddr5x/

Oh, right, I misnamed DDR.

Assuming Moore's law continues, the answer is probably yes. At some point, you'll have more computing power in your pocket.

For example, "ASCI Red" was the #1 in the TOP500 supercomputers list in the year 2000 with about 2 teraflops... roughly the same[1] as an iPhone Pro 14.

The #1 from June 2003 -- almost exactly two decades ago -- is about the same speed as a modern NVIDIA desktop GPU.

As recently as 2014 there were still "supercomputers" near the bottom of the TOP500 list that are about equivalent to 2x RTX 4090 GPUs. That's a rare but readily available configuration for a gaming enthusiast.

So it's about 20-25 years until a supercomputer fits in your pocket, and as little as a decade until you can have one under your desk heating your room.

[1] Most supercomputers measure 64-bit ops, whereas these days people focus on 32-bit ops, but whatever. It's just a factor of two.

>or if it would even be possible to integrate something like this into a smartphone in 20 years.

I am going to answer this as probably not.

If you assume we still get 2X density improvement every 2 years ( which we dont ) and the density shrink applies to every type of logic and SRAM ( which we dont ) and we dont slow down the pace for 20 years ( which we dont as we are already slowing ) and we dont hit any heat limitation ( which we do ) and we dont hit any other fundamental limit of law of physics ( which we will, and get quantum tunnel effect ) etc etc

There are 24 Racks of Computer there. Even if you ignore the I/O and Power. There are 16 Racks of Raw computing. Over 20 years, 10 iteration, you get 1024x increase in density.

16 Racks, or 640U Server, you get 0.625 U by 2043. Half a Server, hardly the size of Smartphone.

“Generative AI, large language models and recommender systems are the digital engines of the modern economy,” said Jensen Huang, founder and CEO of NVIDIA.

Are they really?

For anyone that cares, DGX OS is based on Ubuntu, and there are packages for Red Hat Enterprise Linux as alternative.

https://docs.nvidia.com/base-os/index.html

price? 8-}

Certainly doesn't look as impressive as Tesla's Exapod housing. How does the first principles engineering rate comparing Tesla's Exapod to this?

Well if you want to compare them, Nvidia's has over 10x the memory and has historically sold > 1 units of their HPC products outside their company.

I'm not sure the aesthetic of what are essentially cabinets in a warehouse is that much of a consideration with HPC?

If you are convincing the board to spend that sort of cash you want something photogenic.

Most super computers - at least the ones I’ve visited - have some nice chrome. Their output tends to be a couple of numbers or something equally uninspiring so making them look awesome is key to funding.

Tesla's exapod doesn't and won't exist for the public outside of press releases, for one.

Reply on news.ycombinator.com