Hacker News Reader

Comment by kristopolous | Hacker News Reader

They also have a turnkey product with 256 of these things.

1 exaflop + 144TB memory

https://nvidianews.nvidia.com/news/nvidia-announces-dgx-gh20...

As someone who lived through the first wave of supercomputers (I worked with Cray Y-MP models), it makes me very happy to see the second wave. For a while I thought supercomputing was dead and we would just be connecting lots of PCs with a network and calling that "supercomputers".

I still remember how my mind was blown when I first learned that all of the memory in a Cray Y-MP was static RAM. Transistor-based flip-flops: extremely power hungry, but also very fast. Another way of looking at it is that all of its RAM was what we call "cache".

This, finally, looks like a supercomputer.

SRAM is so stupid fun to play with.

All of a sudden you don't care so much about the inefficiencies of walking linked lists or trees. When everything is "already in cache", you can worry less about cache efficient algorithms!

1 cycle memory access latency is one of the reasons why tiny embedded MCUs can do things with a fraction of the MHZ of their larger counterparts.

Now days of course it is all about tons of memory, tons of bandwidth, craptons of compute, and planning the flow of data ahead of time.

Only within a very small domain was this ever reasonable.

But that's why there's a "super" in "supercomputers"!

Cray loaded a ton of static memory into their computers, then liquid-cooled the whole thing. Sure, the power requirements were through the roof, and you had a whole huge chiller system which you had to run and hope it doesn't fail. If it did fail, you really wanted to shut the machine down fast. From what I recall there was also an emergency propeller inside the back case of the Y-MP 2E, and yes propeller is a much better name for this thing than a "fan". It would delay the inevitable, although dumping those tens of kW of heat into your server room was not something you ever wanted to do.

The whole point of all this was that you could do things that you couldn't with "normal" computers. That's why those were called "supercomputers". And I'm so glad that after a hiatus of about 30 years we're getting another wave of exceptional machines, which aren't just bigger PCs.

L3 cache is all SRAM but you can have pretty significant delays accessing it. Even the fastest memory cells will build up significant addressing delays as you increase in scale.

If you have a “large” SRAM that can be accessed in one cycle, that just means your processor is slow and/or consumes more energy than it should.

> If you have a “large” SRAM that can be accessed in one cycle, that just means your processor is slow and/or consumes more energy than it should.

MCUs are in this category, lots of embedded stuff, including the two areas I'm familiar with: game controllers and lower spec'd wearables.

Very low power usage, CPU speed around 100mhz, so not too slow.

You can do plenty with 100mhz and SRAM!

> the first wave of supercomputers (I worked with Cray Y-MP models)

The Y-MP came out in 1988, sixteen years after CRI was founded, which itself was several years after the CDC6600.

They presumably where born 20+ years before they started working professionally on the Y-MP. So they could easily have been alive or even a teen in 1964 when the CDC 6600 was released.

I worked with the Y-MP models in the 1990s, and no I was not alive in 1964, although I'm not sure how we got there :-)

I watched Jensen’s announcement for this.

He calls it the worlds largest GPU. It’s just one, giant compute unit.

Unlike super computers, which are highly distributed, Nvidia says this is 140 TERABYTES of UNIFIED MEMORY.

My mind still gets blown just thinking about it. My poor desktop GPU has 4 gigabytes of memory. Heck, it only has 2 terabytes of storage!

Is it similar to the mainframe in concept?

No, kinda the opposite: mainframes use a litany of sophisticated and special purpose hardware to achieve their tasks (eg hardware io _channels_), while this is a massively overgrown instance of a single kind of hardware (vector processor).

The thing that made the lightbulb go off in my head w.r.t. mainframes was understanding that mainframe I/O channels are computers. The mainframe had several dedicated computers that each specifically handled I/O to a terminal, printer, disk or tape drive, punchcard reader, etc. Made I/O programming a breeze, as you just had to tell the channel to read or write, specifying a block of memory to use as a buffer, and the channel would DMA out the data to be written, or DMA in read-in data.

It also explains the reason why despite having middling CPU power, mainframes had a reputation for stinkloads of I/O bandwidth so they could process everyone's credit card transactions, airline bookings, and that: the mainframe's CPU was involved very little in I/O, that was all handled by the channel processors!

It may be presented as seamless unified memory, but it isn’t. Underlying framework still has to figure out how to allocate your data to minimize cross-unit talk. Each unit has independent CPU, GPU and (V)RAM, but units are interconnected via very fast network.

How does this compare to HBM High Bandwidth Memory (and GDDR5)? https://en.wikipedia.org/wiki/High_Bandwidth_Memory

From https://news.ycombinator.com/item?id=36211785 :

> EDIT: found answer to my own question in the datasheet: "The NVIDIA Grace CPU combines 72 Neoverse V2 Armv9 cores with up to 480GB of server-class LPDDR5X memory with ECC."

So, this is not stacked RAM like HBM, it's LPDDR5X which a quick search says is 8.5Gbps.

What's old is new again. This is basically an updated arm version of the itanium based SGI Altix.

Keep in mind unified does not mean uniform, the ram is distributed across all the GPUs.

There’s usecases beyond just ML. Sap Hana could theoretically run on this with greater performance. Same goes for a database. Scaling vertically solves a lot of challenges with distributed ledgers.

I think distributed computing will go away soon. As computers become more powerful, the cost of "distributing" and transferring the data would be more than simply executing everything locally. Yes, you can still split the task amongst different nodes, or give different problems to different nodes, but the use-case would mostly be solving distinct problems on each note, that splitting the same task across multiple computers.

Also, with quantum computers, the parallelization/"distribution" of tasks will be done within the same machine, as it can try all solutions and the same time without having to do divide-et-impera algorithms.

Also, in the future, the algorithms will be a lot simpler, and just have FPGA-s like AI chips, where there is no software, the model is directly modelled in the hardware, so each computation is instant (just the time it takes to propagate the electrons or light through the circuit).

I had to look twice at that image, I thought it was a 2 rack-unit device, But, no, it's 24 full 42U racks!!

it's just a rendering. I presume nvidia wouldn't be announcing something that they haven't made and confirmed, I wonder why they chose that image.

Is it just they haven't done the molding of a production installation? Is it possible that their internal instances might not be that presentable?

> 1 exaflop

To be clear, this is floating point quarter-precision operations when using the FP8 tensor core arithmetic unit [1].

[1] https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-su...

Came here to post exactly the same link. Not just that, but 1 exaflops of sparse FP8.

In comparison, Frontier is 1 exaflops of dense FP64. Try to run this Nvidia system as dense FP64, and it performance will degrade two orders of magnitude.

Don't get me wrong, the machine is really impressive, but the advertisement is quite misleading.

Oh my, time to upgrade my Pi 4 Model B.

If you upgrade to a Jetson you get GPU power and you can keep the form factor, win - win.

I want to see Linus play Doom Eternal on it.

Is Crysis no longer a thing?

Crysis is a problem because it is single threaded. That's why it was so hard on computers back in the day.

Being single-threaded is why the original Crysis is hard on computers now(!!). When Crysis was being developed people still thought 5-10Ghz was coming Any Day Now.

You might could hit 5Ghz by overclocking the appropriate CPU. I remember 10 years ago looking at the liquid cooling setups needed to hit 5Ghz on a core.

Stock turbo on a 13th gen i5 (performance cores) is 5.1ghz. All you need is good cooling and a stable power supply.

My 12400F boosts to 5GHz for the majority of each day. It's literally normal for me. I've gotten it to almost 5.3GHz before, but it's not really stable above 5.2GHz.

2007 was well into Intel Core territory, Intel had given up on clockspeed-at-all-costs Pentium 4 Netburst, so it was generally accepted that clocks weren't going to keep up as fast as before.

Even if Crytek took a year off developing new games after the release of Far Cry, that still puts the start of Crysis development before the announcement of Core (early 2005 IIRC).

That could probably all fit in a single semi-trailer. It's amazing how dense computation is getting.

Doesn't this basically shoot up the list of the TOP500 then? Wonder if they offer a >256 they could be top of the list, easy.

TOP500 uses FP64 performance in ranking. nVidia's 1 exaflop claim is the ~4 petaflops of FP8/INT8 * 256. FP64 performance of modern nvidia gpu's is actually far, far less. The ratio to FP32 isn't even 2:1 anymore (not since Pascal I think) since they realize most machine learning is done with FP32 or less.

64 bit (or 'double precision') is still king in the HPC world though, as it is what you will find in large numerical solutions in fields like computation fluid dynamics, nuclear physics, etc.

Ah fair, I should've known. I suppose the precision is still required for scientific purposes. Thankfully ML stuff now gets more appropriate precision for a speed increase.

Reply on news.ycombinator.com