Hacker News Reader

Comment by waynecochran | Hacker News Reader

This seems fast...

     TF32  ....... 1,000 TFLOPS  (tensor core)
     FP64/FP32 ...    60 TFLOPS

I am more interested in the 144-core Grace CPU Superchip. nVidia is getting into the CPU business...

View on HN

50% sparsity and rated at 700W. The new DGX is 10kW!

I was recently researching how you'd host systems like this in a datacentre and was blown away to find out that you can cool 40kW in a single air cooled rack - this might be old news for many, but it was 2x or 3x what I expected! Glad I'm not paying the electricity bill :)

Here's what a propane heater of similar output looks like: https://www.amazon.com/Dura-Heat-Propane-Forced-Heater/dp/B0...

Most of the propane heater is a fan in a tube, the flame is probably quite smaller than a CPU package.

I've got an 8kW wood stove and that thing gets rather hot to touch - as in, you will get a blister... 40kW is a small city car worth of power.

If you think about it, cars can manage cooling 200+kW with a radiator - and a lot of airflow.

That's still an amazing amount of power though. I can't help thinking about the kind of sear you could get on a steak with 40kW of power :D

Supermicro will sell you 40 kW CPU TDP in a single air-cooled rack, not counting the rest of the servers.

Possible, yes. Easy, no. Don't assume any ole air cooled rack (or datacenter) can manage 40k per rack. It makes it really important to do the hot isle/cold isle well, manage the airflow carefully, block any empty rack units, etc. etc. etc.

Yes, wouldn’t want to try and design it myself! I think the datacentre providers I was looking at spend quite a while doing CFD simulations to get it right!

I think the 1PFLOPS figure for TF32 is with sparsity, which should be called out in the name. Maybe ‘TFS32’? I mainly use dense FP16 so the 1PFLOPS for that looks pretty good.

Asked elsewhere, but why FP16 as opposed to BF16?

I'm using older Turing GPUs BF16 would require Ampere. The weights in my models tend to be normalized so the fraction would be more important than the exponent so I would probably still use FP16. I would need to test it though.

Same - plus its SUPER

Reply on news.ycombinator.com