Hacker News Reader

Comment by throwaway84742 | Hacker News Reader

Sheeeit. I’d love some of the stuff they’re smoking. You can build a 100 GPU rig for half as much. Just scatter it around the office so it’s not “in the data center”.

View on HN

Well, those who were willing to pay the ~$150k for the DGX-1 (Volta-16Gb upgraded IIRC), won't necessarily find it too much -- and NVIDIA is after quick money before the markets start slipping down the hyper curve.

You also do get quite some meat compared to the DGX-1: the equivalent of 2x DGX-1 in terms of GPUs and NICs with 4x HBM2 size, plus the NVlink fabric. Plus a more SSD storage, Xeon Platinums to round it off.

Oh, and 350 lbs, let's not forget. :D

Expensive, yes! Will it sell? I'd be very surprised if it didn't!

100 GPU rig would not have 512 GB in one address space, accessible by 16 GPUs. Each GPU can directly address the memory on any of the 16 GPUs.

For the 300k you'll save just hire a full time dev to write you a distributed implementation of your solver.

Hahha, good one. Just that by the time that one dev implements everything, you'll be overtaken by the competition -- at least that's what everyone's fear is, and it's partly warranted.

Many problems are bandwidth-limited. More cores may indeed actually hurt performance as contention for the limiting resource makes it increasingly hard for any core to run efficiently.

https://superuser.com/questions/2489/cpu-cores-the-more-the-...

You can't spend your way out of the latency issues associated with such a distributed architecture, though.

That's one of things NVLink helps with, actually.

100 GPU rig of 1080ti’s would have 1.1TB VRAM overall though. And most jobs don’t scale well beyond 4 GPUs anyway.

That is problem DGX-2 interconnect solves.

Not for currently available deep learning frameworks.

You're right but only because DGX-2 is not shipping yet. ;P

Not only that, but the switch has the exact same throughput as the actual memory of the V100 until you get to fairly large block sizes. It was shown in a presentation.

100 Titan Vs = $300k. And the big deal about this system is the NvSwitch which allows the GPUs to appear to software as a single huge GPU.

So you can run a single neural net model that can use > 12gb memory?

You can already do that with Pascal since it supports unified memory you have an address space of 49bits.

Now it’s even faster since you won’t have to fetch pages from system memory or worse.

But this isn’t new NVLINK supported memory unification of this sort since it’s introduciton but never in this scale.

I’m pretty sure that if their numbers to be believed they have just made the worlds “fastest” single switched fabric.

And what are those numbers for fabric speed?..

900GB/s (GByte) bidirectional.

Oh, that's very impressive!

Do you think they actually mean 900Gb/s (Gbit) here? 18 ports of todays ~50Gb/s serdes would give 900Gb/s. And that'd be twice the bandwidth of their Nvlink 2.0 stuff (~25Gb/s), which would seem a reasonable evolution.

Edit: Turns out that a single Nvlink 3.0 port is 8-bits wide and since 144 50Gb/s serdes on a (big) chip is perfectly doable - 900GB/s must be correct.

The article notes it's a 2B transistor switch chip... which boggles the mind.

PCIe 3.0 x16 already is capable of more than 100gbps per port.

900gbits total bandwidth for 18 NVLink ports would be abysmally slow.

You didn't work back in the day when high end products where if you had to ask how much you couldn't afford it.

In 1980 or so my office mate was working on a way to measure the efficiency of different toilet designs - to try and save water.

I looked at using the then very new use of image recognition hardware (and I think neural nets) but sadly realised that with a base price of hardware at £250,000 around a million in todays money it wasn't practical.

Which was a pity as the organisation was into ml/ai back then and even hired one of the first knowledge engineers (as they wher called back then).

Reply on news.ycombinator.com