Hacker News Reader

Comment by mugsie | Hacker News Reader

Very few applications can make use of such a large amount of GPU cores - and all of them would need much more CPU cores than this provides to coordinate the work.

Any applications that can would be either run on clusters of servers to really parallelise the work, or is so custom you may as well buy a threadripper and a custom case.

Hell most apps can barely take advantage of multi core CPU.

View on HN

I somewhat disagree. Maybe there are few compiled applications that can use those resource, but it's easy to write code that uses all of them.

I have a 40 core CPU machine with a GTX 1080 Ti GPU. I run deep learning models with 90% GPU utilization and those 40 cores are barely used.

I would love to have 3 more GPUs to run in parallel to test different neural network architectures. Sometimes I'll run a CPU script at the same that processes machine learning data that uses all 40 cores.

I would use 4000 CPU cores and 10 GPUs in a machine if I could get them and I don't even do machine learning full time. I'm personally happy to see this trend of more core counts.

This is my point - I dont think it is as easy as you think to scale from 3.5k cores in a 1080ti to 14k ish cores.

Sure you can write code that nominally use all the cores, but I do not think that the performance increase is going to be linair to the core count.

It's not even down to CPU core count - it would be limited by the speed of a single core.

If you run algorithms from sklearn or just plain numpy it would make great use of the CPU. I was surprised to see the CPU just as useful as the GPU, maybe even more, in my ML projects.

CST will use them for EM simulation, with near linear performance increase for added GPU. I have 4 Teslas in my Microway workstation. Fortunately it also allows for the addition of a decent graphics card, so I can run 3x 4k monitors.

You got that last sentence right: I’m not aware of any apps which can take advantage of even half this many cores. Intel is deficient in terms of memory bandwidth, so most things begin to flatten out at 8 physical cores or so, and take a nosedive when you attempt to use all of them.

While you are right that there are very few apps able to take advantage of quad GPUs, for anyone who does serious number crunching it’s not a problem at all to use all of them using popular deep learning frameworks. They aren’t just for deep learning, you can do a lot more with them. And e.g. PyTorch will automatically parallelize certain things across multiple GPUs for you, with near linear speed up.

Yeah - my use case is kind of specialised, but I could use them all, asking as I can use NUMA to pin memory to physical CPUs.

I wpild be interested to see how pytorch scales GPU wise, relative to CPU clock speed. I am sure there is a point where it will fall off, but not having a 4 GPU system I can't check :)

It’s basically linear as long as CPU can feed the GPU with data. That’s because there’s no cross-GPU communication during the expensive part of the computation (ie forward and backward pass of a neural network). You shove data in, GPU computes gradient, you apply weight updates and redistribute new weights (which are usually not that large). Running the same workload on even the highest end general purpose CPU is glacial compared to even a modestly priced Pascal GPU. That said, there is a limitation: you can’t really have more than 16GB per GPU today, no matter how much you pay, and GPUs can’t do branchy workloads like SQL, compression, etc.

Reply on news.ycombinator.com