Hacker News Reader

Comment by kevingadd | Hacker News Reader

For reference since it's kind of buried/obfuscated: Their point of comparison is between an NVIDIA Tesla V100 and a 44-core CPU. The latter is probably something like a Xeon E5-2699, which has a list price of $4115 USD. Hard to find accurate pricing data for the V100, but it looks like it was in the $7-10k USD range back in 2018. Still a cool cost/performance improvement but not as massive as I was expecting before I looked up the test hardware.

View on HN

It's not just that the CPU is cheaper than the GPU. They also claim it's faster than the more expensive GPU. Meaning if you rent a box in the cloud for training it'll cost you less since you'll rent it for less time, and if you buy the hardware you can train more models and thus need less hardware to train as much as with GPUs before.

So if it's 3.5 times faster then you require only roughly 30% of the time (or hardware) it took before, both savings combined seem pretty significant to me.

Edit: quick napkin math, let's take your $7k figure for the GPU and 4115 for the CPU. We need one third of that CPU so 4115 * 0.3 = 1234.5 now 1234.5 / 7000 = approx. 0.176.

18% of what it cost before...

That's fair, though it's much easier and cheaper to rent CPU time because of the overwhelmingly larger amount of supply.

> Their point of comparison is between an NVIDIA Tesla V100 and a 44-core CPU.

Not quite. Further in the paper: "Similarly, for larger Amazon dataset, the red and black line never intersect, and the red and blue line intersects on 8 cores. This means that SLIDE beats Tensorﬂow-GPU with as few as 8 CPU cores and Tensorﬂow-CPU with as few as 2 CPU cores."

Yes, they have a beast machine, but it can outperform the GPU with only 8 cores in some cases.

Also worth noting that that Xeon processor takes somewhere around 350-400W and the V100 is also in the range of 300W. So not huge on energy savings. Although a really cool, potentially industry shaking breakthrough, regardless.

The V100 would need to run for 3.5 times as long, so would use way more energy. E.g. training for 1 hour on CPU: 400W×1h = 400Wh of energy, vs training for 3.5h on GPU: 300W×3.5h = 1050Wh of energy.

Why is this downvoted? This is absolutely right.

I realize TDP is a bit gamed, but Intel shows the E5-2699A as 145W. Curious where you got the 350-400W for the CPU.

Nvidea lists 300W as the TDP for the V100.

Intel TDP is not "gamed", but total bullshit: https://www.tomshardware.com/reviews/intel-xeon-e5-2600-v4-b...

Since the article doesn't specify how power draw was measured, I assume that they measured _total system_ power consumption. This is not to be confused with the CPU power requirements as it includes the entire platform and all losses (e.g. power supply, main board, peripherals, cooling, ...).

The new AMD EPYC 7742 should have double the memory bandwidth (they claim they are memory bound) and 1.5x the cores for a similar list price.

But in all cases, I would not expect that a serious research can outright beat the current NNs by 10x on all dimensions. This will take some time (and may not fully happen) and this paper is certainly a great advance.

Maybe the real benefit here is that due to the asynchronous nature of the algorithm, you could easily scale to more cores and the new approach will scale with it, vs. at some point, you reach a point of diminishing returns with the GPU?

You can get an Intel 10980xe with 18 cores/36 threads for $1030USD which also supports AVX512 and DL-Boost.

https://ark.intel.com/content/www/us/en/ark/products/198017/...

Availability is pretty much non-existent though. Also you can OC it pretty close to 5GHz with decent water cooling.

Reply on news.ycombinator.com