Hacker News Reader

Comment by janwas | Hacker News Reader

That's a bold claim. What are your sources? The two latest things I worked on, JPEG XL and quicksort, see a 1.4 and 1.6x speedup from AVX-512. That's on SKX and includes the much-maligned throttling. On a system level, I doubt moving those to GPU is helpful.

View on HN

Also, moving things to the GPU may be good for throughput but bad for latency depending on the workload, since offloading to GPU has a cost and data exchanges too.

> offloading to GPU has a cost and data exchanges too.

This is bad with dGPUs over the PCIe bus, but not so much with GPUs that share a very fast memory bus with the CPU. In this case, the layout of the data may prove challenging to keep the same for when you use a CPU and a GPU.

Agree. Real-time signal processing with feedback loops is a case in point.

If you're processing a lot of data, you're better off moving it to the GPU. If you're processing only a little data, the speed-up doesn't matter. I wonder how wide is the Goldilocks Zone where AVX512 makes a practical difference?

Golly, another sweeping statement :) It seems to me a GPU might actually sort more slowly.

For 64-bit keys, we sort about 1 GB/s per (5 year old) Skylake core, and perhaps 5-6 parallel.

This (2018) reports 3.5 GB/s: https://benkarsin.files.wordpress.com/2018/10/dissertation.p... And a 6-year old GPU radix sort reports 2.1 GB/s: https://github.com/Bulat-Ziganshin/Compression-Research/tree...

BTW I've worked on a product that used GPUs. That typically requires everything to move to the GPU, which is not always desirable or feasible.

This (2020) reports "Despite the fact that we send the entire data array to the video card and back, sorting on GPU of 800 MB of data is performed about 25-fold faster than on the processor."

https://dev.to/tishden/computing-with-gpu-why-when-how-and-s...

This shows a approximately 20x speedup (2021, graph 1 vs 3): https://www.irjet.net/archives/V8/i7/IRJET-V8I7714.pdf

Thanks for the example! Sounds like 1.6 GB/s on an entire Tesla K80 (300W TDP). This is in fact several times slower than our results on Skylake (with half the TDP), but note that K80 is from 2014.

The "25-fold speedup", as is often the case for such reports, comes from not optimizing the CPU side.

Assuming AVX-512 actually works well (haven't had the opportunity to use it myself), it could be very useful in high-end gamedev. Data-oriented programming is popular there, which makes it much easier to optimize data transformations w/ SIMD. Good compilers can even do some auto-vectorization (it's a nice boost with no additional programmer effort but you can't rely on it much). GPUs tend to be already fully loaded by the intensive rendering workload, so there is a large incentive to squeeze as much as you can out of the CPU.

gpus generally aren't worth it for moderately cheap O(n) operations. avx 512 is really nice because if you match the memory speed, you beat GPU (since the cost of GPU will also be memory speed constrained).

Reply on news.ycombinator.com