Hacker News Reader

I'm really interested in GPU-based compression / decompression.

Anyone know what the current SOTA GPU-based algorithms are, and why they haven't taken off?

Brotli has gotten browser support, so it seems to my naive self that a GPU-based algorithm is just waiting take over.

GPU's are good at massively parallel tasks. Compression is, almost by definition, not a parallel problem. If you want speed you can break it up into chunks and if you are optimizing from throughput there are gains to be made there. But if you are optimizing for compression, the more chunks you break the task up into the less opportunity you have to find ways to compress it. For example a fast compression tool creating an archive of files might split up each file into a different thread which gets the job done fast, but it will loose out on huge gains in compression if there are common parts to files that could have been compressed if there were processed as a single blob. GPUs are designed to do lots of small chunks of work in parallel, CPUs are better at doing bigger jobs faster.

Reply on news.ycombinator.com