Hacker News Reader

zstd looks good but it's only three years old, so use cases need to fit in the lifetime of today's platform. gzip is about to turn 27 and everyone has heard of it, which makes me pretty confident in finding decoders after another decade or two.

(Actually this problem is why I haven't owned a tape drive for a while.)

Sure. Anything, not like text compression is hard.

pigz!

Zstd also has a multithreaded mode. On the CLI `zstd -T0` tells zstd to use all available cores.

Please be aware that this will significantly impact compression ratio. So much actually, that I don't dare use it by default. I'd often use the non-multithreaded mode as for some data (perf.data files with large stack captures, most recently) and reasonable compression speeds of >=50MB/s input multi-threaded modes are not pareto-optimal.

I'll get a small benchmark script for that done over lunch. Come back here later.

> Please be aware that this will significantly impact compression ratio.

Is this a continuation from previous message ? Because multithreading mode of zstd doesn't impact its compression ratio. It's actually an interesting property of zstd : whatever the nb of threads used, the compression ratio remains the same (aka. reproducible).