Hacker News Reader

Comment by bruce_one | Hacker News Reader

I always find [lrzip](https://github.com/ckolivas/lrzip) is under appreciated when it comes to compression discussions; it doesn't suit all circumstances, but it works really well in the ones it does (we're using it with the nocompress flag and then using zstd, hence why it comes to mind :-) )

Edit: it's not well suited to real-time...

View on HN

If you're using lrzip, you should also check out zstd long range mode [0]. It uses a long window (128 MB by default, up to 2 GB), together with an efficient search strategy, and multithreading. For example, a 2 GB window, with 4 threads, at level 10:

    zstd --long=31 -T4 -10

It should be faster than lrzip + zstd and provide about the same results.

[0] https://github.com/facebook/zstd/releases/tag/v1.3.2

Reply on news.ycombinator.com