Hacker News Reader

Have you looked at using LZ4 compression? Supposedly it has a very fast decompressor, so it could be better than uncompressed files for your purposes.

View on HN

I don't think builtin compression is necessary, but it would be better if the CLI supported it similarly to tar.

Compressing a 57.5 MB .hop archive with `gzip -c` returns an 11.7 MB file. This is a similar compression ratio as tar -> tar.gz

Compression is great for sending files over the network, but for fast reads, it might be undesirable

It’s worth benchmarking -- LZ4 can decompress faster than SSD read speeds, so it could well be a net time saving.

How about compressing the files while building the archive instead of compressing the entire archive itself? That ought to preserve reasonably fast reads. It won't give optimal compression but in practice should do alright (and of course will be smaller than no compression).

Reply on news.ycombinator.com