Linux EFI Zboot Abandoning "Compression Library Museum", Focusing on Gzip, ZSTD | Hacker News Reader

Linux EFI Zboot Abandoning "Compression Library Museum", Focusing on Gzip, ZSTD

phoronix.com

70 pointsrbanffy19 comments

The article gives reasons, but it sounds to me like gzip was retained mainly because it is in use. I don’t have a horse in this race, but if only Zstd were left I would feel like even more simplification was accomplished.

Everybody has their own tiny preference for one of the other but really… if it’s not embedded what is the impact?

> - GZIP is tried and tested, and is still one of the fastest at decompression time, although the compression ratio is not very high; moreover, Fedora is already shipping EFI zboot kernels for arm64 that use GZIP, and QEMU implements direct support for it when booting a kernel without firmware loaded;

> - ZSTD has a very high compression ratio (although not the highest), and is almost as fast as GZIP at decompression time.

> but it sounds to me like gzip was retained mainly because it is in use.

Isn't that what the article is literally saying?

They aren't trying to get rid of badly performing compression algos, just ones that are effectively unused.

Yeah I generally agree that leaving just zstd would simplify things, but I also see advantages in leaving 2. It seems plausible that a better thing will come along in the future. It’s nice to leave the abstraction in place that enables having more than one algo in place.

> - ZSTD has a very high compression ratio (although not the highest), and is almost as fast as GZIP at decompression time.

If you're using zstd and its decompression speed isn't at least twice as fast as gzip's, you're using it wrong.

Yes, everyone tunes their compression settings. /s

It's a generalized statement of 'good enough'.

Even untuned, for every single workload I’ve tried I see more than 2x improvement in speed. Usually 5x to 10x

This has been my experience too, I didn't even know that gzip could be tuned to be faster than zstd.

I doubt it can be, unless your data is too small to matter or you’re doing something wrong.

Not actually true if gzip refers to libdeflate[1] or isa-l[2] instead of zlib. Does seem sus here, though,—I couldn’t quickly find which decompression code is used in zboot, but I’d expect it to be closer to zlib. Perhaps their implementation of Zstandard is simpler than the reference one?

[1] https://github.com/ebiggers/libdeflate

[2] https://github.com/intel/isa-l

zstandard with the correct flags and more cpu/time-intensive options, when creating the compressed file, can be almost as good in ratio as xz.

As far as I can tell, one of zstandard's main benefits is compression/decompression speed. Is the ratio still worse than xz though?

At high levels (19), it gives similar ratios to xz and 7z. But you're correct that the main advantage is the comparatively fast compression and very fast decompression speeds.

Its default compression/decompression speed with no options passed is superior to gzip in almost every possible scenario and cpu type. It does have other options which are much more time intensive for creating an archive but which make it more competitive with bzip2, xz, lz4 and similar.

Long Range Zip tends to give faster decompression. Anyone know which command line switches gives optimum results?

why not keep lz4 as well, shouldn't it have the smallest performance hit compared to those two?

This can't be answered with a simple yes/no without benchmarking this specific use case on specific hardware.

For other use cases I've personally never really found lz4 very useful (neither gzip). Usually zstd on either low/medium can be both faster (or at least, fast enough) and better compressed.

Game developers use LZ4 a lot, like a lot. It's fast on both ends.

Booting only requires decompression which mighty explain why LZ4 doesn't shine there.

lz4 is pretty awesome for ZFS compression. It has nearly no impact on read/write and writing/reading data is sometimes even faster than without compression.

Though I use zstd on zfs datasets which I rarely read/write to i.e. backups.

Reply on news.ycombinator.com