Hacker News Reader

Comment by jeffbee | Hacker News Reader

If you're going for decoding speed wouldn't you be using brotli anyway? At a given compression ratio brotli is ~4x faster on decode path, and they are similar on the encode path.

Your corpus may vary etc etc.

View on HN

Contrary to popular belief (sadly, even among the tech crowd here), people use computers for more than just web stuff.

IIRC, in a recent thread about zstd, someone claimed brotli was faster. Perhaps one needs to look at whether the sample being compressed is web stuff.

If you’re too busy to check yourself, the “squash compression benchmark” page has results for dozens of codecs and different inputs.

Brotli is not just for Web stuff.

Brotli is heavily tuned for "web stuff". It's literally got a built-in dictionary full of commonly used HTML fragments:

https://gist.github.com/klauspost/2900d5ba6f9b65d69c8e

Shoot me, works well for the average desktop (=Electron) application then.

Electron is just web stuff.

Sure but it actually benchmarks well broadly maybe despite its heavy web optimizations.

Sure. In theory. But every other response to this comment are clearly talking about web stuff, and in practice brotli is really only used for web stuff.

I don't know what's theoretical about it. There are Debian packages for brotli and it just behaves like a stdio codec, same as the `xz` command.

Brotli has a static dictionary heavily geared towards web formats, and there really is no reason to use brotli outside of web stuff: the point of brotli is that it has browser support and does 15~20% better than gzip.

It's not bad per-se, but if you're not constrained to the web zstd will have a better compression ratio at every throughput (and better throughput at every compression ratio), and if you want maximum compression period it's not close to competing with LZMA-based formats (xz, lzip).

I think he mean is less popular (like zstandard and lz4) compared to zlib and LZMA and BZip2. this is unfortunate because Brotli and Zstandard compress much better than zLib and LZ4 and decompress much faster than zlib lzma and LZ4

I’m using the phrase “in theory” colloquially, which means something like “sure, you could technically use it that way, but nobody really does/it’s very rare”.

Linux distros often ship xz out of the box. I’ve never seen brotli in the wild. YMMV of course - I am sure there are exceptions.

I’ve used brotli myself, but only as it’s included in some parquet read/write libs.

Probably 10-40% of what your browser is served nowadays is brotli, it's in Chrome so all the major CDNs support it by default

I’m sure you’re right, but files-on-disk is another story. I was referring to the latter.

Jpeg-XL makes use of Brotli, and once it hits 1.0 it will most likely be turned on by default in browsers and thus you will probably see a lot of jxl in the wild, particularly since it provides lossless recompression to existing jpeg files with ~20% smaller size as a result.

It’s definitely in use by Meta and Google.

I'm not the OP, but I took their comment to mean "on the command line".

cloudflare uses brotli (not sure if optional or by default)

I don't know about web use, but xz's major advantage for me is the seekable format. We build disk image pipelines based on remote xz-compressed images where only the parts of the file being accessed need to be downloaded and uncompressed. https://libguestfs.org/nbdkit-xz-filter.1.html

But this seems like a patch given to the LZMA project, with an interesting low-level result to boot.

Reply on news.ycombinator.com