I bet switching to LZMA would have saved even more. LZMA beats Brotli nearly every time. zStandard would likely have worked better as well. Brotli is very slow to compress.
https://cran.r-project.org/web/packages/brotli/vignettes/bro...
I'm sure you can use those results to argue that LZMA is superior in some way (e.g. compression speed) but it definitely isn't clear cut superior in other important ways (compressed size and decompression speed are inferior).
I can see why, given those results, that they would use Brotli over LZMA.
And also those here: https://www.percona.com/blog/2016/03/09/evaluating-database-...
Suggest that LZMA compresses better than Brotli except in the case of text documents.
This [1] states that a LZMA has a decompression speed of 70 MB/s, which is about 0.7 seconds. The 334 MB/s speed of Brotli does the same in about 0.15 seconds. So the additional overhead of LZMA (compared to Brotli) in decompression is just 0.65 seconds.
Given these orders of magnitudes, I think optimizing for compression ratio is a much better option than decompression speed.
[1] https://cran.r-project.org/web/packages/brotli/vignettes/bro...
I am concerned that the Brotli v LZMA v GZip paper you cite is not fully representative as it is written by the the Brotli team.
Brotli: 4.77MB -> 1.21MB (8.5s) xz: 4.77MB -> 1.129MB (1.4s)
LZMA does better than Brotli in the case of binaries by a fair bit every time.
The thing that makes Brotli attractive though, is that it has high compression (again, very close to LZMA, sometimes even better) while decompressing MUCH faster than LZMA.
The big downside is that it is very slow in compressing, which makes it mainly suitable for 'compress once, decompress MANY times' type data.