Hacker News Reader

Comment by eknkc | Hacker News Reader

Not a scientific experiment or something like that, just tried it on uncompressed jQuery for fun;

Original file: 268381 bytes

Zopfli (with -i1000) 75730 bytes in 950ms

Gzip (with -9) 79388 bytes in 30ms

It's really slow but the difference seems significant enough. (They have more data here: https://zopfli.googlecode.com/files/Data_compression_using_Z...)

View on HN

Importantly, though, the decompression speed is the same as deflate (e.g, gzip). It's only slower during compression.

As I suspected, XZ performs better at least for jquery. I was disappointed it was missing from benchmarks.

For comparison on my machine:

zopfli -i1000 takes [real, sys]=[1069ms, 21ms], 75730 bytes.

xz -9 takes [148ms, 15ms], 69476 bytes.

(Though as mentioned as a pro of zopfli, xz would lose on decompression speed for a larger file.)

Yeah but the point here is to produce a .gz file which is compatible with standard tools like gunzip, PNG decoders, your browser's handling of "Content-Encoding: gzip", etc.

If you control the compressor and decompressor then applying new tricks to producing a Deflate stream is far less interesting, just use a better compressor.

Can we get browsers to support LZMA? :) (Hey, a bug report! https://bugzilla.mozilla.org/show_bug.cgi?id=366559) But yeah, zopfli does seem like an excellent fit when you can't force a particular decompressor.

and bzip2 gets it down to 65195 bytes.

I've not used xz, I'm sure it's good, but the point isn't really about which compression algorithm is the best in the world, but which works best in the (depressingly limited) real world conditions.

If it is any indicator, current versions of FreeBSD and some Linux distros have "xz" as part of the base system, along with their gzip and bzip2 counterparts, including xzcat/xzless/etc., newsyslog/logrotate compression options for xz, xz-compressed tarballs for source distributions (tar.xz / txz), as well as binary packages for FreeBSD compressed with the format.

LZMA compression has gained a lot of traction over the past few years.

Thanks - I meant to update my comment after I went and looked up xz, I've used 7zip and LZMA before, but I've never directly come across a .xz file, odd!

Tried compressing the minified jQuery, got:

  32653 gzip -9
  31763 7z -tgzip -si -mx=9 -mfb=258 -mpass=16
  31691 zopfli -i1000

IOW, it yields a much smaller benefit as data gets harder to compress. Of course this is not at all surprising.

For my tests I used the default compression levels, so 6 (out of 1-9) for gzip, 6 (out of 1-9) for xz, and 15 (out of 5-1000) for zopfli. My input was the linux kernel 3.8.1 source tarball. This was run on a macbook pro with OS X 10.8.2 and a 2.2 GHZ i7 processor.

  time    size    format
  n/a     987640  uncompressed
  18.44   210896  gzip
  281.44  143128  xz
  3604.21 200640  zopfil

gzip with 32KB dictionary and 128 word size

code.jquery.com/jquery-1.9.1.js

  original  268381

  gzip       76070

  zopfli     75730

Zopfli only has 340 bytes savings and 20x time increase.

Not worth it.

Reply on news.ycombinator.com