Hacker News Reader

Comment by ck2 | Hacker News Reader

You can already get 3% more compression out of zlib by giving it a bigger dictionary like 7zip can optionally do.

code.jquery.com/jquery-1.9.1.js

  original  268381

  gzip       76070      (32KB dictionary, 128 word size)

  Zopfli     75730

340 bytes is not worth a 20x time increase for compression.

View on HN

340 bytes is not worth 20x time increase. 0.4% decrease in file size IS worth the time. If you can measure the amount of time it takes to download those last 340 bytes, then you can start to count how many downloads it takes before its worth it. (If download speed was constant, it would be about 224 downloads).

PNG seems like a good candidate for this style compression. A constant time initial cost for recurring savings in bytes transmitted seems like an excellent use of CPU time to me.

As an aside, I ran the Zopfli program with 10000 iterations on jquery-1.9.1.js:

  268381 jquery-1.9.1.js
   75622 jquery-1.9.1.js.gz

"A constant time initial cost for recurring savings in bytes transmitted seems like an excellent use of CPU time to me."

Wish more people understood this aspect of web scale! well said :)

A fairer (in my opinion) comparison, with minimised jquery:

    92629 jquery-1.9.1.min.js
    32666 (35.26%) gzip -9
    31691 (34.22%) zopfli with standard compress
    31688 (34.20%) zopfli with max compress

     31783 gzip with 32KB dictionary and 128 word size
     31764 above with filename stripped

I am too lazy to strip out the filename so there is a few bytes penalty, so it's more like 31764

So in this case it's

    31688 
      vs
    31764

76 bytes

I guess the real question is what is the decode penalty time if any?

It is if you serve the file a hundred billion times.

If that's your logic and proprietary compression engines are okay despite not being in any browsers, then 7zip ultra can get it to 69531 bytes? That's 5000+ bytes lower than Zopfli

bzip2 can get it to 65536 bytes which is a whopping 10k smaller than zopfli

Oh wait, I see that zopfli is deflate compatible so that means browsers can still decode it. Fascinating.

I wonder if there's any chance of browsers supported xz? Maybe?

I was just talking about this with some other Firefox developers today, actually. So...yes, there's a chance. Not a very big one, mind you. But a guy can dream!

There is a compression-time, decompression-time trade-off whenever you use compression. Xz (or LZMA) is interesting in that its decode time is quite small compared to its encode time, unlike others like bz2. For large, static content the trade-off is quite good. However, with smaller files, like css and javscript sources, you really need to do some benchmarking to be sure.

On-demand xz will probably never be worth it however, the encode time just takes too long. You would have physically transferred the data by the time it has finished compressing.

Reply on news.ycombinator.com