Studying Lossy Image Compression Efficiency
blog.mozilla.org
blog.mozilla.org
As for the study, I have to admit I did not read it. But the first thing to note is that computed metrics (SSIM, PSNR) are not reliable approximations of perceived quality. Optimizing for either will produce a codec that pleases the computer, not the eye. This is why the tuning of image and video codecs for psychovisual effects is such a laborious, manual process -- you have to have a human viewer to assess the quality.
In the long term I think it would be good if a new still-image format was derived from the intra portion of some reasonably advanced video codec, as they all beat JPEG handily. This is essentially what WebP is, but if I recall correctly, Google didn't really give VP8 the love it needed when they acquired On2. Instead they rushed with a "release early, release often" mentality, presumably hoping the community would jump the boat first and do the grunt work for them later (this was during html5 video wars when some thought Theora sucks and others thought h.264 is too encumbered -- VP8 was Google's resolution). The details are shady but the curious should search for the debate around early VP8 (I think the x264 folk did some extensive notes). Needless to say I think, VP8 (and WebP) wasn't and isn't the game changer Google would've hoped.
To avoid patents, a reasonable choice would IMHO be to look at Xiph.Org's upcoming video codec Daala. Curiously Mozilla has the right man in house since Monty moved from Red Hat to Mozilla like, almost like yesterday... Daala however isn't quite ready yet as the Xiph guys are busy researching out new ideas and unicorns. But the results so far are very interesting to say the least; anyone interested should check out Monty's demo pages.[1]
As others have said, JPEG is "good enough" for most of us, and even JPEG is used rather inefficiently, often with too high quality. So there's no compelling reason to push for a new format here-and-now. Give it some time and commit once there's something significant I say. In the future we'll thank ourselves for not having deployed all that junk over minor increments.
In test of objective metrics SSIM was one of the best (0.8 spearman correlation with human scores) http://www.ponomarenko.info/tid2008.htm
IHMO the study has been done very well. They've even eliminated potential differences coming from colorspace conversion and chroma subsampling. If anything, this study favors WebP that can't do full-resolution color.
However it still doesn't accurately represent preceived quality; time and time again certain psy optimizations for instance are shown to reduce SSIM while perceived quality goes up. Conversely, a codec can optimize for SSIM alone and score well on a test that ranks by SSIM; but the perceived quality could be better. For this reason e.g. x264 has an option to disable psy optimizations that hurt the objective metrics.
There are other things that can hurt metrics a lot while doing nothing perceptible, though these are rather like bugs or errors in testing methodology, and as such would likely be noticed in a well conducted study. See also: https://wiki.xiph.org/Notes_on_testing_theora#General_points...
BTW HEVC-MSP is limited to 4:2:0 only as well.
If you use a greyscale/luma-only metric, you have to encode greyscale from the start, not convert a color image after encoding. Otherwise you're measuring chroma bit cost, but not chroma distortion. Thus, invalid.
Actually, is PSNR-HVS-M greyscale-only as well? In that case, only the RGB-SSIM results would be valid...
And actually, thinking about it even more, converting to greyscale at the tail end just for Y-SSIM and IW-SSIM is fundamentally flawed (if it doesn't result in exactly the same bits as the luma plane), as now you're measuring distortion from an image that the codec was never asked to encode!
They're not aiming to be RAW killers and OpenEXR killers. They're not making anything new possible, they're just polishing the same old thing.
Photographers are increasingly avoiding JPEG because of limited dynamic range it has. WebP (due to its video heritage) has even lower dynamic range than JPEG and can't preserve full resolution of chroma channels.
JPEG-killers add simple alpha channel that 17-year-old PNG has, but nobody even thought about adding alpha with blending modes (e.g additive, to be able to create lights, or per-channel to be able to represent subpixel AA).
After all these years there still isn't a format that can save a Photoshop layer and represent non-trivially blended transparency correctly (closest you can get is images wrapped in SVG with filters, uhh).
There are other ways you can hack around it. One is to save the JPEG at double width and save the alpha channel as a gray scale image on the right, and then use css masks for webkit, canvas for firefox, and CSS filters for IE to add the alpha channel when the page renders. A similar technique can be used to do transparent video: http://jakearchibald.com/scratch/alphavid/
For better results, you actually want to pad the right side of the non-alpha image to a multiple of 8 so that you hit JPEG block boundaries. Then you repeat the edge pixels so that you don't get artifacts at the edges.
Handling artifacts from transparent pixels turning non-transparent is a little trickier, but one strategy is to apply a gaussian blur (radius 8) to the original image, discard the alpha channel, and then fill all completely transparent pixels in the original with the blurred version.
Backstory here, gleaned from eavesdropping on Mozilla employees and Gecko engineers: Google and Facebook are leaning on Mozilla to implement WebP in Firefox, while Mozilla's been resisting due to (in their eyes) the lack of measurable improvement over JPEG. Furthermore, as joosters alludes to elsewhere in this thread, most JPEG libraries set the default compression level way too conservatively, and most users of the libraries don't bother to ever deviate from that default. Merely increasing the compression level from the default can yield a 10% decrease in image size without degrading quality whatsoever as measured by the standard metrics mentioned in the OP.
But there's the rub: the standard metrics kinda suck! Nobody's yet come up with a really solid algorithm to compare the quality of different image formats and compression ratios. So all this research we're doing may very well be useless and misguided. :P
(I'd love it if someone who actually knows what they're talking about can pitch in. Corrections welcomed!)
This highlights a big problem with work on better image compression - we can already do better, but no-one bothers. If you're trying to introduce a whole new codec, you've got a big hurdle to jump.
Agreed, I think the big hurdle is simply that jpeg is 'good enough' for web use, and that a transition to a new standard lossy image format just isn't 'worth the hassle' so to speak.
As for webp, I personally use it for lossless encoding instead of PNG on personal files these days, but I don't really use it for lossy as I don't think it makes a big enough difference versus jpg and jpg is supported everywhere.
That said the current webp implementation is based upon VP8, so I suppose the next implementation will be based upon VP9 which would likely make it competitive with the HEVC based image codec, and unlike the HEVC based one it isn't patent/royalty encumbered.
https://groups.google.com/a/webmproject.org/d/topic/webp-dis...
If you get all modern browsers to agree to support a new codec, but they all render them slightly differently, nobody is going to use it.
Anyway, I have this strange impression that Mozilla is more akin to spent ressources dismissing the format rather than implementing it or finding an alternative.
It's not covered by this study though.
It worked well, for some browsers. Now I expect it would work well on most of them, though I haven't looked at it in years.
It was received with zero interest. That may mean no one needs JPEG with alpha.
You can read more at http://jim.studt.net/jpeg-alpha/
That's not necessarily true. If you only have completely-transparent and completely-opaque pixels color type 2 will work, and if your total unique colors (RGBA) number less than 256 you can still do plaetted.
It's so hard to spread knowledge about image optimization. There are still developers who use (non-animated) GIF images.
Photoshop has still makes JPEG optimization an option :( It probably made sense on 8Mhz Macintosh, but today it takes user longer to tick that checkbox than it takes to optimize a JPEG.
But if you just compare file sizes then pngquant2 wins by a wide margin.
Mozilla is acting responsibly here. It's not just effort to add the format to their browser. The format will spread to entire ecosystem, requiring websites to support it (e.g. can you upload photo or avatar in WebP?), image viewers, image editors, rich text editors, etc. When Facebook implemented Chrome-only WebP users complained, because they couldn't use images anywhere else.
Also take into account that bandwidth doubles about every 5 years[1], so if WebP is giving 20% gain, it only advances the web by 1 year. What WebP makes possible today will be possible anyway next year, and we won't have to spend years re-building interoperability that JPEG has built over 20 years.
WebP isn't just about efficiency over JPEG, the biggest benefit comes from replacing PNG and GIF, which people use for alpha transparency and animation respectively. These are practical, immediate problems that today's web developers face, can they wait a few more years until Xiph approved codec does better?
Yes, because of the IP/patent issues, which do not apply to GIF (nowadays), PNG, and JPEG.
It's highly reminiscent of fight over the ACA in Congress. "Repeal it". Rather than work with making the bill better or fixing the problems, just filibuster.
The bitstream format of VP8 (which WebP is a derivative of) is frozen.
> It's highly reminiscent of fight over the ACA in Congress. "Repeal it". Rather than work with making the bill better or fixing the problems, just filibuster.
Come on. Google is not a legislative body.
What I'm saying is, rather than say "no", be constructive. I see Mozilla saying "No", but I don't see them actually proposing anything that solves the other problems WebP is trying to solve. No studies have been presented on lossless, transparency, or animated encoding efficiency for example.
WebP offers huge immediate savings on PNG/GIF use cases today, and this is especially important on mobile. Developers need these options now, not 2 years from now.
Mozilla supports (and developed) APNG, which has existed for a lot longer than WebP. Google has thus far not implemented APNG (bug here, open since 2008 [1]). You might want to be careful about casually throwing around words like "NIH" when it comes to animated image formats…
The fact is that both browsers are perfectly free to not implement standards that they think are harmful, without having to come up with some better solution right away. Whenever we implement something on the Web, if it gets traction we are stuck with it forever. So we should move carefully.
Don't you think we should be making sure standards are sizable improvements over the status quo instead of just accepting every marginal improvement? If WebP is indeed worse than JPEG, then authors have to choose between better quality (JPEG) and alpha channels/transparency (WebP). That's not a very good decision to force them to make.
[1]: http://code.google.com/p/chromium/issues/detail?id=1171
If I prepare two animated/transparent images of 100 frames today, what is the filesize of APNG vs WebP?
To me that's the real question. My social network streams are filled full of animated GIFs now and they are huge. The G+ mobile client uses WebP for these and the savings are substantial.
I don't know. Those are subjective terms. You can draw your own conclusions from the study…
> The real question is, what if you don't compare WebP to JPEG, but instead, compare it to APNG, then how does the calculus come out?
I don't know off the top of my head. I agree that this should be studied.
What I'm quibbling with is the notion that we should just rush to adopt large proposals like WebP if they're a marginal improvement in any area. Like I said, on the Web we are stuck with mistakes forever if they gain traction. Maybe the story would have been different if WebP had been just the WebP-lossless-animated portion. (I don't call the shots, except in Servo, but I would personally have no issues with adopting WebP-lossless-animated.) But WebP is tied together with the lossy compression that seems to have issues relative to JPEG. "Two steps forward, one step back" is less attractive than "two steps forward".
Google says WebP versus PNG is 28%, so I would expect the results for WebP-animated versus APNG to be similar. IMHO, a 28% improvement is not at the level of "so good we must rush to standardize this now, warts and all".
This whole issue seems sort of like SPDY. HTTP-NG went on for a more than a decade and produced nothing. Google rolled out SPDY, and suddenly we have an HTTP 2.0 spec being discussed. Sometimes the perfect is the enemy of the good, and vendors rolling out implementations forces the community to act and put resources on an issue.
Even if WebP doesn't end up as the ultimate new replacement for JPEG/PNG, the fact that Google, Facebook, and others are rolling it out will likely force some movement to get serious about finding a replacement. Ideally, one doesn't roll out an implementation until there is a consensus spec agreement, but the real world usually doesn't match up the the ideal.
Look at how asm.js's rollout forced V8 to adapt. Even if V8 isn't adopting asm.js, Mozilla shipping it forced the V8 team to spend more time optimizing those paths.
I don't particularly care if WebP is the final image standard or not. What I don't want to see is Mozilla sitting on their hands waiting for a solution to fall into their lap, and putting all their resourcesin Daala.
I expect standards working groups to actually have working members, actively engaged in putting forth positive proposals, not people who just sit around and raise objections. That was one of the problems ARB had for years. If that doesn't happen, then you end up getting steamed rolled by proprietary proposals (e.g. Cg, CUDA, et al)
http://www.reddit.com/r/programming/comments/1ottdn/mozilla_...
I sometimes wonder why something gains traction on reddit and not on hacker news or vice versa.
You have to be significantly better than JPEG to really make a compelling argument as the successor, though. IPv6 is undoubtedly better than IPv4 and the upgrade started over a decade ago.
[1] http://weblogs.mozillazine.org/asa/archives/2009/06/theora_v...