JPEG-XL vs. AVIF and Others: 27 Images Compared
giannirosato.com
giannirosato.com
Nonetheless, there's no question that JPEG XL was sabotaged. AVIF was enabled by default in Chrome the day it was implemented, JXL had to go through an experimental period. At some point, Mozilla developers were told to stop working on improving their JXL implementation and they couldn't even merge already developed patches.[1]
Google's justifications for the removal were baffling. They said there's "not enough interest" despite all the hype, all the major companies requesting JXL support.[2] They claimed JXL doesn't bring "sufficient benefits", despite the numerous advantages, such as lossless JPEG recompression. They published their flawed[3] benchmarks weeks after they made the decision to remove JPEG XL.
BTW, the removal happened immediately after Adobe announced they are supporting JPEG XL in one of their products. Almost as if they realized that JXL may be gaining too much support and it's time to pull the plug.
So who's behind this if not Google? To me, the most likely suspect is AOM, the Alliance for Open Media. The people who are involved in AOM also the ones who make the decisions in the browsers. What are their motives? I don't know, pettiness? Maybe they just want their format to win.
[1] - https://phabricator.services.mozilla.com/D119700#3977128
[2] - https://bugs.chromium.org/p/chromium/issues/detail?id=117805...
[3] - https://cloudinary.com/blog/contemplating-codec-comparisons
If you look at https://github.com/libjxl/libjxl/graphs/contributors, ~everybody except Jon Sneyers is a Google employee (I think the AVIF ratio is similar, but I didn't check).
So the question isn't about which format is better, but who won internal politics and it's clearly the AVIF folks - for reasons which aren't really possible to tell from the outside.
Apple LOVES proprietary standards like Lightning despite harming UX because of their "fuck you pay me" attitude towards their customers.
I advise against glorifying a rich asshole who died from cancer because they thought they could cure it with fruit juice.
Browsers are already absurdly complex, and are constantly receiving HTML, CSS and JS features that significantly increase complexity. Are we really to believe that adding support for a certain image format, which is basically as modular as it gets (decode the file into a buffer), is too much complexity?
Obviously, this is a case of them showing favoritism toward codecs based on the IP owners of those codecs.
There has been enough 3rd party interest to create multiple AV1/AVIF implementations. JPEG-XL has its original team (doing great work), and then… commenters lamenting, but nobody else invested enough to actually to write the code.
Browsers' goal is not just to have every benchmark-winning format. They also need to minimize attack surface, risk of bug-compatibility (which is why one and only implementation is not sufficient), and watch out for long-term costs of code size, maintenance, and technical debt. New codecs can be added at any time, but can never be removed.
AV1 got in, because video really needed an answer to the looming threat of patented expensive H.265. There was a risk of repeating the pains of H.264 support, and this time without Cisco's licensing loophole. Then AVIF got in, because AV1 was already in. It's that simple.
What do you mean by "the attempt"? There are at least three independent decoder implementations:
jxl-oxide (Rust): https://github.com/tirr-c/jxl-oxide
J40 (C): https://github.com/lifthrasiir/j40
jxlatte (Java): https://github.com/thebombzen/jxlatte
None of them is feature complete, but it's because each of them is a hobby, single-person project. You're making it sound like some billion-dollar company tried to implement JPEG XL and failed, because it's too complex or something.
However, it should be noted that those independent implementations, despite being hobby projects, helped to iron-out some minor issues in the standard.
>Then AVIF got in, because AV1 was already in.
AVIF uses its own container format that had to be implemented. Implementing AV1 does not give you AVIF support for free.
It's cool that there's a Rust implementation in progress. If it works out, it will make it much easier for me to support JXL. I've been waiting to RIIR the j40 instead.
AVIF (HEIF) container format is MP4, which browsers already parse for video. There is a bit of extra work to extract image-specific data, but it's minor compared to AV1 implementation. AVIF even contains a completely redundant copy of key AV1 metadata (av1C) to spare implementations from parsing the AV1 payload.
Which is why they're not considered implementations to meet this bar. You need multiple FULL implementations to ensure that it is actually FULLY viable, otherwise you risk taking on a potentially broken format that you can't remove.
> You're making it sound like some billion-dollar company tried to implement JPEG XL and failed, because it's too complex or something.
I doubt any big company actually tried (or tried very hard). Cost of implementation becomes a kind of implicit test here, because companies like things cheap.
> AVIF uses its own container format that had to be implemented. Implementing AV1 does not give you AVIF support for free.
It gives AVIF support for cheap due to code/algorithm re-use. Companies like things cheap, so that's a big selling point.
So if an individual unpaid volunteer spends their time doing a 2nd implementation, <bigco> might consider it for their browser. But they won't pay for a 2nd implementation with their own staff, even if they own a humongous chunk of the browser market (which comes with responsibilities) and even if it is a better algorithm.
Sounds about right.
We make incremental gains as a human race, not necessarily optimal ones.
The main thing you want to have though, is a standard that cannot be unilaterally changed in arbitrary ways by a single party. However flawed it may be, ISO does have a long history of standardization, and it has procedures in place to ensure that changes are scrutinized and can only be applied when approved by the national standardization bodies.
[1]: https://news.ycombinator.com/item?id=35165640 "Firefox 111.0 enabled Origin private file system access"
[2]: https://news.ycombinator.com/item?id=33374402 "SQLite in the browser with WASM/JS"
I wish Mozilla had gone ahead and enabled it by default in Firefox, as I'd hoped it would. Still salty it didn't happen (yet, at least). It seems like such a clear choice to developers that there's actually people being vocal about it (as evidenced by the sparse but constant stream of e.g. posts on HN about browser image codecs, which is a topic _no one_ finds sexy otherwise), but browser vendors still won't budge. It's weird to see them say "sure it's better, but it's not better enough than the alternatives". There's a large enough discrepancy in my own perception of the merits of jxl and that of the people making choices for browsers that I genuinely wish I understood them, because it feels like I'm missing something.
I still convert my pictures to jxl locally and, and the space savings are enough of a reason to go through the trouble of converting - it just seems like a no-brainer that it'd be even more valuable on the web, as bandwidth savings are generally even more important than disk storage, and then there's still the plethora of other features on top of that.
Even if that bandwidth is “free”, it can be significant enough to prevent access by end users with slow/spotty connections. Depending on the use case, that can have the consequence of also basically obsoleting the site serving those users. Just as some innovations enable whole new use cases, reverting the innovation can have the opposite effect.
I am sorry it is hard for me to not take a jab at this considering their codec is patent free so their should be zero IP owners. /s
Anyway JPEG XL is a miniture project physically. A compressed wasm binary is around 175 kB. I don't think complexity is an issue there. [1]
For example with defensive license termination: they can sue you, but if you try to sue them you have to rip off AVIF from all your products: https://aomedia.org/license/patent-license/
>Defensive Termination. If any Licensee, its Affiliates, or its agents initiates patent litigation or files, maintains, or voluntarily participates in a lawsuit against another entity or any person asserting that any Implementation infringes Necessary Claims, any patent licenses granted under this License directly to the Licensee are immediately terminated as of the date of the initiation of action unless 1) that suit was in response to a corresponding suit regarding an Implementation first brought against an initiating entity, or 2) that suit was brought to enforce the terms of this License (including intervention in a third-party action by a Licensee).
AOM has noble goals: royalty-free codecs for the web. JXL is also a royalty-free codec that is very well suited for the web (as well as many other use cases for still images). There is no reason why we can't just have both supported by browsers.
JPEG XL could be twice as efficient as AVIF everywhere, and every blog on the internet could benchmark it, and I suspect it still wouldn't make a difference.
Give it another couple years and maybe we'll have another format 5-10% better than JXL, at which point I hope everyone starts getting angry that format isn't included in browsers... (actually VVC intra should be at that performance level today, not that it has any chance of becoming the basis of a web image format...)
- slow power hungry encoding
- slow decoding (though this has gotten better very quickly)
- terrible lossless performance
- no lossless path from jpeg like jxl.
- no support for "exotic" formats like > 12 bpc or many extra channels.
Saying avif should be the only format is like saying everyone should depreciate png and just use jpeg. It doesn't cover all use cases, though frankly I am getting sick of slow pngs.
Because it won’t be just the web that uses JPEG. However whoever wins the support of the web will become THE universal image format for the foreseeable future.
> Give it another couple years and maybe we'll have another format 5-10% better than JXL, at which point I hope everyone starts getting angry that format isn't included in browsers...
That’s the thing. Once a format has become THE standard it won’t be replaced for a very long time. Might as well get the best bang for buck and pick the best standard that’s availability now.
Yes. Considering HEIC surpassingly did outperform AV1 and JPEG-XL in most of the benchmarks. You can expect VVC intra to be even better.
>Give it another couple years and maybe we'll have another format 5-10% better than JXL
Well yes. That is on the assumption everyone will settle on AVIF because that 10% is comparing to AV1. That wasn't true until Apple made the move. And AVIF, or AV1 the codec in general likes to remove details. So I would argue it isn't even a great image codec in the first place until it is fixed ( which the AOM has no intention of improving ).
The good thing ( or bad thing ) about JPEG XL is that they have now port some of the improvement to existing JPEG via JPEGLI, which gives ~10-15% improvement depending on image type without breaking compatibilities. This improvement on JPEG alone means the next generation replacement codec needs to be much better for it to be meaningful.
This isn't true if you're talking about encoding size efficiency. See the "Quality" chart at https://storage.googleapis.com/avif-comparison/index.html
Even JPEG XL cheerleader Cloudinary says, "JPEG XL can obtain 10 to 15% better compression than AVIF" overall, which isn't a sufficient incremental benefit. https://cloudinary.com/blog/the-case-for-jpeg-xl
Not to mention JPEG XL can do lossless and progressive on top, and that avifenc is much slower than cjxl.
Also annoying was the fact I actually had to debug libavif to work out what the problem was, as `avifEncoderAddImage()` would just return a generic error that wasn't helpful when the dimensions were too big, but `avifRGBImageAllocatePixels()` and `avifImageRGBToYUV()` didn't complain about the dimensions beforehand.
It's also disappointing for a video codec released in 2018. For context, in 2019 Sony released an 16k screen. Granted, that was a home cinema screen, and 16k displays will be a niche product for the forseeable future, but it's also not something unimaginably large.
And of course that's assuming that the viewer is static. If you assume that viewers can move to a point where the screen is bigger than their field of view (say a large display in a museum) or that the viewer can zoom the video (just as we routinely zoom images today) there are even fewer limits to reasonable resolutions.
If you had a "human sized" display (let's say 2m x 2m), that you could stand next to, and it showed footage of a real size analog caliper that you had to read, I wouldn't be surprised if you needed something like 128K video or more.
But I agree that for regular video content, 8K is good enough.
Sure if you had pinhole irises to look through instead of blobs of organic tissue.
I don't know about the libavif API, but certainly libavif's avifenc will happily create large files that don't conform to the AVIF advanced profile.
I couldn't even get it to save at the size of 8192 x 5464, which is the R5's full resolution.
I couldn't even get it to save 8192 x 5464 size files which is the native format of the R5 camera, that was too wide.
It seems to be a limitation with the libavif API or maybe the codec.
Most of my panoramas are horizontal (15k x 4k), but a few are vertical.
I still have to occasionally convert some images back to jpeg, but the savings from storing all of them in jpeg-xl instead is worth it to me (and avif support is spotty in the same areas, it is only really ahead in browsers and photoshop)
The progressive enhancement feature alone could be quite amazing, and its competitors don’t offer that- No more thumbnail generation! Its competitors can’t offer that! Do they realize how many security holes everyone’s use of “ImageTragick” generates, and how much caching infrastructure is devoted to images?
also, how does he fail to see this as a potential competitive advantage over browser competitors?
Can you expound on this?
No less than 630 vulnerabilities, and almost everyone uses this library, although VIPS is gaining tiny marketshare.
still not great, but this give some context to understanding the number.
This does not happen for JPEG.
>The number of transferred bytes is important because it affects the latency and hence the user experience. Using this estimate, we calculated that between June 1st and 28th, 2019, the average BPP (weighted by the number of bytes transmitted with that BPP) was 2.97, 2.24 and 2.03 for small, medium and large images, respectively. We confirmed that there was no considerable difference between mobile and desktop platforms. This data indicates that image compression algorithms should target rich images with a relatively high BPP values, aiming to produce 1 BPP images almost free of artefacts instead of 0.5 BPP images with artefacts that are not too intrusive.
It has been quite well know by now, at least to those who are familiar with JPEG XL that is doesn't do as well in Non-photographic photos. But it is still much better than WebP or JPEG.
I dont know if we could potentially further improve Non-photographic photos, but if you are look at the chart JPEG-XL has a perfect score at below BPP 1.0 already. Compared to AV1 encoder having many years of head start.
[1] https://www.spiedigitallibrary.org/conference-proceedings-of...
I’m not sure that “the Chromium team” is a real thing. Chromium is a cross-company project. People from different companies and different teams work on it together. Google has a Chrome team.
edit: I’m not familiar with the Google JPEG-XL situation, but generally speaking, it should be possible to ship a feature in Chromium without Google’s involvement because there are three non-Google Blink API owners (last time I checked).
They also mention it’s not free. That would be ok. Just say your test is image quality of free image formats.
But that statement just undermined the trust I had in the author up to that point.
I should include that under "Takeaway," I do say that " ... this is a non-scientific test," & it shouldn't be considered objective despite the use of metrics. While I find SSIMULACRA2 correlates very well with what my eyes see with these images & most other images, 27 landscape/architecture images aren't enough to draw any objective conclusions here & I'd need to do more testing in the future to produce research-quality work.
I appreciate what you did. Your article is far above a simple “I looked at five pictures and decided this one was the best“ kind of comparison that you sometimes see. You clearly tried to do a good and fair job with your contenders.
The results were interesting to see. This isn’t an area I spent time in so I really didn’t know what to expect going in other than thinking there must be something better than the venerable JPEG.
Why everyone is all up in arms about the fact that the industry is finally standardizing on a good royalty-free image format is beyond me.
If we were picking a single royalty-free image codec for the 2018-2026 range, it should be JXL.
Image encoding is a subset of video encoding, so any limits are mostly arbitrary, and in AV1's case image-only encoding was a consideration from the start. AVIF in particular supports color depths up to 12 bits, wide color gamuts, many color spaces and standards for color space signaling, etc., so I'm curious what aspect you consider limited.
This isn't true. There are a number of things that make sense for images but not videos. 1. Progressive rendering. 2. Big images (up to 2^30 by 2^30) 3. More channels (up to 4099) (this can be useful for transparency, depth, or tracking random other things for scientific purposes) 5. High bit depth (up to 32 bit) (this is mostly useful for sciency stuff) 4. Lossless encoding. Basically no one wants lossless video (it's way too big), but lossless images often make sense.
12-bit is fine for delivery, but it's not enough for authoring. Cameras are typically 14-bit and during authoring you'll want some footroom on top of that.
Also avif is not usable for lossless: it is slower than png and often worse. Lossless is crucial for authoring workflows.
For high-fidelity lossy compression, avif can be worse than even jpeg.
Avif cannot do CMYK at all, so for printing use cases it is not usable.
Basically it is targeted at web delivery, but it's not really a general-purpose image format that can be applied in a broad range of use cases.
Forget lossless originals, this is about the average images that users see online, like 1000x1000 (or smaller) and 200kb (or so).
As the head of tech for a site with a few tb of user-supplied jpeg files (which are thumbnailed), I can see the appeal…
As Mozilla put it, adopting a new format increase the attack surface ( that is already quite large ) of a browser, they can not simply take the reference decoder and integrate it into their codebase, they also have to use resources toward maintaining it. Firefox currently integrate jxl support behind an experimental flag, but that could mean the feature is in testing and could be deprecated in order to better allocate their resources.
Why can’t they take the reference decoder?
As for the attack surface, why would their own codes be less vulnerable?
They don’t tend to support each other’s codecs either, not because of attack surface
Well, Chrome doesn't support bunch of things, and the world didn't end. The usual topic is, that is supports too many things actually, more than it should.
The bugreport even states: "There is not enough interest from the entire ecosystem to continue experimenting with JPEG XL". Not sure on how Google "measured" that.
Having developers decide they won't invest into your toy format forever is not FUD.
"uncertainty, doubt": "not enough interest from the entire ecosystem to continue experimenting"
"fear": "By removing the flag and the code in M110, it reduces the maintenance burden and allows us to focus on improving existing formats in Chrome"
The news about it being rejected increased JXL's profile tenfold and there's a lot more interest now.
(Not to mention the fact that the developer tooling for H2 Push was terrible to say the least for a long time)
To me it seems Chromium/Google are somewhat out of touch on how slow the web and adoption of new things happens.
In this case, it was also behind an experimental flag, I don't know how the f** they expect people to show interest if it's practically not usable on the web.
When you add a jpeg-xl.library, every app on the system instantly supports loading and saving that image format.
You could end up with sites using some Windows-only formats (e.g. some internal MS Office format), iPhone-centric sites would serve patented HEIC, and then browsers on other platforms would be forced to adopt these formats.
Not every plugin dumped in the local OS is as hardened as web-facing codecs. People would likely get pwned through some legacy fax codec from their scanner software.
Users seeing broken images is not a good UX, and users installing random code-running plugins, because websites tell them to, is even more annoying and dangerous.
We've got decades of experience, and scars in the User-Agent header, showing that developers can't be trusted to make sensible technical choices.
In a well-designed datatype system, an image datatype would, for decoding take a compressed image and return a decompressed image. Possibly with streaming support.
It would not be able to do anything else, not having capabilities to anything else than necessary.