Hacker News Reader

Comment by acdha | Hacker News Reader

This is why I’m more sympathetic to browser developers being slow to adopt new formats. WebP is a marginal advantage over JPEG (mostly transparency) which hasn’t seen much success but now that’s translated into multiple high-priority security holes and we’re all going to be spending the next month deploying patches everywhere which links against libwebp.

That doesn’t mean we shouldn’t do new things but I think as developers we’re prone to underestimate the cost pretty heavily.

View on HN

Firefox has a nice solution which gives an extra layer of sandboxing which can be applied to individual libraries rather than a whole process: https://hacks.mozilla.org/2021/12/webassembly-and-back-again...

Yes - not saying it’s impossible to improve or that we shouldn’t add new things to the web platform, only that it always costs more than we anticipate.

One example: that’s great for Firefox but that helps a format become more common, which increases the likelihood of tools integrating a library but not using a browser-grade sandbox. How many apps end up passing user uploads through something like ImageMagick, for example? (And how many years will it be before the last container image is updated?)

For clarity, you don't really need a "browser-grade sandbox" for this -- the wasm2c route that Firefox/RLBox are using is quite lightweight. The only runtime support necessary is basically malloc/free + a signal handler to recover from OOB accesses (and even the signal handler is optional if you can tolerate a runtime slowdown). It's usually pretty easy to compile and link wasm2c's output with an executable that can speak the C ABI.

(I've contributed to wasm2c and think it's a cool tool.)

Fair - I was really just thinking about the ton of things on my Linux systems which link against libwebp.

To be fair issues in jpeg decoding libraries also have been used in the past as vector for malware payload. While the webp ecosystem is far less mature, I am pretty sure that "older" format handling would also have its fair share of security issues.

But your reasoning is valid, it seems like a few weeks ago, netizen were arguing that jpeg xl should be adopted as fast as possible, and for that to be possible the browser developer "only needed to include the reference decoder code into their codebase" at "very little cost".

> netizen were arguing that jpeg xl should be adopted as fast as possible, and for that to be possible the browser developer "only needed to include the reference decoder code into their codebase" at "very little cost".

Because otherwise AVIF should not have made into the codebase. High-profile C/C++ projects can't prevent all security bugs, but they can make them easier to find and harder to get in. AVIF and JPEG XL roughly have the same impact in this regard (written in C++, uses Google's standard convention, tested and fuzzed regularly, and so on).

Then why is it C/C++? To be allowed into the browser, a new codec ought to be implemented in a memory-safe language.

Because browser vendors have already invested too much into the existing C/C++ code base. They can thus accept new code with the same degree of coding standard.

Those same vendors are using Rust and Swift now which have comparable performance and solid interoperability. It seems like time for a policy saying new internet-facing code be implemented in a safe language.

Browser vendors are ahead of you on this. They just need time to put it into practice.

Yes - I know both Mozilla and Chrome are doing this. The main thing I was thinking is that it’d be a lot more supportable if the response was, say, “we’ll add JPEG-XL only in Rust”.

> High-profile C/C++ projects can't prevent all security bugs, but they can make them easier to find and harder to get in. AVIF and JPEG XL roughly have the same impact in this regard (written in C++, uses Google's standard convention, tested and fuzzed regularly, and so on).

Isn’t all of that true of libwebp? I’m sympathetic to the argument that it’s a lot of work to replace C but I’ve been hearing that C/C++ will be safer with enough diligence and better tools since the 90s and it hasn’t happened yet.

Correct, hence "[they] can't prevent all security bugs". They are written in C/C++ primarily because their primary users (e.g. web browsers) will still use C/C++ anyway, so the interoperability is more important. This is changing thanks to Rust and other languages, but the pace is still much slower than we hope.

AV1 video support was already a baseline since Youtube and other websites really want to use it, so AVIF didn't add any attack surface with the decoder. (this is also unlike WebP vs VP8, since WebP added a lossless mode which is quite literally the part this vuln is in)

HEIF container parsing was the additional attack surface added by AVIF, and while it's probably more complex than JPEG-XL's container alone, it's definitely less complex than a full JPEG-XL decoder.

To be clear, I’m not saying we shouldn’t add new formats - more that it’s one of those “take the developer’s estimates for how long it’ll take and double them, and then put a zero on the end” situations. I’m not sure WebP was worth it but AVIF definitely is and both of them do have one advantage over other formats if they can share code with VP8 and AV1, respectively, since browsers were definitely going to ship those.

What I wonder is whether this is saying there should be two tiers, where stuff like JPEG XL might be implemented in WASM for better sandboxing so browsers could more easily add support with less security risk and then possibly “promote” it to a core format if it proves successful and there’s enough performance win.

Except no one will want to ship an encoder/decoder for performance (network & CPU) as well as requiring JS. It would take of images to break even on the cost of shipping the WASM—so these things would never get adopted.

But would have been these bugs discovered if the browsers had not adopted the new format? There are many other image formats and their libraries which are most likely riddled with bugs, but nobody cares because they are not used by major software (certainly not the people capable of finding and exploiting these bugs because of poor time reward ratio). Being unused for longer wouldn't make them any less buggy.

I can confirm that. I talked about it in my last blog post.

This vulnerability was found in Apple's ImageIO library as used in their iMessage application on iOS...

https://citizenlab.ca/2023/09/blastpass-nso-group-iphone-zer...

Found to be part of a Pegasus spyware installation exploit. This is from the company that sells spyware to dictators so that they can arrest & murder people critical of them…

The security holes are caused by C/C++ being unsafe language to develop with, not new image formats. If image, and other, encoders and decoders do not use unsafe languages, it’s unlikely they introduce any such bugs.

That would definitely help, but it doesn’t eliminate the problem entirely (consider for example the attacks on hardware accelerators). I do think that’d be a good policy: new codecs have to be written in Rust or run in WASM.

It is a good idea. When WebAssembly becomes more widespread, it is likely we can drop fixed encoders altogether. Though at this stage the performance impact might be still too much to warrant anything like this.

So every website would ship with its own codecs? There aren’t shared caches across domains. Why would anyone choose that much latency+badwidth+compute (which new require JS enabled) to use a new codec over the native ones? Certainly not the small–medium sites.

Because of the total saved bandwidth. Codec binaries are few hundred kb at most, while high quality images are dozens of megabytes.

My JPEG XL decoder uses WASM: https://github.com/niutech/jxl.js

Nice! I did a quick proof of concept for JPEG 2000 a few years back but it really needed the sophistication of your multithreading and SIMD work.

It's also a far more complex format than JPEG, and complexity almost directly correlates with defect density.

That said, I also blame the culture of making code more complex than necessary, along with developers who barely understand the details.

So that slowness has bought nothing? You have bad format with exploits in the wild while using that as an excuse to gate great format. (which the browsers could "slowly" RIIR instead to address the safety issue directly)

I don’t follow your argument - can you clarify?

Slow browser devs didn't stop this marginally better format from being added with CVEs such as this, but it does prevent significantly better formats from being added, so being slow isn't the way to go, the safety issues should be addressed with safety tools like memory safe languages and other wuffs instead of slowing down the progress

First, I would submit that possibly people have learned from this - WebP is 12 years old and both the level of craft and tools have improved since then. If you suggested Rust you would have been proposing far more of a gamble than now.

Second, I don’t think this is preventing better formats due to safety conservatism - AVIF is at roughly 83% and climbing – but it does support the argument that the Chrome team has too much unilateral control over the web. I don’t think their opposition to JPEG-XL is solely based on the support cost.

Agree. This is one of the reasons it's better to go with older and more reliable JPEG for viewport streaming. An exploit chain would need to penetrate screen capture images to pass to the client. Browser zero days do occur and this is why it's important to have protection. For added protection people often use browser isolation, which specifically adds another layer of protection against such zero days.

If you're interested in an open source (AGPL-3.0) solution for zero trust browser isolation check out BrowserBox on GitHub. We are using JPEG (now WebP) for viewport pixels: https://github.com/dosyago/BrowserBoxPro

Technically, we did try using WebP due to its significant bandwidth gains. However, the compute overhead for encoding versus JPEG introduced unacceptable latency into our streaming pipeline, so for now, we're still against it. Security is an additional mark against the newer standard, as good as it is!

“JPEG is more secure than WebP” is the wrong takeaway here.

Is it? OK, what's the right takeaway to you?

Image decoding is complex

Hm. Ok but why is that the right takeaway?

Eh, I think websites adopt webp for their smaller size than anything else.

Yes but you have to pay a LOT in bandwidth for a <10% savings to be worth the cost of supporting an entire extra toolchain and dealing with the support issues (better now but it took a decade not to have “I right-clicked and now I can’t open it” reports from users). Google and Facebook serve that much but most people do not.

For some datacenters, that 10% saving would be worth the effort and could push back costly maintenance to increase egress bandwidth.

And I would argue that beside Facebook, the end user right clicking and saving the image for them to use in an inappropriate manner ( downloading the image is not the issue, using it without permission would cause copyright infringement ) would be an issue for some of the website that are hosting the image.

> For some datacenters, that 10% saving would be worth the effort and could push back costly maintenance to increase egress bandwidth.

No argument - my point was simply that very few sites on the web fall into that category.

> And I would argue that beside Facebook, the end user right clicking and saving the image for them to use in an inappropriate manner

That’s only true for a subset of sites, only to the extent that this wasn’t covered by fair use, and it came up enough that it was a common objection.

From experience, in many cases it's 50% savings when done correctly and considerably makes the app\website faster on large images when you have 20-50 images to load on one page.

Interesting - I’ve never seen that much compared to mozjpeg and other optimized codecs. We also lazy-load large images.

> the cost of supporting an entire extra toolchain and dealing with the support issues

Why I love features like Fastly's Image Optimizer. No extra work on our end but we get the bandwidth savings https://www.fastly.com/products/image-optimization

We use webp internally for storing very small images that are cropped out of larger images (think individual bugs on a big strip). Webp lets us get them small enough we can store the binary image data directly in postgres which was a lovely simplification.

(We evaluated it for storing a bunch of other stuff but didn't find it worth the compatibility and need to transcode problems)

It would be cool if an app could use an user-provided “codecs” for all sort of common (media) things. That way I can determine an acceptable risk myself customized for my circumstances.

Maybe I’ll use an open, safe but feature incomplete webp implementation because I don’t care if three pixes are missing or the colors are not quite correct. Maybe I’ll provide a NULL renderer because I just don’t care.

I know this sounds stupid, but a man can wonder.

You can use WASM for image codecs, like my JPEG XL decoder: https://github.com/niutech/jxl.js

There is also WebCodecs API: https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_A...

Did you know the path for decoding JPEGs on Apple hardware goes through the kernel?

Do you have a reference for this? that seems like a huge security issue with 0 benefit since decoding can be done easily in userland.

Performance

You mean AppleJPEGDriver? Yes, but I'm pretty sure I learned about that from one of your older comments.

it also has lossless capability as well, that's a big one to me, it can replace png and jpeg for 99% of use cases.

Yeah, I'm not saying it has _no_ value but I don't think it's enough so to be compelling - it's widespread enough that we're probably going to be supporting it for the next few decades but the time where it had an advantage will be what, half a decade?

It's much better from my experience.

I mean, transparency is nice but I think it is reasonable to ask how much value a format will add relative to the cost & risk. WebP has already peaked but for compatibility it’ll need to be supported for decades to come.

Reply on news.ycombinator.com