Hacker News Reader

Comment by AceJohnny2 | Hacker News Reader

I do not understand why this article focuses so much on encode speed, but for decode, which I believe represents 99% of usage in this web-connected world, give a cursory...

> Decode speed is not really a significant problem on modern computers, but it is interesting to take a quick look at the numbers.

View on HN

Anything more than 100 MB/s is considered "enough" for the internet because at that point your bottleneck is no longer decoding. Most modern compression algorithms are asymmetric, that is, you can spend much more time on compression without significantly affecting the decompression performance, so it is indeed less significant once the base performance is achieved.

When you actually want good latency, using the throughput as a metric is a bit misguided.

As others pointed out, that's why JPEG XL's excellent support for progressive decoding is important. Other formats do not support progressive decoding at all or made it optional, so it cannot be even compared at this point. In the other words, the table can be regarded as an evidence that you can have both progressive decoding and performance at once.

If you don't have progressive decoding, those metrics are essentially the same.

There is no "throughput vs latency" here, there is no "start-up time" for decoding an image that's already in RAM. If a decoder decodes at 100MiB/s, and a picture is 10MiB, it's decoded in 0.1 seconds. If the decoder decodes at 1 MiB/s, the same picture is decoded in 10 seconds.

This isn't true for web use cases. There being able to start decoding from when the first bits arrive vs the last bits can make a difference (which JPEG-XL does a lot better than other image formats because it first sends all the DC coefficients which lets the website display a low resolution version of the image and then fill in the detail as the rest of the image comes through.

That's about progressive decoding, which isn't what this benchmark is focusing on. When benchmarking decode speed, throughput is a perfectly good metric. If the article wanted to talk about progressive decode performance, it would require a complete redesign, not just a change in metric from "throughput" to "latency".

During the design process of pik/jpeg xl I experimented on decode speed as a personal experience to have an opinion about this. I tried a special version of chrome that artificially throttled the image decoding. Once the decoding speed gets into the 20 megapixels per second the feeling coming from the additional speed was difficult to notice. I tried 2, 20 and 200 megapixels per second throttlings. This naturally depends on image sizes and uses too.

There was a much more easy to notice impact from progressive images and even sequential images displayed in a streaming manner during the download. As a rule of thumb, sequential top-to-bottom streaming feels 2x faster as a waiting rendering, and progressive feels 2x faster than sequential streaming.

Decoding speed is important for battery time.

If a new format drains battery twice as fast, users don't want it.

This matters way more for video (where you are decoding 30 images per second continuously) than it does for still images. For still images, the main thing that drains your battery is the display, not the image decoding :)

But in any case, there are no _major_ differences in decoding speed between the various image formats. The difference caused by reducing the transfer size (network activity) and loading time (user looking at a blank screen while the image loads) is more important for battery life than the decoding speed itself. Also the difference between streaming/progressive decoding and non-streaming decoding probably has more impact than the decode speed itself, at least in the common scenario where the image is being loaded over a network.

Agreed. For web use they all decode fast enough. Any time difference might be in progression or streaming decoding, vs. waiting for all the data to arrive before starting to decode.

For image gallery use of camera resolution photographs (12-50 Mpixels) it can be more fun to have 100+ Mpixels/s, even 300 Pixels/s.

> This matters way more for video (where you are decoding 30 images per second continuously) than it does for still images.

OTOH video decoding is highly likely to be hardware accelerated on both laptops and smartphones.

> For still images, the main thing that drains your battery is the display, not the image decoding :)

I wonder if it becomes noticeable on image-heavy sites like tumblr, 500px, etc.

Assuming the websites are using images of appropriate dimensions (that is, not using huge images and relying on browser downscaling, which is a bad practice in any case), you can quite easily do the math. A 1080p screen is about 2 megapixels, a 4K screen is about 8 megapixels. If your images decode at 50 Mpx/s, that's 25 full screens (or 6 full screens at 4K) per second. You need to scroll quite quickly and have a quite good internet connection before decode speed will become a major issue, whether for UX or for battery life. Much more likely, the main issue will be the transfer time of the images.

The parent was talking about battery life.

Mobile phone CPU can switch between different power state very quickly. If the image decoding is fast, it can sleep more

And the one you're replying to is also talking about battery life. The energy needed to display an image for a few seconds is probably higher than the energy needed to decode it.

Aside from all points you raise I find the discussion about battery life a little absurd in light of how negligible it is compared to the impact poorly written JavaScript in the context of web apps. For example, I noticed this morning that my bank somehow pins one CPU thread to 100% usage whenever I have their internet banking site open, even when nothing is being done. AFAIK there is no cryptocurrency nonsense going on, and the UI latency is pretty good too, so my best guess is that their "log out automatically after ten minutes of inactivity" security feature is implemented through constant polling.

I wasn't able to convince myself about that when approaching that question with with back-off-the-envelope calculation, published research and prototypes.

Very few applications are constantly decoding images. Today a single image is often decided in a few milliseconds, but watched 1000x longer. If you 10x or even 100x energy consumption of image decoding, it is still not going to compete with display, radio and video decoding as a battery drain.

Is it practical to use hardware video decoders to decode the image formats derived from video formats, like AVIF/AV1 and HEIC/H264? If so that could be a compelling reason to prefer them over a format like JPEG XL which has to be decoded in software on all of today's hardware. Everything has H264 decode and AV1 decode is steadily becoming a standard feature as well.

No browser bothers with hardware decode of WebP or AVIF even if it is available. It is not worth the trouble for still images. Software decode is fast enough, and can have advantages over hw decode, such as streaming/progressive decoding. So this is not really a big issue.

No, not really - mostly because setup time and concurrent decode limitations of HW decoders across platforms tend so undermine any performance or battery gains from that approach. As far as I know, not even mobile platforms bother with it with native decoders for any format.

In some use cases the company is paying for the encoding, but the client is doing the decoding. As long as the client can decode the handful of images on the page fast enough for the human to not notice, its fine. Meanwhile any percentage improvement for encoding can save real money.

Companies and economies who optimize the cost of clients as if their own can see diffuse benefits from it. Even if they consider 0.01 % of that cost it can lead to better decisions. If that will make the website load faster or allow for more realistic product pictures, it can lead to faster growth or less returns etc.

Real-time encoding is pretty popular, for which encoding speed is pretty important.

Because that’s Cloudinary’s use case… they literally spend millions of dollars encoding images

My server will encode 1,000,000 images itself, but each client will only decode like 10.

That isn't saying much or anything.

But you may have fifty million clients, so the total "CPU hours" spend on decoding will outlast encoding.

But the person encoding is picking the format, not the decoder.

But the server doesn't necessarily have unlimited time to encode those images. Each of those 1 million images needs to be encoded before it can be sent to a client.

Reply on news.ycombinator.com