Optimizing images with the HTML <picture> tag | Hacker News Reader

Optimizing images with the HTML <picture> tag

jfhr.me

89 pointsjfhr48 comments

> I don’t think there’s a perfect answer. You simply cannot predict with 100% certainty what the browser would choose. But you can get a good approximation by parsing the User-Agent header

Isn't it the purpose of the "Accept" HTTP header? For example the last version of Firefox sends "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8" when fetching a page.

Yup, exactly. Seems like a strange omission. Using the <picture> tag should make that obsolete though, as the browser can chose which image to fetch without involving the backend, maybe that's why?

I've been surprised lately by the number of people that don't know about or understand HTTP headers. One discussion I saw was on a project that was trying to decide what language to serve a webpage in. They spent a good amount of time choosing a geo-location provider and library and then deciding what language to default to for locations that often had multiple languages. I asked why they didn't just use the Accept-Language header, which they weren't aware of.

I wish more people knew about this: https://github.com/for-GET/http-decision-diagram and https://github.com/for-GET/know-your-http-well

Same thing with core UNIX commands, and so many other areas. We are doomed to incessantly reinvent the wheel and turn versatile abstractions into single-purpose complications. One must imagine Sisyphus happy.

Google famously uses geolocation to set language instead of Accept-Language. They have their reasons, which I disagree with.

https://news.ycombinator.com/item?id=30638590

Agreed, this experience is terrible.

Exactly. Just have jpg in the code and serve webp based on header. Problem solved.

If you can do avif with better results than webp then do that.

You can also vary on the data save if you are really keen.

You’d need to serve the proper resolution too

You can't easily use that with a plain* CDN though - you would need to add a Vary: Accept header and cache misses will increase as every variant of Accept: will cache a different version

* it seems fixable with things like Lambda@Edge and I wouldn't be surprised if a smart CDN which already does on-the-fly image compression had implemented this too

You can't parse the User-Agent header (which is the suggested method in the article) with a plain CDN either, so that argument is moot.

> I’d still recommend including JPEG XL versions of your images, because chances are that browser support will come eventually.

Likely not. Chrome is actually deprecating JPEG XL, the news broke yesterday.

https://www.phoronix.com/news/Chrome-Deprecating-JPEG-XL

WTF!? Jpeg xl was looking awesome, with lots of community interest: https://jpegxl.info/

This sucks.

> - There is not enough interest from the entire ecosystem to continue experimenting with JPEG XL - The new image format does not bring sufficient incremental benefits over existing formats to warrant enabling it by default

One caution here, as I'm just recently was working on automated image encoding for user submitted images - be careful of AVIF/JXL's encoding time.

I ran some benchmarks with a raw 48mp iPhone 14 dng file (converted to png to start with since jxl had issues going from dng directly) with imagemagick to illustrate.

jpg conversion: 1.74s

webp conversion: 3.77s

avif conversion: 67.96s

jxl conversion: 42.74s

Of course there's ways to optimize these, but still it's worth considering that these newer formats require an order of magnitude (if not more) increase in time to encode. If you're doing this for your own static site, it's worth doing. If you're dealing with user submitted images, make sure you understand the tradeoffs.

> avif conversion: 67.96s > jxl conversion: 42.74s

Holy shit-balls? Also I thought that quad bayer would come down to 12MP for the output?

try sharp

https://sharp.pixelplumbing.com/

I am using sharp to transform jpeg to avif images. The conversion also takes very long, for my images about 45s each.

Yep, we do about 30TB of user images a month, looked into converting to avif for storage savings, but the processing time basically stopped that. Ended up on webp.

I don't know much about AVIF (I guess the same concern applies though), but for JPEG XL there are actually two knobs for compression: distance or quality (`-quality` in ImageMagick) and effort (`-define jxl:effort`).

Distance is the target perceptual metric (Butteraugli), while effort controls a variance of the perceptual metric of the encoded image---which can be never accurate. Higher effort means more consistent quality but more computation as well. The default for cjxl, and probably also ImageMagick, is 7 ("squirrel"), which is very high and very slow as you noted. This is much like Brotli defaulting to the highest setting (-11) because both assume that you can spend much more time in compression. If it's not your assumption you should change that.

I do feel that this aspect of JPEG XL is not well communicated. Normally you have a single knob (quality) and increasing quality means more time spent for compression, but in JPEG XL this can be easily decoupled---very useful concept but also relatively alien one. A common mistake is to set the highest effort (9, nicknamed quite appropriately, "tortoise") for speed benchmark, which suggests that there will be people doing this in the production and complaining that JPEG XL compression is very slow, not realizing they can live with much lower efforts.

I am using the picture tag on my photography blog [1] to serve optimized images for a range of breakpoints. I also used to serve WEBP versions of the photos but noticed that (in my audience) the browser support was sufficiently low to drop the generation step for them and just serve various size/quality combinations of JPG. If time allows, I’ll also add a super low-res placeholder image in the background to prevent the slightly jarring layout jumps on slow desktop connections.

In combination with static hosting from S3 with CloudFront for caching, handwritten CSS + JS (except from Turbo), this yields a close-to-SPA snappiness and feel even on meager connections (as tested from rural Germany, fellow Germans may understand the implications).

The picture tag and the aspect-ratio CSS rule are real MVPs for dealing with photographs.

[1] https://44hz.de

Set width and height to the ratio of the image to prevent those layout shifts.

Thank you for the hint. I’ll revisit trying that when I can spare some time. I vaguely recall that doing so interfered with the way I ensured the consistent passepartout around my photographs while being responsive, but then again we are talking about CSS here and I just may have gotten things the wrong way around.

As a secondary tip: I believe you need to make sure that one of width or height is set to “auto” (in CSS). There was a good post on this on either CSS Tricks or SmashingMagazine. If I get some time I’ll try to find it and link it.

> I’ll also add a super low-res placeholder image in the background to prevent the slightly jarring layout jumps on slow desktop connections.

I try that, but don't end to work fine, thanks to the alt text showing, and displacing the background placeholder image

Are you sure about the browser support for WEBP being low? I thought that it didn't work with Safari before but they added it?

Actually, no. I started some projects in meat space in 2020 and did not yet find time to check in on this (or to add newer photographs, fwiw). I’ll have another look at the current support. Thank you for reminding me.

A significant downside to the <picture> element, and alternative image formats in general, is that when most users wanna download the image they expect an image format they already know how to work with. To most users an .avif or .webp are an annoyance because they reasonable expect most of their tools to be unable to open these.

It's disappointing that browser vendors haven't picked up on this and offered a "Save as PNG/JPEG/GIF" option when downloading images, but for now if it seems reasonable that if any user would want to download an image you're displaying then you should probably stick to the legacy formats.

Google search result do this weird trick. When you hover on a link the line at the bottom the your browser window shows the actual URL. But if you do "copy link URL" on it, you get a Google tracker URL in your clipboard.

Couldn't one do the same thing to make users get jpegs when they try to save a wepb? How bad would it be?

It could be used but this really seems like websites trying to "fix" browser UX. In cases like this where the problem is generic it seems like it is best for the browser to provide the UX it thinks is best for the users (possibly with preferences to allow the user to decide globally without needing to configure every single site).

A significant downside to the <picture> element, and alternative image formats in general, is that when most users wanna download the image they expect an image format they already know how to work with. To most users an .avif or .webp are an annoyance because they reasonable expect most of their tools to be unable to open these

Certainly not the case with WebP, which was announced by Google 12 years ago. On a recent version of macOS, Preview, the Mac’s default image and pdf viewer can open WebP and AVIF files, making it easy for Mac users to convert to another format if they wish. Also 3rd party graphics apps have supported WebP for years now.

AVIF support isn’t as widespread yet but that will quickly change now.

BTW, the iOS defaults to saving photographs in HEIC, which the average consumer has never heard of.

Webp support on Linux is spotty. The desktop file explorer for Gnome doesn't support it out-of-the-box for example.

My friend is using an old Mac laptop what can't display webp on webpages. A Mac laptop!

For me the biggest win in pages with lots of images is the img loading=lazy flag- https://developer.mozilla.org/en-US/docs/Web/Performance/Laz...

This only loads the image when it scrolls into view

Nitpick: The image is loaded when it’s near the viewport so loading is likely finished when the user reaches the image.

Thank you, this is exactly what I've been looking for and didn't know it.

> Safari doesn’t support the image/avif MIME type, so it will skip that one.

With iOS 16, current one, and newest Safari with macOS Ventura that support was added, see https://caniuse.com/avif

Fun fact: the "Your browser support avif" image in that post at https://jfhr.me/your-browser-supports.avif is currently... a png file.

    $ curl -s https://jfhr.me/your-browser-supports.avif | file -
    /dev/stdin: PNG image data, 400 x 100, 8-bit grayscale, non-interlaced

(same for the jxl one, but firefox apparently doesn't try to load it because of the type. The webp and jpeg ones are what they say they are.)

I would not use HTTP2 Server Push for anything, see: https://developer.chrome.com/blog/removing-push/

The title lost the essential "picture" tag part of it, probably to html sanitization

yeah I should've thought of that. fixed it now

MDN says that img tag purpose is not just to serve as a fallback, but also to specify size “and other attributes”. Does it also include alt attribute? This is important. Can someone confirm that?

Yes, all attributes from <img> applies to the selected image, not just width or height.

The article says

  The final <img> tag also specifies common attributes like width, height and alt text. Those attributes will apply regardless of which source the browser chooses!

The official spec for the picture element is here https://html.spec.whatwg.org/multipage/embedded-content.html... and it says

  The picture element is a container which provides multiple sources to its contained img element to allow authors to declaratively control or give hints to the user agent about which image resource to use, based on the screen pixel density, viewport size, image format, and other factors. It represents its children.

So I read that as the alt text (as well as other attributes other than the actual src url) come from the img tag

Adding something I don’t currently see mentioned explicitly: the img tag is the actual presented media element, picture/source are not displayed without an img tag. So the img tag itself isn’t even a fallback, only its src attribute.

It's also the img tag that you style in CSS. So for all layout purpose, it's still the img element that defines the container and the picture element is more a syntactical bracket to associate other source-sets with it. (This isn't very HTML-y, but, well…)

You can also use the picture tag to serve different images based on the browser's dimensions, to serve the best content and fast. So people with a TV get a wide image while a phone gets a small version, for example. The browser itself makes the choice and GETs the image from a list in the picture tag.

This could also conceivably be used for user tracking purposes as a way to determine without javascript the user's browser window setting. (I assume CSS can do this too.)

This made me look up jpeg2000 support. And then that made me sad :D

FTFY: HTML <picture> tag

Reply on news.ycombinator.com

Optimizing images with the HTML &lt;picture&gt; tag

Optimizing images with the HTML <picture> tag