High Efficiency Image File Format (HEIF/.heic) | Hacker News Reader

High Efficiency Image File Format (HEIF/.heic)

en.wikipedia.org

57 pointscapableweb86 comments

While it's great at compressing the image, it's PITA to work with until browsers support it. What's strange to me is that it's the format Apple uses by default on iOS, but even their own browser doesn't support it (!?). So when a user uploads a picture from an iPhone, you either have to load up an extra JS library (~1MB size last I included one in a web app) to process it or upload it to a remote server for processing.

AVIF I think is already well on the way. HEIC is DOA.

AVIF is supported on pretty much every modern browser now. Even Safari is adding support for it with the upcoming Safari release.

At this point AVIF vs JPEG XL is a more interesting discussion than anything vs HEIC.

Is it enabled by default in Firefox yet? I've just tried:

https://libre-software.net/image/avif-test/

on two Linux machines with Firefox 105, and both showed the alt image text, so I guess maybe not, although it's showing up as enabled in about:config.

Edit: https://avif.io/ seems to work and display converted .jpg files, so maybe the first URL is old and doesn't work.

Firefox supports AVIF out of the box for a while already.

Only still images apparently. Not supported in Safari yet (though its coming) and Edge has no support:

https://caniuse.com/avif

HEIF doesn't work in any web browser:

https://caniuse.com/heif

What's the point of animated AVIF vs for example AV1 video file?

My guess is as a drop-in replacement for GIF; use the <img> element instead of the more complicated <video> element, an inherent lack of audio, no need to specify looping or lack of video controls, etc.

I think Safari already supports using at least one actual video format as the `src` in an <img> element but it's weird to do so.

The first URL doesn't even show the fallback JPEG for me on Firefox for Android so there's probably something else interfering with the use of that website.

My browser seems to render AVIF just fine on Ubuntu and Manjaro, with Nvidia and Intel GPU backends. If it works with hardware acceleration on top of Nvidia's shitty drivers, I assume it'll work everywhere.

I've heard some old bug reports about colour management and AVIF not working together well in Firefox, that was the reason support had been baked in for a whole but the format was still locked behind a flag. I suppose they fixed that now that they enable the format by default.

The avif image URLs of these page are broken. at least, give me a nice 404 page

Accurate. My experience (as someone who has encoded a lot of both AVIF and JPEG XL images) is that JPEG XL tends to have superior detail preservation at medium to high bitrates, as well as preserving grain significantly better at all bitrates. It's also slightly less prone to banding than AVIF, but more prone to noticeable artifacting on sharp edges, which only goes away at higher bitrates.

It's pretty clear at this point that JPEG XL is a superior codec for e.g. photography. I think the question is going to be whether the browsers decide that it's worth supporting on the web, where lower quality images are the norm.

Some comparison images: https://afontenot.github.io/image-formats-comparison/#eaglef...

As a hobby photographer, this is a deeply interesting topic. I keep all my scans (yes, film still exists) as TIF files, but my clients don't know what to do with a TIF (and even fewer know how to tell a printer that the image uses a particular colorspace) so I usually send them the TIFs, plus some high-quality JPEGs. Why not HEIC/AVIF/etc? Because people know what a JPEG is.

Why TIF vs for example PNG? Are there any benefits?

I’m under the impression that TIFF is lossless (like FLAC) while png is lossy (like ogg).

PNG is not lossy. However, it doesn't support metadata anywhere nearly as well as TIFF, so I still use that for lossless photography storage.

Ah there you go! Didn’t know that :)

TIFF is a container. The payload it contains can be uncompressed, compressed losslessly with lzw//deflate/lzma/zstd, but it can also be lossy compressed with JPEG, WebP or even JPEG XL.

Meanwhile, PNG is always compressed, lossless.

Ogg is a container, you can have lossless FLAC in ogg.

I also didn’t know that! Thanks for letting me know

At least some camera raw files are also TIFF under the hood, e.g. Canon's CR2. TIFF also natively supports more than 8 bits per color channel, so you can still edit the photo and don't necessarily lose information in the process, whereas a single exposure adjustment on a PNG could lead to clipping and irrecoverable highlights or shadows.

Do you have any more information about this? I thought CR2 was floating point. Any details about how they store the mosaiced data in a TIFF would be very interesting, I assume.

EXIF is a TIFF file. I think most modern JPEG files are EXIF packages. A quick way to check, is to look at the first two bytes of a file. They are sort of a “BOM marker,” and will be either “II” or “MM”.

TIFF is really a package format. It’s so flexible, that it’s damn near impossible to write a “full-fat” TIFF reader (I’ve done that. Never again).

TIFF is so flexible, that you can define an image to have a 6-bit R channel, and a 12-bit G channel, for instance. I think some medical or scientific images may have actually done that.

You can also have pretty much any type of compression that you want, for the image data.

But most people use TIFF to store uncompressed “raw” image data. These files can be crazy big, but also won’t have compression artifacts.

Lossless jpeg xl could be a good choice for perfect archive copy, if needed. no idea about metadata though

In many categories of photos or images, JPEG XL [1] actually does better than AVIF in low bitrate settings. Which is the only advantage AVIF has over JPEG XL. And I am increasingly convinced that not losing details is the correct solution rather than smoothing things out in AVIF.

With progressive decoding, JPEG XL is also much a better fit for the web.

I just dont see how we continue to ignore JPEG XL and want to force AVIF adoption everywhere, when XL is technically superior, including encoding and decoding complexity. We literally have representatives from different companies and industries, most have been silent in the image / video codec format war begging for browser vendors to support JPEG XL.

But of course the cult and ideology somehow had people convinced the only accepted codec are AVIF, AV1 and Opus.

[1] https://twitter.com/jonsneyers/status/1563442356493230080

> In many categories of photos or images, JPEG XL [1] actually does better than AVIF in low bitrate settings.

It's not really consistent, but at small sizes JPEG XL sometimes falls apart, especially if the image contains a lot of smooth textures and clean lines that are easily compressed by AV1. https://afontenot.github.io/image-formats-comparison/#air-fo...

I don't think you need any kind of conspiracy here, AV1 is an open codec after all. The problem is more that (a) AVIF has been around for longer, (b) browsers have to implement most of the spec anyway because they need to have AV1 support, and (c) the hardware decoding story for AVIF is probably going to be better for quite a while, given its closeness to video. AV1 is still a great codec, and Opus is best-in-class for general purpose audio.

Also, I agree with you that optimizing an image codec for high quality use cases is the ideal, but a lot of casual users prefer blurred images with missing detail to noticeable artifacts.

One of the surprising fact from Cloudflare stats, was that most image on the web are actually 0.7bpp+ with the median being 1.0bpp. So image with 0.24bpp is surpassing small. And that is where progressive decoding works best, showing 0.3bpp image at launch while continue to download other details. There are problematic scenarios with JPEG XL, but I think a lot more could be optimised with encoder once they have 1.0.

Do you have a link for this? I tried searching for it but couldn't find any details. Mostly I'm wondering if those numbers include just JPEG or also include other formats. E.g. including PNG would tend to drag the median / mean up. Also, if sites are currently using 1.0 bpp JPEGs, I can only assume that they'll want to reduce sizes further when switching to a better codec, since 1.0 bpp is quite low for JPEG.

Note that there are samples where JPEG XL still has way too many artifacts at e.g. 1.3 bpp: https://afontenot.github.io/image-formats-comparison/#pont-d...

I do agree with you, though, that if a site is targeting 1.0 bpp or more, they're usually going to see better results with JPEG XL than AVIF.

> progressive decoding

If I'm not mistaken, it isn't enabled by default for JPEG XL and reduces the encoding efficiency by about 10%.

Late to Reply ( It was Chrome, not Cloudflare, which is arguably more representative )

https://www.spiedigitallibrary.org/conference-proceedings-of...

Figure 1

> The number of transferred bytes is important because it affects the latency and hence the user experience. Using this estimate, we calculated that between June 1st and 28th, 2019, the average BPP (weighted by the number of bytes transmitted with that BPP) was 2.97, 2.24 and 2.03 for small, medium and large images, respectively. We confirmed that there was no considerable difference between mobile and desktop platforms. This data indicates that image compression algorithms should target rich images with a relatively high BPP values, aiming to produce 1 BPP images almost free of artefacts instead of 0.5 BPP images with artefacts that are not too intrusive.

This is excellent, and does indeed suggest that JPEG XL should be broadly preferred in the browser. Thanks for the reference.

I vote for JPEG XL. The fact that you can completely eliminate things like imagemagick or 3rd party scaling workflows (simply retain the max-size original and truncate the stream proportional to the pixel area requested!) is huge IMHO.

> Even Safari is adding support for it with the upcoming Safari release.

Safari for iOS 16.0 supports it now for still images. Safari for 16.1 apparently adds support for animated image sequences.

AVIF support comes to macOS with macOS Ventura.

https://twitter.com/jensimmons/status/1547693585922789381

What is the patent situation for JPEG XL vs AVIF?

Both officially are free, but AVIF possibly has stronger defensive backing given it's defended from patent aggressors by AOM.

pretty sure Apple wants their Photo app to be one of the walled gardens, they're not looking to make it browseable in their browser.

and while I'm here, if you export/backup your heic photo library from your Mac, the .mov files from "live" photos are separate files, and not part of the .heic (unless they are both)

> pretty sure Apple wants their Photo app to be one of the walled gardens…

They're doing an awful job of it then.

If you read the link above you already know that HEIC (HEVC in HEIF) isn't an Apple invention, but an ISO standard that Apple chose to adopt. With macOS Ventura, Photos will also support AVIF (AV1 in HEIF).

In addition to those HEIF-based formats, Apple Photos supports any image file/container and media formats you might expect should work, along with RAW formats from these camera: https://support.apple.com/en-us/HT212821 You can also export to JPEG, PNG, and TIFF.

There's a whole ecosystem of apps that can work with the Apple Photos catalog directly, an extension system for 3rd-party developers, blah blah blah.

Actually, that’s built in via iCloud. I view my photo library in the browser.

>While it's great at compressing the image,

Cloudinary compared tens of thousands of samples and found HEIF turns out to be the best at compressing images, even better than AVIF.

So I do think HEIF could have its uses outside of the web. And it would be interesting to see how VVC performs as an image format.

I do wish the world pays more attention to JPEG XL though. Which is technically better in almost every single way.

If the webpage declares that it wants JPEGs or PNGs, but not HEICs, Safari will transparently convert the file before the webpage sees it.

But why would iOS default to an image format it doesn't support??

> HEIF and HEVC offer better compression than JPEG and H.264, so they use less storage space on your devices and iCloud Photos, while preserving the same visual quality.

iOS supports HEIF, Safari does not.

Why Safari does not, still, 5 years later, I can't tell you.

Safari is part of iOS though. If it was a third-party browser engine, this would've made sense; the browser lets the user upload files from their image photo library, but can't render some files in the library. But we're talking about the system browser engine, which is made by the same people who make iOS and is shipped as a core part of iOS. You'd think they'd at least think to make sure iOS's browser supports iOS's image format, or auto-convert to jpeg or something.

> we're talking about the system browser engine, which is made by the same people who make iOS and is shipped as a core part of iOS

The Apple employees who work on WebKit, Safari's open source browser engine, or even those who make Safari from WebKit are not the same Apple employees who work on iOS's file system or Photos.app or anything else found on an iPhone when you first take it out of the box.

Still, it is odd that iOS 16, including Safari, has added support for AVIF but HEIF still doesn't work in Safari.

I don't know why you're arguing, it sounds like we agree. Apple makes the product iOS, it's weird that one part of iOS (the camera/photos system) is completely incompatible with another part (the browser). You'd have expected them to add HEIF support to their browser, delay shipping HEIF until it's supported in their browser, or automatically convert from HEIF to a format that's supported by their browser.

Your argumentative tone would've made sense if my criticism was directed at the Safari team for not implementing HEIF support. But my criticism is towards Apple for failing to make a cohesive product.

It's probably because the two teams don't have a common direct report until they get to an SVP who has hundreds of issues and conflicts on their plate.

> Still, it is odd that iOS 16, including Safari, has added support for AVIF but HEIF still doesn't work in Safari.

Exactly. Safari had the motivation/resources to add AVIF support, but not HEIF. So it seems like Apple chose to back the wrong horse (HEIC) at first and is now pivoting to AVIF

> So when a user uploads a picture from an iPhone, you either have to load up an extra JS library

When I'm uploading a picture straight from an iPhone, using built-in Safari, it's JPG not HEIF. It's got a different name though - RFC 4122 UUID with ".jpeg" extension, not IMG_XXXX.JPG

File uploaded to a tiny Python/Flask app, to make sure no processing is done behind the scenes, is seen by file(1) as: "9E83B1D4-8AF1-4468-9095-0EE3FEEDF5BE.jpeg: JPEG image data, Exif standard: [TIFF image data, big-endian, direntries=X, orientation=X, xresolution=X...]" and can be opened with Firefox as your usual JPEG. HTML form asks for enctype="multipart/form-data" and doesn't specify anything about image formatting nor does it care what file type is being uploaded.

Ah yes, the image format I get when I airdrop photos off my iphone that 90% of the internet doesn't handle.

and also the format of desktop backgrounds, taking up hundreds of MB on every Mac...

These are the dynamic backgrounds that change throughout the day, they're a collection of ~16 images (not a single image).

You can change the camera settings to use “most compatible”, which reverts to JPG.

Note that this also changes the capture format to JPEG, and so will only work with new photos that you take.

If you want to AirDrop your existing HEIF photos, or want to AirDrop RAW (ProRAW) photos, or if you prefer to continue to capture in the higher-quality HEIF format, you can use this "AirDrop JPEGs" shortcut.

https://www.icloud.com/shortcuts/e455df464625443baca41366b9a...

For Windows 10, instead of getting the $1 HEVC / HEIF one from the Microsoft Store, opt for the free version for OEM's (named "HEVC Video Extensions from Device Manufacturer") instructions at https://www.howtogeek.com/680690/how-to-install-free-hevc-co...

There have been multiple remote code execution vulnerabilities in the codec since that was last updated, and MS clearly does not plan to patch it. It’s a very bad idea to install this.

Citation/link?

I'd certainly like more details. The latest CVE number that i can find for such flaws looks to be CVE-2022-30188, which Microsoft claims is patched:

https://msrc.microsoft.com/update-guide/vulnerability/CVE-20...

But I will say that there is a disturbing number of such vulnerabilities listed for this codec.

The last time I checked this, there were a few open CVEs that had been patched in the paid version, but the free one was not updated. It looks like the version numbers are similar now, so maybe my caveat no longer applies.

If you want to view them on Windows through system apps you need to install 2 codecs from the Windows Store, one of which costs $1 but you can also get it for free if you know the secret URL.

So ridiculous...

What's the secret URL?

ms-windows-store://pdp/?ProductId=9n4wgh0z6vhq

https://apps.microsoft.com/store/detail/hevc-video-extension...

The default photo format of iPhones. If that's important. Support was added to imagemagick a year-or-so ago.

In imagemagick, a HEIC image of quality "75" is approximately the same file size as an image of JPEG quality "95".

For the same quality "number", HEIC images are often larger than their JPEG equivalents, which can be confusing if you assume the number directly indicates a "quality" level, which it apparently doesn't.

I'm pretty sure it's also the default on newer Samsung Android.

> which it apparently doesn't.

It doesn't, because after 50+ years of research, there is still no consensus on image quality.

Even though there's no consensus, there are a bunch of great metrics that are in use such as Butteraugli. I believe JPEG-XL uses it.

In JXL the quality setting has been tuned especially well to give predictable speed/visual quality output. See Jon Sneyers's posts on twitter showing how quality variance is kept minimal: https://twitter.com/jonsneyers/status/1542560897306185729

Ahh the good old no context Wikipedia article

PSA: HEIC supports 16 bit images, Apple supports HEIC, but Apple does not support 16 bit HEIC images, only 8 bit (as far as I can figure it out)

I'm actually using lossy DNG (8 bit JPEG with a tonemap under the hood) as my universal storage format for casual photos.

Is it just me who thinks that the only valid archival format for images is lossless? Lossy can be saved for delivery to clients and downsampling and other such use-cases.

Lossless formats work great for some content categories (e.g. audio, images), while others (e.g. film) are more practically archived using formats designed with archival purposes in mind (e.g. ProRes 4444/4444 XQ).

Yes, lossless video is still too big to be practical for most purposes, but this was about images.

Mixed feeling about this, is this an enabler of (Apple's) life photos? I love those. But Nextcloud is my sync backend for picture and there it never worked, I simply could not watch the photos in the browser. Recently though Nextcloud switched to just uploading JPEGS, also mixed feelings about that, they are not original quality but my pictures web interface in Nextcloud is working nicely again...

> is this an enabler of (Apple's) life photos?

No. Live photos predate support for HEIF, it's just a photo and a video file next to one another (they're not even bundled on-device, that's just an abstraction of PHAsset).

Well I guess it could have changed with HEIF, but while I know the container supports derivations and sequences I'm not sure it supports such a wide disparity as "image and related video".

Hi capableweb Thanks for your work on improving Stable Diffusion

Hey, that's a unrelated but still welcome praise :) Thank you!

HEIC is a bad option. Check AVIF: https://en.wikipedia.org/wiki/AVIF

The image format even YEARS after Apple extensively used it can still not be saved into by Adobe Photoshop nor be displayed in Apple's own web browser Safari.

In PDF files, how to use HEIF or AVIF images (instead of jpegs, or whatever is used) to reduce final file size & increase quality?

Is HEIF constricted by patents? I can't think of any reason support is so terrible for them.

HEIF is basically a H.265 I frame. So yes.

High Efficiency Image File Format (HEIF)

ah brilliant

Do you not think HEIF might be interesting to HN readers, or that everybody already knows all about HEIF, or something I can't think of?

No, I don't think a wikipedia link with absolutely not context is interesting. In fact, I think it's low effort and shouldn't be automodded away.

That's oddly superior/hostile. I was vaguely familiar with the format as the choice on recent versions of iOS. Now I know HEIF is a container, and HEIC files indicate that the codec is HEVC. And I know it's part of MPEG and, to my surprise, is more efficient than JPEG. And it made me think, are there generalizable cases where JPEG still wins? Maybe I'll look into it. And all that from a few seconds. I appreciate the submitter.

Edit: And now I'm learning things from other commenters who have been inspired to provide interesting tidbits they know about HEIF!

You should email the site mods about this viewpoint, with a link to your comment; I think they’d be interested to hear your case, and they have the tools to do this readily available if they agree with you. Or they may have context for how links like this could be of interest, or why they aren’t automodded, or etc. (Use the contact link in the footer if you’re interested.)

It would be cool if the person who posted the article also added a comment providing context. I thought the wiki article was great, but more context may result in a richer discussion.

I think those that need to know about it, will find out the same way everybody does - when they wonder why they can't access the photos taken on their phone.

Reply on news.ycombinator.com