Hacker News Reader

Comment by treve | Hacker News Reader

It seems a bit crazy that for a new image format, developers need to rely on an operating system shipping an API, instead of depending on an (open-source?) library.

View on HN

Well, that's a two-pronged issue: one is that it is a patent minefield unlike ever before seen, the other is that it is incredibly slow in software at any quality level.

Also, it is legally terrifying to build anything on HEVC, I'm frankly surprised Apple is even trying. The basic license fees are easily ten times as much for HEVC as they were for H.264, and there are known applicable patents which aren't even in the pools (both pools! there are two!). There are also potentially license fees that apply to individual performances/transfers of content, so if you're Netflix or something you'll end up paying for every unit of content shipped.

For the lot of these reasons, the Alliance for Open Media (including the Daala team (Xiph and Mozilla), The Thor team (Cisco), Microsoft, Nvidia, AMD, Intel, Mediatek, Netflix, Google, and a good couple dozen other major players) has formed and will probably be freezing a standard bitstream format this year on a format that outperforms HEVC for a given bitrate, a given CPU time target, and on energy consumption (and implementation complexity) in hardware.

>both pools! there are two!

There's actually three now, but the newest one hasn't announced their pricing yet: http://velosmedia.com/

And don't forget all the people not in pools! It's like a support program for lawyers...

The format is incredibly complex kitchen-sink of features and legacy ISO specs. There is an open-source library for it, but it's written in C++, and issues filed about it make it look like it's insecure (i.e. a large complex C++ codebase written by less than an expert) and not even entirely compatible with Apple's implementation.

So I think it's fine to leave HEIF as basically an internal format of Apple's camera app, and for everything else export to formats more suitable for interchange (e.g. BPG which uses same the compression scheme, but is a simple lightweight container).

I dunno, I think the container format sounds pretty reasonable, at least at a high level (I haven’t tried to implement it or anything). Many of those features [choice of lossy or lossless compression; extensible embedded metadata; multiple images per file; animation; transparency; embedded thumbnail; non-destructive indication of preferred crop/rotation; choice of color spaces and bit depth; ...] are extremely useful, but currently are only supported (in conjunction) by completely proprietary app-specific formats like PSD or the like.

I can imagine at least a dozen use cases for this format that current formats don’t serve at all.

See this post implementing it in ffmpeg: https://ffmpeg.org/pipermail/ffmpeg-devel/2017-August/215003...

I guess the main possibilities are (1) use an existing openly specified format, for which there is already at least some published working code, even if it isn’t perfect, or (2) invent a brand new format, and hope you don’t screw it up somehow by overlooking something important. I’m not entirely convinced #2 would have been any better (does Apple have a good reputation for inventing data serialization formats?), and inre #1 I’m not sure if there’s any extant choice better than this new format with support for all of the uses Apple cares about.

Considering that all the widely supported formats throughout the industry have various severe limitations so that it’s impossible to cover the typical use cases (even for just web images, say) using less than 3 or 4 completely different formats, it’s not clear that anyone else has figured out how to really solve this one either.

Too bad that the world couldn’t find someone with good taste, extensive experience, and a forward-looking vision to make a more future-compatible image format about 15–20 years ago.

In this particular case, (1) and (2) were pretty much the same, as the format that became HEIF was drafted up in 2013 and finished in 2015, by Nokia, as a contribution to the proceedings of the joint MPEG/ITU-T JCT-VC working group working on the standard MPEG-H; it was accepted.

Previously, with Apple 'live photos', having two separate files sitting on the disk was good enough.

HEIF is very new, but it's now blessed by the premier standardization organizations in this space as a component of the next wave of media coding. Aside from the the innate value of standards in which industry players agree on common formats and semantics, Apple's move needs to be seen in context of the rival 'Alliance for Open Media' [1], which includes many big names working on future open media codecs outside of the usual MPEG / ITU-T / contributors' patent pool combo. Apple's adoption of an MPEG standard is also signalling its commitment to that particular brand of standardization.

[1] http://aomedia.org/about-us/

It was over engineered from a Image format perspective, but it is very likely Apple has a lot of things in mind and things like Live Photos or whatever they are making.

Webp supports most of those features actually, iirc. I mean HEIF is to HEVC as WEBP is to VP8, right?

Not that webp is as incredible as Google claimed, but that was always a given with the inferior codec it’s based upon.

They can do it themselves if they want to. Apple is including support with High Sierra for everybody but anyone can add support to any formats on their own apps without involving Apple at all. Many apps, like VLC and HandBrake for example, have been supporting H.265 for a while.

It looks like this is also true for mpv. Honestly, having to deal with codecs is a waste of time from the user's perspective. I seriously love how players like mpv and VLC can play pretty much every format under the sun without additional configuration.

Without any GPU support sadly.

Yeah, while I can understand Apple picking up HEVC, this image format makes very little sense to me.

Outside of Apple I can't see this have any uptake due to being patent/royalty laden, which means that images you encode using this format need to be reencoded again (and degraded in case of lossy, which is the most common type) in order to be shown on the web or on other operating systems/devices.

Meanwhile, every Apple user’s photo library will be half the size compared to JPEG. For many users photos are the single largest category in their iOS device storage. More importantly for Apple, they (want to) store everyone’s photos in their cloud, so they need half the storage and bandwidth to pull that off.

For sending to other people, most photos are going to be downsized anyway so a transcoding isn’t the end of the world.

Apple doesn't like downsizing messages. iMessage sends full resolution, multi-MB pictures.

That's a setting - though I don't know the default.

Irrelevant because we are talking about compression formats.

It is very likely in the next 12 - 24 months time, every Smartphone shipped will have paid the HEVC patents. So while the PC industry may be a little different, all mobile phone will have HEVC support soon.

Now whether the software for HELF support will catch up is another question.

Presumably both are options, but the OS library has hardware acceleration.

It looks like FFmpeg can do it, albeit in a somewhat roundabout way.

http://jpgtoheif.com

How does WebP compare?

Keep in mind WebP is essentially a VP8 I-frame in a RIFF container [1]. VP8 is the predecessor video codec to VP9; VP9 being the codec targeted by its makers to be in competition with HEVC which is commonly deployed inside HEIF -- whether they succeeded in this promise is an exercise left to the reader, possibly by following sources such as the ones compiled on Wikipedia [2].

But not all compression techniques that are applicable to video are relevant with still images. WebP's performance has been evaluated and some have found it lacking [3], while some have found it good [4].

Discounting the fact that the HEIF container has some useful features on its own, the picture inside is often an HEVC I-frame, which is the same idea as Fabrice Bellard's BPG. Therefore, comparisons between VP8 and BPG may also be indicative of typical HEIF-as-it-will-be-typically-deployed performance.

[1] https://developers.google.com/speed/webp/docs/riff_container [2] https://en.wikipedia.org/wiki/VP9 [3] https://en.wikipedia.org/wiki/WebP#Criticism [4] https://blog.cloudflare.com/a-very-webp-new-year-from-cloudf... [5] https://bellard.org/bpg/

Personally I'm not impressed with the quality of the lossy encoding, it's not enough of a improvement on JPEG to make it worthwhile, for lossless however it's the best format I know of right now, compresses MUCH better than PNG, and both compression and decompression is MUCH faster.

FLIF compresses better than webp, but it also compresses/decompresses more slowly, and the difference in size is quite small so overall for lossless compression, WebP is my choice.

Oh, thank you!! Music to my ears! I designed the WebP lossless format. Our most recent effort PIK tries to achieve the same in the lossy world that WebP lossless has done in the lossless coding -- a practical, efficient, fast and simple compromise.

You're welcome, and huge thanks for WebP.

PIK had flown under my radar, looks really interesting. Any estimate of when a first release will be out ?

Patents.

One gets you guaranteed immunity if you sign up for three different patent pools and pay a massive arm and a leg. The other claims to be I encumbered but the reality isn’t so clear.. but you’d probably have Google’s lawyers to help you out if it came to that.

Reply on news.ycombinator.com