Daala: Modern video compression for the internet | Hacker News Reader

Daala: Modern video compression for the internet

github.com

146 pointssetra38 comments

Daala uses quite a few innovative approaches to video encoding. There's a good set of articles on some of the approaches they've tried at https://www.xiph.org/~xiphmont/demo/.

Since the formation of the Alliance for Open Media, Mozilla/Xiph's work has mainly shifted towards helping them develop AV1 (for standardization by the IETF hopefully next year) which is mostly based on what would have been Google's VP10. That said, their work on Daala was not wasted as they are working on integrating the most successful parts of Daala into AV1 where it makes sense, such as their entropy coder.

They'll also need to make sure they stick to AV1 for a long-ass time. H264 is fully accelerated. VP8 was supposed to become a bigwig thing, some (alpha) hardware supported started to pop up and then Google nixed it and switched to VP9. Only now are VP9 accelerated CPUs popping up and now they're already planning to switch to AV1 soon.. no damn way the smaller CPU builders (everyone not Intel or AMD) will build in decoding support for another standard that'll be nixxed in a few years. That's also why H265 will (sadly) be victorious, even despite heavy licensing. H264 has been in use for so long without big changes that CPU builders felt comfortable dedicating silicon to it. Unless AOmedia (= Google, Cisco etc.) give a guarantee they won't switch up to 'AV2' or whatever in a few years, it won't see large hardware support.

On a sidenote: screw Google for switching YouTube over to VP9 for some bandwidth savings, full well knowing 95%+ of people does not have VP9 acceleration and thus having YouTube absolutely slaughter their laptop battery. H264ify fixes this, but still. ALWAYS put the user first.

Edit: to be clear, I know all the big builders (Intel, Nvidia, AMD etc) are part of AOMedia. However, they also all promised VP8 support which never happened, and VP9 support which is more than a generation late.

> no damn way the smaller CPU builders (everyone not Intel or AMD) will build in decoding support

Companies smaller than Intel or AMD like Chips & Media, VeriSilicon, Polycom, and Broadcom are part of AOMedia: http://aomedia.org/about-us/

ARM is also part of AOMedia. So I don't think the claim that there will be no hardware support for AV1 adds up. If anything, AV1 is already well positioned for good hardware support before it's even released.

For those comment surfers, here's the list of all the current members of the AOM:

Adobe, Allegro¹, Amazon, AMD¹, Amlogic¹, ARM¹, ateme, BBC R&D, Broadcom¹, Chips & Media¹, Cisco, Google, Intel¹, Ittiam, Microsoft, Mozilla, Netflix, Nvidia, Polycom¹, Socionext¹, VeriSilicon¹, Vidyo, XILINIX¹

¹ indicates hardware or hardware IP companies.

An article on the codec indicates: > The group is aiming to freeze the bitstream sometime between the end of the 2016 and March 2017. Expect to see browser-based support soon thereafter, with the first hardware support within 12 months after that. http://www.streamingmedia.com/Articles/Editorial/Featured-Ar...

Wow, thank you for the h264ify mention! I can now play 4K YouTube videos on my MacBook Pro, who'da thought?

I don't see how VP9 hardware decoders shipping late means that H.265 wins? There's zero H.265 content on the web right now, so if anything H.265 would need to catch up. Also, the ffvp9 software decoder in Firefox is quite efficient. Hopefully Chrome will eventually ship it too.

90% of TV series and movies I watch come in x265 at half x264 file size.

VP9 is already in all Chromium based browsers. And no, its not nearly as efficient, because there's no hardware acceleration/ You can clearly see this in any Chromium browser: Watch a YouTube video without the H264ify extension and Activity Monitor open. Your 'Energy Use' will spike to about 100. With H264ify its only 40. Safari even stays in the upper 20s.

But it uses the libvpx decoder, not ffvp9. https://blogs.gnome.org/rbultje/2014/02/22/the-worlds-fastes...

Also check that you're getting the same resolution on both. H.264 requires more bits, so you might be getting a lower resolution (which also takes less energy to decode).

Both are 1080p60FPS. Also, VP9 will never be more efficient than H264 on my machine since I have a 2015 13" Macbook Pro, and that model only has H264 acceleration in the CPU. And the 2015 Pro is more modern than 90% of the machines out there..

But doesn't that have an Intel Iris 6100 in it (https://www.apple.com/macbook-pro/specs-2015/)? If so you should have accelerated VP8 and VP9 decoding. Maybe it's not exposed by the drivers on macOS.

Nope, only VP8 (which is dead). I meant fixed hardware acceleration (Intel Quick Sync). Draws a lot less power than the GPU. This is the full Quick Sync acceleration sheet for each generation:

Version 1 (Sandy Bridge) Quick Sync was initially built into some Sandy Bridge CPUs, but not into Sandy Bridge Pentiums or Celerons.

Version 2 (Ivy Bridge) The Ivy Bridge microarchitecture included a "next-generation" implementation of Quick Sync.

Version 3 (Haswell) The Haswell microarchitecture implementation is focused on quality, with speed about the same as before (for any given clip length vs. encoding length).[citation needed] It has seven hard-coded quality/performance levels (called "target usages"), compared to the three in previous generations. The highest-quality TU1 setting is intended to be higher quality than Ivy Bridge's version, and the highest speed TU7 setting should be faster, higher-quality, and more battery-friendly for mobile devices. This generation of Quick Sync supports the H.264/MPEG-4 AVC, VC-1 and H.262/MPEG-2 Part 2 video standards.

Version 4 (Broadwell) The Broadwell microarchitecture adds VP8 hardware decoding and encoding support. Also, it has two independent bit stream decoder (BSD) rings to process video commands on GT3 GPUs; this allows one BSD ring to process decoding and the other BSD ring to process encoding at the same time.

Version 5 (Skylake) The Skylake microarchitecture adds a full fixed-function H.265/HEVC main/8-bit encoding and decoding acceleration, hybrid and partial HEVC main10/10-bit decoding acceleration, JPEG encoding acceleration for resolutions up to 16,000×16,000 pixels, and partial VP9 encoding and decoding acceleration.

Version 6 (Kaby Lake) The Kaby Lake microarchitecture adds full fixed-function H.265/HEVC Main10/10-bit encoding and decoding acceleration & full fixed-function VP9 8-bit & 10-bit decoding acceleration & 8-bit encoding acceleration.[13][14]

Broadwell has VP9 acceleration with the right drivers:

https://communities.intel.com/thread/59216

https://en.wikipedia.org/wiki/Broadwell_(microarchitecture)#...

So you just have a driver problem. Apple was playing with adding WebP support in beta versions of iOS a while back. So maybe they'll add WebP and WebM support in the next major version of macOS and iOS.

> GPU Accelerated

Not what I'm looking for. The GPU still uses about 2x more energy than the dedicated Quicksync silicon, although its better than only using the CPU.

In case you're wondering, I did read the second link, it actually uses the first link as a source.

I'm not trying to be contrarian btw, for me (and my Macbooks battery) I just feel that the distinction between GPU acceleration and a dedicated decode core is an important one.

I can't upvote this enough.

They need to resist the urge to treat file formats the same rapidly iterating way as web APIs -- the way the VP8/VP9/VP10 series has been treated so far. Rapid evolution and the lack of well-defined profiles will discourage hardware implementation or lead to a proliferation of profiles the hardware decoders can't play.

The fragmentation will discourage the format's use beyond online video, and the next ITU-T or MPEG format will win out for any and all uses where longevity is important. As for online video, this recipe will likely be an archival nightmare regardless [1].

[1] https://news.ycombinator.com/item?id=12980359

There is definitely a lag between the first software solutions popping up and the first hardware solutions being available.

But there are lots of off the shelf product like the ARM Mali Video IP that you can just buy and slap onto your product. This is what many SOC vendors do.

These codec engine IP products have their entire business model built around the two facts that:

1) New codecs will be released

2) New codecs will be a lot more computationally expensive. For example: h265 offers up to 50% bandwidth savings compared to h264 but requires about 3x CPU cycles for decoding. During encoding the issue is a lot worse.

Full disclosure: I worked in the ARM Video Codec group.

This is way out of my depth, but how different are these decoders? Could we have some more general purpose hardware instead of specific hardware for every format?

I know that one of the design goals for Daala was being friendly to multi-core CPU and GPU decoding. However, you can always improve performance using custom hardware.

Part of Daala will be reused in AV1 http://aomedia.org/

How does this compare to VP9/10 by a "human quality" metric? There's lots of cherry-picking about stats which makes an encoding sound impressive on paper but result in a worse perceptible quality. This is a really good article about testing encoders: https://blogs.gnome.org/rbultje/2016/05/02/the-worlds-best-v...

Additionally, adoption is really slow of VP9 already so I don't see anyone switching to Daala soon.

> I don't see anyone switching to Daala soon

No one will switch to Daala. They'll switch to AV1. As other commenters have noted, the AV1 codec is built from Google's VP10, Mozilla and Xiph's Daala, and Cisco's Thor. Wikipedia has a good summary of AV1: https://en.wikipedia.org/wiki/AOMedia_Video_1

And if you're interested in the source, the AV1 git repository is: https://aomedia.googlesource.com/

We use a wide variety of metrics to compare codecs, not just the usual PSNR metric:

https://arewecompressedyet.com/?job=av1_rt_ctrl_new_twitch%4...

While none of these metrics are as good as the "human quality" metric, some are better than others, in different ways, so we try to optimize across the board.

On that note, have you considered adding Netflix's VMAF to AWCY?

Last time this was discussed on YC: [1] Except that time it was the "Thor" codec.

MPEG-2 is effectively out of patent now for almost all uses. Here's MPEG-LA's patent list.[2] The one remaining unexpired patent [3] only applies to error correction for pay-per-view satellite broadcasts.

[1] https://news.ycombinator.com/item?id=10042469 [2] http://www.mpegla.com/main/programs/M2/Documents/m2-att1.pdf [3] https://www.google.com/patents/US7334248

But what's the advantage of MPEG-2 over VP9? VP9 is royalty-free for all use cases and outperforms MPEG-2. Are there particular expired MPEG-2 patents that make sense to include in AV1?

The key is that now all royalty-free codecs can use MPEG-2 technologies as their base instead of having to work around the lack of them.

Okay, but which technologies in particular? Since they've already been worked around, what's the advantage?

Workarounds might have had to sacrifice efficiency or simplicity.

Might have, but again what's an example? What's a technique from MPEG-2 now available that could have improved VP9 or can improve AV1?

B-frames.

This all reminds me that modular smartphones would be awesome if the hardware video codec chip could be a module.

How far did Daala progress? They planned to start beating H.265 both in quality and in efficient decoding in 2016/2017. Did that happen?

Here is the most recent update on Daala, from IETF 97:

https://www.ietf.org/proceedings/97/slides/slides-97-netvc-d...

> They planned to start beating H.265 both in quality and in efficient decoding in 2016/2017. Did that happen?

If I'm not missing something, this suggests that Daala's quality will be comparable to H.265. No doubt encoders for both will continue to get better.

Not just comparable, the goal was to be ahead.

A good goal to be sure, but one that it doesn't look like they'll make.

IMHO, "comparable to H.265" may not be as inspiring but is still very impressive. If they can do this without stepping on existing IP, it'll be an incredible achievement.

Looks like a number of parts are used in AV1, but not lapped transforms?

Does it use a middle-out algorithm?

Reply on news.ycombinator.com