Qualcomm is adding AV1 support, which could be huge for online video
protocol.com
protocol.com
The people involved in AV1 had a software decode plan B that has worked better than most of them hoped.
If you combine that with encoding choice based on software decode impact, most big players have started with software rollouts, but still seen gains.
"You need hardware support for a codec" is often the same kind of delaying tactic as "renewables need storage". You do both in parallel, not enter an infinite loop where you don't do either because the other isn't complete yet.
https://e360.yale.edu/features/in-boost-for-renewables-grid-...
That's also a business decision, just with a bigger number attached. And hitherto it's been far cheaper to use gas. But that's now a "conflict mineral".
I see this claim again and again on HN, but in reality "entrenched interests" ship with support for lots of royalty-free audio, video, and image formats.
The reality is that AV1 is just not ready for prime-time yet. As TFA notes, it's a chicken-and-egg problem, and few companies have stepped up to be among the first chickens.
If this rumor is correct, the first phones with Qualcomm-powered AV1 support will ship in 2023. At that point, expect the AV1 IP wars to begin. AV1's future depends on how that goes.
Intel added AV1 decoding to their onchip GPU's in 2020 [0]. Why didn't it begin then?
[0] https://github.com/intel/media-driver/releases/tag/intel-med...
I don't have any special insight into Sisvel's strategy, but I'd guess there are a few factors. This assumes that Intel isn't already a licensee — Sisvel has 32 AV1/VP9 licensees last I looked, and I believe only a few are public.
- Very little content is delivered as AV1, and where it is used it's not exclusive, so content distributors could easily kill Sisvel's golden goose today. They can't move too soon.
- I imagine Sisvel is happy to focus on low-hanging fruit while AV1's popularity (presumably) soars. They'll continue to build their stable of voluntary licensees until AV1 use hits a tipping point where the ROI for harder targets makes sense. The day Apple announces support for AV1, Sisvel will be celebrating harder than HN'ers.
- When push comes to shove, I'd guess that Sisvel will most likely build a start building legal wins by going after smaller companies first, then work their way up to AOM members.
Sisvel needs AV1 to be successful in order to extract maximum revenue. Qualcomm's support increases the odds that will happen, but it doesn't guarantee it. That necessary success probably happens a minimum of 2 years after broad device support.
Things are moving, but not quickly.
VLC shipped an optimized x86 software decoder very recently, but Handbrake has yet to find an open source software encoder in good enough shape that they can add it for general consumer use.
It very well may be AV1 and H.266 will play the same roles in the future.
But Google has never been a coherent company.
AV1 is an open, royalty-free codec, so I don't understand how Google could help competitors.
Maybe Google is in a place to give away knowledge, although I doubt they know more about a competitor's chip design.
VP9 came too close on the heels of VP8. Thanks to WebRTC in particular, VP8 encoding support is everywhere. AV1 has a lot more backing, thanks in no small part to the H.265 licensing mess.
At least for iOS/OSX it is only supported for WebRTC and nothing else.
This is very hyerbolic. There's plenty of places that the HEVC licencing catastrophe stopped its use. The release of 3 more MPEG codecs that compete with it is not a good sign for example.
- The reference silicon design for an AV1 decoder was very space-inefficient, and so many chip vendors refused to use it. This, however, necessitated the development of internal or 3rd-party decoding IP which slowed uptake considerably.
- AV1, partially due to it's patent-free design, has an extremely complicated compression process. So complicated, that even in software it is still a few dozen times slower than H.264 encoding. This, plus the first issue mentioned above, means on-chip AV1 encoding is a ways away.
- AV1, despite being "patent-free," has had a patent troll (Sisvel) come along and claim that they have a bunch of patents applying to it. They want 20 cents per device. AV1's founding group claims to have a legal defense fund for members but the terms and coverage are unclear. Sisvel wants 20 cents per device using AV1, so now you need to get the lawyers involved to figure out whether to bet on paying Sisvel or bet on taking the AV1 founder's view of the matter.
- Finally... there's AV2 coming. Though a ways away, it will use recently-expired patents to further help its compression capabilities, new techniques that couldn't be completed on time for AV1, and will feature a more optimized reference silicon implementation that might be actually easily usable by chip designers this time.
https://en.wikipedia.org/wiki/AV1#History
"While still working on the format, the encoder was not targeted for production use and speed optimizations were not prioritized. Consequently, the early version of AV1 was orders of magnitude slower than existing HEVC encoders. Much of the development effort was consequently shifted towards maturing the reference encoder. In March 2019, it was reported that the speed of the reference encoder had improved greatly and within the same order of magnitude as encoders for other common formats."
Other encoders also exist that are much faster, listed in a different section of the wiki.
https://en.wikipedia.org/wiki/AV1#Software_implementations
I didn't know about Sisvel and the hardware design problems, although Sisvel is in the wiki. The specific license fees that you listed are interesting.
b) CoreGraphics and CoreAnimation are underpinned by Metal. If you were building an OS that was going to be used by every device you make (computer, phone, tablet, monitor, watch, TV streamer, headset) would you rely on an open source project that you don't control and may have different values, wishes, roadmap etc than you.
Apple's XNU kernel is a derivation of open source OSFMK and FreeBSD kernels, and Apple's Darwin has large components of FreeBSD userland. They use (se)L4 for running their Secure Enclave.
I remember in the early days of MacOS where entries from the FreeBSD release notes would appears as word-for-word copies in the Apple ones.
Apple's involvement with OpenGL for decades didn't mean they wouldn't abandon it and roll their own.
Apple's involvement with MPEG for decades doesn't mean they won't abandon it and roll their own.
(I don't know codes well, this is an honest question.)
In comparison compression systems designed for playback, like AV1, can have more occasional key-frames (points that you can easily jump to, then work your way to other frames), and it is more ok to have compression artifacts, especially ones that are just going to look like motion blurs when played back.
So the compromises between features and size of the output stream are different.
As someone who mainly cares about blu-ray / large movies, I was under the impression that HEVC was becoming the gold standard. Is that not the case, and/or is AV1 the next iteration, and/or do they solve different problems?
AV1 is supposedly a "free" format, but I wouldn't bet my business on it. The patent situation is unclear. https://en.wikipedia.org/wiki/AV1#Patent_claims
Except that's not how legal cases work.
Just because all three of you infringe doesn't mean lawsuits will result for all three.
It's quite likely they would allow Youtube and Netflix to continue infringing to promote the growth and adoption of the codec whilst they go after smaller players who won't be able to put up a fight.
This is why grassroots-scale efforts to defend against patent trolls are so important - you absolutely have to prevent precedence case building for patent trolls or it becomes so much harder down the road.
You can try and invalidate the patent but in a situation like this where the patent holders are serious companies and legitimate innovators it's unlikely to get you far.
And yes it is very common to bully lots of smaller players rather than get into an expensive protracted lawsuit with a large one.
I am not aware of this being the case for either company.
In a lot of jurisdictions a 3rd party can join a lawsuit, it's called an intervention [1].
I'm not fully aware on US law, but e.g. in NL all large internet providers joined a lawsuit as defendants when a copyright enforcer wanted 1 to block The Pirate Bay.
Notice also that the encoder implementation is as important as the codec format itself for the quality/bitrate ratio.
Where are you getting that information?
Paying off _some_ patent holders doesn't put you in the clear for all the others. No licensing pool indemnifies you against all other claims. There's never an exhaustive list of patents that apply to any particular product. "Competing product's patent situation is unclear" is always true, no matter the product (unless it's at least some 20+ years old, which should make it _somewhat_ safe because applicable patents expired). Bringing it up to favor one product over another is FUD though because it's true for the other product just as well.
The patent regime is an extortion racket: patent offices hand out exclusive rights for money, without accepting any responsibility that these rights are warranted. They don't even guarantee that their own catalog is free from conflicting rights.
VVC was finalized in 2020 and there is some influence backing it already where although DVB will evaluate AV1, they've already hopped on VVC.
I guess it is quite expensive to implement AV1 encoder.
Edit: I meant no consumer hardware support it.
Oh I did a typo. There still is no consumer-grade product supports AV1 encode.
You're still right that there's no standard, consumer-focused card, but that's "yet" – the product is just launching later this year.
H.263, H.264, VP8, HEVC, and AV1 certainly were all used by various videoconferencing apps before hardware blocks reached consumer devices.
One can assume next gen nvidia gpus(rumored for the end of this year) will have it as well.
If you mean AV1 specifically, do you think software encoders won't get anywhere near x264, which can do it in less than one core?
From what I understand, AV1’s encoding complexity is more like H.265 than H.264. It’d be more appropriate to look at the performance of x265 to see what might be possible.
Current video call applications look like crap
>also most twitch streams
Don't they use hardware encoding when possible? And higher bitrate than video calls.
>do you think software encoders won't get anywhere near x264, which can do it in less than one core?
Not at full quality
Their purpose is to dump the frame into compliant stream in least possible time, so the compliant decoder can decode it back. It doesn't concern itself with using less bits; if it does, it is nice, but not deal breaker.
So if the hardware encoders can produce VP9/HEVC/AV1 at roughly the same bit-budget, it doesn't make sense to use the more complicated one. It makes sense to use the one, where you do have to pay less fees, though.
In 2019, Twitch predicted roll-out of AV1 support by 2024, but then the pandemic happened: https://www.streamingmedia.com/Articles/ReadArticle.aspx?Art...:
> For head content, it's okay to streaming multiple formats because the viewership is huge, so streaming multiple formats, although increase our cost, it still actually save our traffic cost, so it's still worth while. But for the tail content, it's very different. We can only afford streaming one single format, so our strategy is currently still doing H.264 using hardware, high-density hardware solution, but we're hoping towards 2024, 2025, the AV1 ecosystem is ready, we want to switch to AV1 100%.
>
> Jan Ozer: Did you say 2024 and 2025?
>
> Yueshi Shen: 2024, this is our projection right now. But on the other hand, so as I said, our AV1 release will be, for the head content will be a lot sooner, we are hoping 2022-2023 we are going to release AV1 for the head content. But for the head content, we will continue to stream dual-format, AV1, H.264. But for the tail content, we are hoping towards five years from now to AV1, whole eco, every five-year-old device supports AV1. Then, we will be switching to AV1 100%.
The initial roll-out will probably just involve transcoding at sub-source quality settings to save bandwidth with no encoding to be done by the streamer.
Right now, platforms have only talked about AV1 for transcoding purposes, so there's not the same urgency to add encoding support on consumer-level GPUs. Even thought it'd be amazing to see AV1 streams and YouTube videos, it's mostly on the server-end to cut costs and make things easier on people with data plans and cellular.
Later generations often have more tools, which can lead to bigger die size, but this can also lead to energy gains from those tools depending on content/usage.
See cisco's presentation about adding AV1 to Webex and how it could outperform H264 in this niche due to having specific tools for it.
In Netflix type content the grain emulation for example could make a big difference.