Netflix’s AV1 Journey: From Android to TVs and Beyond
netflixtechblog.com
netflixtechblog.com
So now that h.264, h.265, and AV1 seem to be the three major codecs with hardware support, I wonder what will be the next one?
Hopefully AV2.
This is a big victory for the patent system.
I'm not sure what you mean by "patent system" having a victory here, but it's not that the goal of promoting innovation is happening.
IIRC AV1 decoding hardware started shipping within a year of the bitstream being finalized. (Encoding took quite a bit longer but that is pretty reasonable)
Yeah, that's... sparse uptake. A few smart TV SOCs have it, but aside from Intel it seems that none of the major computer or mobile vendors are bothering. AV2 next it is then!
I am eagerly awaiting for AV2 test results.
Eventually people and companies will associate HEVC with "that thing that costs extra to work", and software developers will start targeting AV1/2 so their software performance isn't depending on whether the laptop manufacturer or user paid for the HEVC license.
[1] https://arstechnica.com/gadgets/2025/11/hp-and-dell-disable-...
But this just indicates that HEVC etc. is a dead end anyway.
This is so 90's. Why not AV2026 or AV26.0 ? /s
We already have some of the stepping stones for this. But honestly much better for upscaling poor quality streams vs just gives things a weird feeling when it is a better quality stream.
Imagine a criminal investigation. A witness happened to take a video as the perpetrator did the crime. In the video, you can clearly see a recognizable detail on the perpetrator's body in high quality; a birthmark perhaps. This rules out the main suspect -- but can we trust that the birthmark actually exists and isn't hallucinated? Would a non-AI codec have just showed a clearly compression-artifact-looking blob of pixels which can't be determined one way or the other? Or would a non-AI codec have contained actual image data of the birth mark in sufficient detail?
Using AI to introduce realistic-looking details where there was none before (which is what your proposed AI codec inherently does) should never happen automatically.
https://en.wikipedia.org/wiki/JBIG2#:~:text=Character%20subs...
The material belief is that modern trained neural network methods that improve on ten generations of variations of the discrete cosine transform and wavelets, can bring a codec from "1% of knowing" to "5% of knowing". This is broadly useful. The level of abstraction does not need to be "The AI told the decoder to put a finger here", it may be "The AI told the decoder how to terminate the wrinkle on a finger here". An AI detail overlay. As we go from 1080p to 4K to 8K and beyond we care less and less about individual small-scale details being 100% correct, and there are representative elements that existing techniques are just really bad at squeezing into higher compression ratios.
I don't claim that it's ideal, and the initial results left a lot to be desired in gaming (where latency and prediction is a Hard Problem), but AI upscaling is already routinely used for scene rips of older videos (from the VHS Age or the DVD Age), and it's clearly going to happen inside of a codec sooner or later.
AI upscaling built in to video players isn't a problem, as long as you can view the source data by disabling AI upscaling. The human is in control.
AI upscaling and detail hallucination built in to video codecs is a problem.
AI compression doesn't have to be the level of compression that exists in image generation prompts, though. A SORA prompt might be 500 bits (~1 bit per character natural English), while a decompressed 4K frame that you're trying to bring to 16K level of simulated detail starts out at 199 million bits. It can be a much finer level of compression.
This is very true, but we're talking about an entertainment provider's choice of codec for streaming to millions of subscribers.
A security recording device's choice of codec ought to be very different, perhaps even regulated to exclude codecs which could "hallucinate" high-definition detail not present in the raw camera data, and the limitations of the recording media need to be understood by law enforcement. We've had similar problems since the introduction of tape recorders, VHS and so on, they always need to be worked out. Even the phantom of Helibronn (https://en.wikipedia.org/wiki/Phantom_of_Heilbronn) turned out to be DNA contamination of swabs by someone who worked for the swab manufacturer.
The coding side of "codec" needs to know what the decoding side would add back in (the hypothetical AI upscaling), so it knows where it can skimp and get a good "AI" result anyway, versus where it has to be generous in allocating bits because the "AI" hallucinates too badly to meet the quality requirements. You'd also want it specified, so that any encoding displays the same on any decoder, and you'd want it in hardware as most devices that display video rely on dedicated decoders to play it at full frame rate and/or not drain their battery. It it's not in hardware, it's not going to be adopted. It is possible to have different encodings, so a "baseline" encoding could leave out the AI upscaler, at the cost of needing a higher bitrate to maintain quality, or switching to a lower quality if bitrate isn't there.
Separating out codec from upscaler, and having a deliberately low-resolution / low-bitrate stream be naively "AI upscaled" would, IMHO, look like shit. It's already a trend in computer games to render at lower resolution and have dedicated graphics card hardware "AI upscale" (DLSS, FSR, XeSS, PSSR), because 4k resolutions are just too much work to render modern graphics consistently at 60fps. But the result, IMHO, noticibly and distractingly glitches and errors all the time.
Modern video codecs are complex beasts. It's not as simple as "take a macroblock and find some motion vectors that minimize residual below a certain threshold, else code as intra-block". They have hundreds of mutually-exclusive techniques to compress a certain area of the frame. Determining which technique will require the smallest residual is done with fast early-skip heuristics that often make the wrong decision. The official manual for x265, the H.265/HEVC encoder (https://x265.readthedocs.io/en/master/cli.html), has literally hundreds of options, almost all about tuning these myriad of heuristics for your particular input.
AI can be used to enhance things like early-skip heuristics. "This block looks like it'll benefit from a really-detailed motion search in this particular area" or "we'll save bits if we bypass the DCT step and quantize the block directly" or "this frame should definitely be a B-frame". Encoders already use heuristics to do this (brute forcing all possible decisions to find which is optimal is too slow), but they don't always make the best decision. An AI could be used to improve that.
Now when I say AI, I'm not talking about massive, multi-billion weight monstrosities that synthesize nonsense, but extremely simple neural networks with a few thousand weights. The popular Opus codec uses a simple NN to estimate whether or not a frame of audio is speech or music, and uses that determination to decide whether to use their speech-optimized algorithm (SILK) or their music-optimized algorithm (CELT) to encode that particular frame. It's a short read but a very good one: https://jmvalin.ca/opus/opus-1.3/
This could be extended to video encoders without being used for interpolation where they would be liable to synthesize things that aren't there, DLSS-style.
That'd be h264 (associated patents expired in most of the world), vp9 and av1.
h265 aka HEVC is less common due to dodgy, abusive licensing. Some vendors even disable it with drivers despite hardware support because it is nothing but legal trouble.
(And yes, even for something like Netflix lots of people consume it with phones.)
Where did it say that?
> AV1 powers approximately 30% of all Netflix viewing
Is admittedly a bit non-specific, it could be interpreted as 30% of users or 30% of hours-of-video-streamed, which are very different metrics. If 5% of your users are using AV1, but that 5% watches far above the average, you can have a minority userbase with an outsized representation in hours viewed.
I'm not saying that's the case, just giving an example of how it doesn't necessarily translate to 30% of devices using Netflix supporting AV1.
Also, the blog post identifies that there is an effective/efficient software decoder, which allows people without hardware acceleration to still view AV1 media in some cases (the case they defined was Android based phones). So that kinda complicates what "X% of devices support AV1 playback," as it doesn't necessarily mean they have hardware decoding.
If it was a stat about users they’d say “of users”, “of members”, “of active watchers”, or similar. If they wanted to be ambiguous they’d say “has reached 30% adoption” or something.
Also, either way, my point was and still stands: it doesn't say 30% of devices have hardware encoding.
AV1 was specifically designed to be friendly for a hardware decoder and that decision makes it friendly to software decoding. This happened because AOMedia got hardware manufacturers on the board pretty early on and took their feedback seriously.
VP8/9 took a long time to get decent hardware decoding and part of the reason for that was because the stream was more complex than the AV1 stream.
I think they certainly go hand in hand in that algorithms relatively easier for software vs previously are easier for hardware vs previously and vice versa, but they are good at different things.
Bit masking/shifting is certainly more expensive in software, but it's also about the cheapest software operation. In most cases it's a single cycle transform. In the best cases, it's something that can be done with some type of SIMD instruction. And in even better cases, it's a repeated operation which can be distributed across the array of GPU vector processors.
What kills both hardware and software performance is data dependency and conditional logic. That's the sort of thing that was limited in the AV1 stream.
He's not talking about simple bit shifts. Imagine if you had to swap every other bit of a value. In hardware that's completely free; just change which wires you connect to. In software it takes several instructions. The 65 bit example is good too. In hardware it makes basically no difference to go from 64 bits to 65 bits. In software it is significantly more complete - it can more than double computation time.
I think where software has the advantage is sheer complexity. It's harder to design and verify complex algorithms in hardware than it is in software, so you need to keep things fairly simple. The design of even state-of-the-art CPUs is surprisingly simple; a cycle accurate model might only be a few tens of thousands of lines of code.
> I'm not claiming that software will be more efficient. I'm claiming that things that make it easy to go fast in hardware make it easy to go fast in software.
The actual constraints on what makes hardware or software slow are remarkably similar. It's not ultimately the transforms on the data which slow down software, it's when you inject conditional logic or data loads. The same is true for hardware.
The only added constraint software has is a limited number of registers to operate on. That can cause software to put more pressure on memory than hardware does. But otherwise, similar algorithms accomplishing the same task will have similar performance characteristics.
Your example of the bitshift is a good illustration of that. Yes, in hardware it's free. And in software it's 3 operations which is pretty close to free. Both will spend far more time waiting on main memory to load up the data for the masking than they will spend doing the actual bit shuffling. The constraint on the software is you are burning maybe 3 extra registers. That might get worse if you have no registers to spare forcing you to constantly load and store.
This is the reason SMT has become ubiquitous on x86 platforms. Because CPUs spend so much time waiting on data to arrive that we can make them do useful work while we wait for those cache lines to fill up.
Saying "hardware can do this for free" is an accurate statement, but you are missing the 80/20 of the performance. Yes, it can do something subcycle that costs software 3 cycles to perform. Both will wait for 1000 cycles while the data is loaded up from main memory. A fast video codec that is easy to decode with hardware gets there by limiting the amount of dataloads that need to happen to calculates a given frame. It does that by avoiding wonky frame transformations. By preferring compression which uses data-points in close memory proximity.
Where did you read that it was designed to make creating an hardware decoder easier?
Ok, I don't think I'll find it. I think I'm mostly just regurgitating what I remember watching at one of the research symposiums. IDK which one it was unfortunately [1]
[1] https://www.youtube.com/@allianceforopenmedia2446/videos
Will it, though ?
Why create a SW spec and hope that the HW will support it ? Why not design together with HW ?
2020 feels close, but that's 5 years.
I'm running an LG initially released in 2013 and the only thing I'm not happy with is that about a year ago Netflix ended their app for that hardware generation (likely for phasing out whatever codec it used). Now I'm running that unit behind an Amazon fire stick and the user experience is so much worse.
(that LG was a "smart" TV from before they started enshittifying, such a delight - had to use and set up a recent LG once on a family visit and it was even worse than the fire stick, omg, so much worse!)
[0] https://github.com/RootMyTV/RootMyTV.github.io [1] https://github.com/throwaway96/downgr8
I don't see anything in that comment implying such a thing. It's just about the uptake of decoders.
They mentioned they delivered a software decoder on android first, then they also targeted web browsers (presumably through wasm). So out of these 30%, a good chunk of it is software not hardware.
That being said, it's a pretty compelling argument for phone and tv manufacturers to get their act together, as Apple has already done.
When I'm watching something on YouTube on my iPhone, they're usually shipping me something like VP9 video which requires a software decoder; on a sick day stuck in bed I can burn through ten percent of my battery in thirty minutes.
Meanwhile, if I'm streaming from Plex, all of my media is h264 or h265 and I can watch for hours on the same battery life.
Hopefully, we can just stay on AV1 for a long while. I don't feel any need to obsolete all the hardware that's now finally getting hardware decoding support for AV1.
Continous improvement ? CADT ? What shall the next one bring ? Free meals ?
Basically, a network effect for an open codec.
I am not sure if this is a serious question, but I'll bite in case it is.
Without DRM Netflix's business would not exist. Nobody would license them any content if it was going to be streamed without a DRM.
I don't agree. If people refused to watch DRM-protected content, they would get rid of it.
For example, Pluto TV is a free streaming service that has much content without DRM. GOG lets you buy DRM-free games. Even Netflix itself lets you stream DRM-free content, albeit in low resolution.
The low resolution option is something many rightsholders accept, but from a product proposition perspective it's difficult to explain to many customers. They're just grumpy that they paid for content and can only watch it in SD, that reduces your customer satisfaction. Better to do nothing than a poor job sometimes.
They just want DRM because it makes them even more money. Or at least they think it does. I have yet to find a single TV show or film that isn't available on Bittorrent so I don't think the DRM is actually preventing piracy in the slightest. I guess they want it in order to prevent legal tools from easily working with videos, e.g. for backup, retransmission etc.
I can confirm that while there are serious issues with Widevine (and to a lesser extent PlayReady), the protection measures aren't totally ineffective. My work in improving security had measurable results saving significant amounts of money and reducing content leakage. One memorable time my colleague and I had a call with a big rights owner who tracks the piracy of their assets and they said "Can you tell us what you've been doing recently? Because the amount of piracy from your platform has dropped significantly."
Anti-piracy and content security is also a differentiator between platforms when bidding for content deals. Rights owners will absolutely give the best deals to the provider who provides more assurance and avoid platforms which are leaky buckets.
I know that doesn't fit the narrative, but until recently this was literally my job.
I don't think I've ever looked for a recent show and not seen a pirate version.
Just thought I'd extract the part I found interesting as a performance engineer.
Still impressive numbers, of course.
It's not obvious whether there's any automated way to reliably detect the difference between "use of HDR" and "abuse of HDR". But you could probably catch the most egregious cases, like "every single pixel in the video has brightness above 80%".
That sounds like a job our new AI overlords could probably handle. (But that might be overkill.)
My idea is: for each frame, grayscale the image, then count what percentage of the screen is above the standard white level. If more than 20% of the image is >SDR white level, then tone-map the whole video to the SDR white point.
So all music producers got out of compressing their music was clipping, and not extra loudness when played back.
Exacly this. I usually do not want high dynamic audio because that means it's either to quiet sometimes or loud enough to annoy neighbors at other times, or both.
Like HDR abuse makes it sound bad, because the video is bright? Wouldn't that just hurt the person posting it since I'd skip over a bright video?
Sorry if I'm phrasing this all wrong, don't really use TikTok
Sure, in the same way that advertising should never work since people would just skip over a banner ad. In an ideal world, everyone would uniformly go "nope"; in our world, it's very much analogous to the https://en.wikipedia.org/wiki/Loudness_war .
OTOH pointing a flaslight at your face is at least impolite. I would put a dark filter on top of HDR vdeos until a video is clicked for watching.
For things filmed with HDR in mind it's a benefit. Bummer things always get taken to the extreme.
The "normal" video should aim to be moderately bright on average, the extra peak brightness is good for contrast in dark scenes. Other comments comparing it to the loudness war ar apt. Some music streaming services are enfircing loudness normalization to solve this. Any brickwalled song gets played a bit quieter when the app is being a radio.
Instagram could enforce this too, but it seems unlikely unless it actually effects engagement.
99.9% of people expect HDR content to get capped / tone-mapped to their display's brightness setting.
That way, HDR content is just magically better. I think this is already how HDR works on non-HDR displays?
For the 0.01% of people who want something different, it should be a toggle.
Unfortunately I think this is either (A) amateur enshittification like with their keyboards 10 years ago, or (B) Apple specifically likes how it works since it forces you to see their "XDR tech" even though it's a horrible experience day to day.
The solution is for social media to be SDR, not for the UI to be HDR.
HDR is meant to be so much more intense, it should really be limited to things like immersive full-screen long-form-ish content. It's for movies, TV shows, etc.
It's not what I want for non-immersive videos you scroll through, ads, etc. I'd be happy if it were disabled by the OS whenever not in full screen mode. Unless you're building a video editor or something.
eventually, it'll wear itself out just like every other over use of the new
I know how bad the support for HDR is on computers (particularly Windows and cheap monitors), so I avoid consuming HDR content on them.
But I just purchased a new iPhone 17 Pro, and I was very surprised at how these HDR videos on social media still look like shit on apps like Instagram.
And even worse, the HDR video I shoot with my iPhone looks like shit even when playing it back on the same phone! After a few trials I had to just turn it off in the Camera app.
I have zero issues and only an exceptional image on W11 with a PG32UQX.
KDE Wayland went the better route and uses Gamma 2.2
Unless you're using a video editor or something, everything should just be SDR when it's within a user interface.
Because I want the Rec. 2020 and 10-bit color encoding, I must also choose HDR, as these features are usually only available together, even if I do not get any serious advantage from HDR and HDR-encoded movies can usually be viewed well only in a room with no light or with dim light, otherwise most of them are too dark.
The latest android release has a setting that is the HDR version of “volume leveling”.
But... did I miss it, or was there no mention of any tool to specify grain parameters up front? If you're shooting "clean" digital footage and you decide in post that you want to add grain, how do you convey the grain parameters to the encoder?
It would degrade your work and defeat some of the purpose of this clever scheme if you had to add fake grain to your original footage, feed the grainy footage to the encoder to have it analyzed for its characteristics and stripped out (inevitably degrading real image details at least a bit), and then have the grain re-added on delivery.
So you need a way to specify grain characteristics to the encoder directly, so clean footage can be delivered without degradation and grain applied to it upon rendering at the client.
Any movie or TV show is ultimately going to be streamed in lots of different formats. And when grain is added, it's often on a per-shot basis, not uniformly. E.g. flashback scenes will have more grain. Or darker scenes will have more grain added to emulate film.
Trying to tie it to the particular codec would be a crazy headache. For a solo project it could be doable but I can't ever imagine a streamer building a source material pipeline that would handle that.
"I can't ever imagine a streamer building a source material pipeline that would handle that."
That's exactly what the article describes, though. It's already built, and Netflix is championing this delivery mechanism. Netflix is also famous for dictating technical requirements for source material. Why would they not want the director to be able to provide a delivery-ready master that skips the whole grain-analysis/grain-removal step and provides the best possible image quality?
Presumably the grain extraction/re-adding mechanism described here handles variable grain throughout the program. I don't know why you'd assume that it doesn't. If it didn't, you'd wind up with a single grain level for the entire movie; an entirely unacceptable result for the very reason you mention.
This scheme loses a major opportunity for new productions unless the director can provide a clean master and an accompanying "grain track." Call it a GDL: grain decision list.
This would also be future-proof; if a new codec is devised that also supports this grain layer, the parameters could be translated from the previous master into the new codec. I wish Netflix could go back and remove the hideous soft-focus filtration from The West Wing, but nope; that's baked into the footage forever.
From the creator's PoV their intention and quality is defined in post-production and mastering, color grading and other stuff I am not expert on. But I know a bit more from music mastering and you might be thinking of a workflow similar to Apple, which allows creators to master for their codec with "Mastered for iTuenes" flow, where the creators opt-in to an extra step to increase quality of the encoding and can hear in their studio the final quality after Apple encodes and DRMs the content on their servers.
In video I would assume that is much more complicated, since there are many quality the video is encoded to allow for slower connections and buffering without interruptions. So I assume the best strategy is the one you mentioned yourself, where AV1 obviously detects on a per scene or keyframe interval the grain level/type/characteristics and encode as to be accurate to the source material at this scene.
In other words: The artist/director preference for grain is already per scene and expressed in the high bitrate/low-compression format they provide to Netflix and competitors. I find it unlikely that any encoder flags would specifically benefit the encoding workflow in the way you suggested it might.
That's good, since that's what I said.
"The artist/director preference for grain is already per scene and expressed in the high bitrate/low-compression format they provide to Netflix and competitors. I find it unlikely that any encoder flags would specifically benefit the encoding workflow in the way you suggested it might."
I'm not sure you absorbed the process described in the article. Netflix is analyzing the "preference for grain" as expressed by the grain detected in the footage, and then they're preparing a "grain track," as a stream of metadata that controls a grain "generator" upon delivery to the viewer. So I don't know why you think this pipeline wouldn't benefit from having the creator provide perfectly accurate grain metadata to the delivery network along with already-clean footage up front; this would eliminate the steps of analyzing the footage and (potentially lossily) removing fake grain... only to re-add an approximation of it later.
All I'm proposing is a mastering tool that lets the DIRECTOR (not an automated process) do the "grain analysis" deliberately and provide the result to the distributor.
> if the delivery conduit uses AV1, you can optimize for it
You could, in theory, as I confirmed.
> It's already built, and Netflix is championing this delivery mechanism.
No it's not. AV1 encoding is already built. Not a pipeline where source files come without noise but with noise metadata.
> and provides the best possible image quality?
The difference in quality is not particularly meaningful. Advanced noise-reduction algorithms already average out pixel values across many frames to recover a noise-free version that is quite accurate (including accounting for motion), and when the motion/change is so overwhelming that this doesn't work, it's too fast for the eye to be perceiving that level of detail anyways.
> This scheme loses a major opportunity for new productions unless the director can provide a clean master and an accompanying "grain track."
Right, that's what you're proposing. But it doesn't exist. And it's probably never going to exist, for good reason.
Production houses generally provide digital masters in IMF format (which is basically JPEG2000), or sometimes ProRes. At a technical level, a grain track could be invented. But it basically flies in the face of the idea that the pixel data itself is the final "master". In the same way, color grading and vector graphics aren't provided as metadata either, even though they could be in theory.
Once you get away from the idea that the source pixels are the ultimate source of truth and put additional postprocessing into metadata, it opens up a whole can of worms where different streamers interpret the metadata differently, like some streamers might choose to never add noise and so the shows look different and no longer reflect the creator's intent.
So it's almost less of a technical question and more of a philosophical question about what represents the finished product. And the industry has long decided that the finished product is the pixels themselves, not layers and effects that still need to be composited.
> I wish Netflix could go back and remove the hideous soft-focus filtration from The West Wing, but nope; that's baked into the footage forever.
In case you're not aware, it's not a postproduction filter -- the soft focus was done with diffusion filters on the cameras themselves, as well as choice of film stock. And that was the creative intent at the time. Trying to "remove" it would be like trying to pretend it wasn't the late-90's network drama that it was.
You are ignoring the fact that the scheme described in the article does not retain the pixel data any more than what I'm proposing does; in fact, it probably retains less, even if only slightly. The analysis phase examines grain, comes up with a set of parameters to simulate it, and then removes it. When it's re-added, it's only a generated simulation. The integrity of the "pixel data" you're citing is lost. So you might as well just allow content creators to skip the pointless adding/analyzing/removing of grain and provide the "grain" directly.
Furthermore, you note that the creator may provide the footage as a JPEG2000 (DCP) or ProRes master; both of those use lossy compression that will waste quality on fake grain that's going to be stripped anyway.
Would they deliver this same "clean" master along with grain metadata to services not using AV1 or similar? Nope. In that case they'd bake the grain in and be on their way.
The article describes a stream of grain metadata to accompany each frame or shot, to be used to generate grain on the fly. It was acquired through analysis of the footage. It is totally reasonable to suggest that this analysis step can be eliminated and the metadata provided by the creator expressly.
And yes I'm well aware that West Wing was shot with optical filters; that's the point of my comment. The dated look is baked in. If the creators or owner wanted to rein in or eliminate it to make the show more relatable to modern audiences, they couldn't. Whether they should is a matter of opinion. But if you look at the restoration and updating of the Star Trek original series, you see that it's possible to reduce the visual cheesiness and yet not go so far as to ruin the flavor of the show.
I don't need to provide evidence that the resulting difference in quality is negligible because you can play with ffmpeg to verify it yourself if it want. I'm just telling you from experience.
I understand your logic that it could be built, and how it would skip steps and by definition have no loss of quality in that part. But process- and philosophy-wise it just doesn't fit, that's all I'm explaining.
And the fact that JPEG2000/ProRes are lossy is irrelevant. They're encoded at such high quality settings for masters that they become virtually lossless for practical purposes. That's why the source noise is so high-fidelity in the first place.
For example, we could GREATLY improve compression ratios if we could reference key frames anywhere in the file. But devices only have so much memory bandwidth and users need to be able to seek while streaming on a 4g connection on a commuter train. I would really like to see memes make use of SVG filters and the like, but basically everyone flattens them into a bitmap and does OCR to extract metadata.
It's also really depressing how little effort is put into encoding, even by the hyper-scalers. Resolution (SD, HD, 4k and 8k) is basically the ONLY knob used for bitrate and quality management. I would much prefer to have 10 bit color over an 8K stream yet every talking head documentary with colored gradient backgrounds has banding.
Finally, there is the horror that are decoders. There a reference files that use formal verification to excise every part of a codec's spec. But Hollywood studios have dedicated movie theaters with all of the major projectors and they pay people to prescreen movies just to try and catch encoding/decoding glitches. And even that fails sometimes.
So sure, anything is possible. Flash was very popular in the 56k days because it rendered everything on the end device. But that entails other tradeoffs like inconsistent rendering and variable performance requirements. Codecs today do something very similar: describe bitmap data using increasingly sophisticated mathematical representations. But they are more consistent and simplify the entire stack by (for example) eliminating a VM. Just run PDF torture tests through your printer if you want an idea of how little end devices care about rendering intent.
Removing real film grain from content and then recreating it parametrically on the other side is not the same thing as directly encoding it. You are killing a lot of information. It is really hard to quantify exactly how we perceive this sort of information so it's easy to evade the consequences of screwing with it. Selling the Netflix board on an extra X megabits/s per streamer to keep genuine film grain that only 1% of the customers will notice is a non-starter.
In the case of fake grain that's added to modern footage, I'm calling out the absurdity of adding it, analyzing it, removing it, and putting yet another approximation of it back in.
Why is it absurd? The entire encoding process is an approximation of the original image. Lossy compression inevitably throws away information to make the file size smaller. And the creation of the original video is entirely separated from the distribution of it. It'll be stored losslessly for one thing.
The only question that matters is does the image look good enough after encoding. If it doesn't look good enough then no one will watch it. If it does look good enough then you've got a viable video distribution service that minimizes its bandwidth costs.
So? That fact only emphasizes the absurdity and loss potential here:
1. Acquire "clean" digital footage.
2. Add fake grain to said footage
3. Compress the grainy footage with lossy compression, wasting a bunch of the data on fake detail that you just added.
4. Analyze the footage to determine the character of the fake grain, calculate parameters to approximate it later with other fake grain
5. Strip out the fake grain, with potential for loss of some original image details
6. Re-add fake grain with the calculated parameters
If you don't see the absurdity there, I don't know what to tell you.
The creation of the original video is separate from the distribution of the video. In distribution the video will be encoded to many formats and many bitrates to support playback on as many devices at as many network speeds as possible.
The distributed video will never exactly match the original. There simply isn't the bandwidth. The goal of video encoding is always just to make it look good enough.
Now you can be mad about two things nobody else notices.
Good that the OCAs really work and are very inspiring in content delivery domain.
I actually blamed AV1 for the macro-blocking and generally awful experience of watching horror films on Netflix for a long time. Then I realized other sources using AV1 were better.
If you press ctl-alt-shift-d while the video is playing you'll note that most of the time that the bitrate is appallingly low, and also that Netflix plays their own original content using higher bitrate HEVC rather than AV1.
That's because they actually want it to look good. For partner content they often default back to lower bitrate AV1, because they just don't care.
I wonder if it has more to do with proximity to edge delivery nodes than anything else.
Even fixing that issue the video quality is never great compared to other services.
Meanwhile pirated movies are in Blu-ray quality, with all audio and language options you can dream of.
You can also include "vf=format:film-grain=no" in the config itself to start with no film grain by default.
I do sometimes end up with av1 for streaming-only stuff, but most of that looks like shit anyway, so some (more) digital smudging isn’t going to make it much worse.
The problem you see with AV1 streaming isn't the film grain synthesis; it's the bitrate. Netflix is using film grain synthesis to save bandwidth (e.g. 2-5mbps for 1080p, ~20mbps for 4k), 4k bluray is closer to 100mbps.
If the AV1+FGS is given anywhere close to comparable bitrate to other codecs (especially if it's encoding from a non-compressed source like a high res film scan), it will absolutely demolish a codec that doesn't have FGS on both bitrate and detail. The tech is just getting a bad rap because Netflix is aiming for minimal cost to deliver good enough rather than maximal quality.
I’m ok with that for things where I don’t care that much about how it looks (do I give a shit if I lose just a little detail on Happy Gilmore? Probably not) and agree that faking the grain probably gets you a closer look to the original if you’re gonna erase the grain for better compression, but if I want actual high quality for a film source then faked grain is no good, since if you’re having to fake it you definitely already sacrificed a lot of picture quality (because, again, the grain is the picture, you only get rid of it by discarding information from the picture)
But for anything modern, the film grain was likely added during post-production. So it really is just random noise, and there’s no reason it can’t be recreated (much more efficiently) on the client-side.
Most new UHD, yes, but otherwise BRD primarily use h264/avc
There also are no scene rules for AV1, only for H265 [1]
With the SVT-AV1 encoder you can achieve better quality in less time versus the x265 encoder. You just have to use the right presets. See the encoding results section:
https://www.spiedigitallibrary.org/conference-proceedings-of...
https://wiki.x266.mov/docs/encoders/SVT-AV1
https://jaded-encoding-thaumaturgy.github.io/JET-guide/maste...
This problem is only just now starting to get solved in SVT-AV1 with the addition of community-created psychovisual optimizations... features that x264 had over 15 years ago!
FGS makes a huge difference at moderately high bitrates for movies that are very grainy, but many people seem to really not want it for HQ sources (see sibling comments). With FGS off, it's hard to find any sources that benefit at bitrates that you will torrent rather than stream.
Bigger PT sites with strict rules do not allow it yet and are actively discussing/debating it.Netflix Web-DLs being AV1 is definitely pushing that. The codec has to be a select-able option during upload.
It’s more like “why does Netflix kill my battery within an hour when I used to be able to play for 20”
The only way I can get them to serve me an AV1 stream is if I block "protected content IDs" through browser site settings. Otherwise they're giving me an H.264 stream... It's really silly, to say the least
I've found the ratio of a fixed quality vs CPU load to be better, and I've found it is reasonably good at retaining detail over smoothing things out when compared to HEVC. And the ability to add generated "pseudo grain" works pretty well to give the perception of detail. The performance of GPU encoders (while still not good enough fory maybe stringent standards) is better.
Uh what. (Embedded) hardware lasts a long time (and it should!). TV's around the globe are not all built after 2018. H264 is still the gold standard if you want to be sure a random device has hardware acceleration.
I make use of this by taking a USB hard drive with me on trips. Random TV's rarely have issue with my H264 catalogue. It'll be a while before I look at AV1 for this. Sure, I wish I could benefit faster, but I don't want people to throw out perfectly good hardware either!
Sounds like they set HEVC to higher quality then? Otherwise how could it be the same as AVC?
Netflix developed VMAF, so they're definitely aware of the complexity of matching quality across codecs and bitrates.
They could do the Big Tech way: make it all 'free' for a good while, estinguish/calm down any serious competition, then make them not 'free' anymore.
In the end, you cannot trust them.
VP9 isn't H.265 level. That is the marketing spin of AOM. And even AOM members admit VVC is better than AV1.
Liking one codec or whether it is royalty free is one thing, whether it is performing better is another thing.
Other developers ran into a ton of issues with licensing HEVC for their own software which is still a complete pain.
Anyway, people are now looking at what's next. VVC came out quite a while ago, and AV1 more recently, but when people are looking for the current sota codec with at least some good support, they end up choosing between the two, realistically. And yeah, VVC has advantages over AV1 and they are very different technically. But the market has pretty loudly spoken that VVC is a hassle no one wants to mess with, and AV1 is quickly becoming the ubiquitous codec with the closest rival VVC offering little to offset the licensing troubles (and lack of hardware support at this point as well)
Anyway, just saying. VVC is a huge pain. HEVC still is a huge pain, and though I prefer it to vp9 and it has much better quality and capabilities, the licensing issue makes it troublesome in so many ways. But the choice almost always comes down to vp9 or HEVC, then AV1 or VVC. Though at this point it might as well be, in order, h264, vp9, HEVC, AV1, and no one really cares about VVC.
I wish everyone knew the difference between patents and copyright.
You can download an open source HEVC codec, and use it for all they care according to their copyright. But! You also owe MPEG-LA 0.2 USD if you want to use it, not to mention an undisclosed sum to actors like HEVC Advance and all the other patent owners I don't remember, because they have their own terms, and it's not their problem that you compiled an open source implementation.
https://www.androidcentral.com/streaming-tv/chromecast/netfl...
This would mean if you have a new device and an old device in the same network the Netflix app might show only your old device! What?
Maybe I'm misunderstanding what they've done here?
So why isn’t it AV1 higher? The post doesn’t say, so we can only speculate. It feels like they’re preferring hardware decoding to software decoding, even if it’s an older codec. If this is true, it would make sense - it’s better for the client’s power and battery consumption.
But then why start work on AV2 before AV1 has even reached a majority of devices? I’m sure they have the answer but they’re not sharing here.
Honestly not complaining, because they were using AV1 with 800-900~kbps for 1080p content, which is clearly not enough compared to their 6Mbps h.264 bitrate.
It's really sad that most people never get to experience a good 4K Blu-ray, where the grain is actually part of the image as mastered and there's enough bitrate to not rely on sharpening.
I dont think that is true of any streamers. Otherwise they wouldnt provide the UI equivalent of a shopping centre that tries to get you lost and unable to find your way out.
What’s the logic with changing the title here from the actual article title it was originally submitted with “AV1 — Now Powering 30% of Netflix Streaming” to the generic and not at all representative title it currently has “AV1: a modern open codec”? That is neither the article title nor representative of the article content.
AV1 is good enough that the cost of not licensing might outweigh the cost of higher bandwidth. And it sounds like Netflix agrees with that.
h264 is a very good codec.
in double blind testing, at the same bitrate, you'll pick the "new" codec every time. but yes, they're trying to save money on bandwidth. it's annoying
AV1 definitely is missing some techniques patented by h264 and h265, but AV2 is coming around now that all the h264 innovations are patent free (and now that there's been another decade of research into new cutting edge techniques for it).
Our title policy is pretty simple and attuned for maximum respect to the post’s author/publisher and the HN audience.
We primarily just want to retain the title that was chosen by the author/publisher, because it’s their work and they are entitled to have such an important part of their work preserved.
The only caveat is that if the title is baity or misleading, we’ll edit it, but only enough that it’s no longer baity or misleading. That’s because clickbait and misleading titles are disrespectful to the audience.
Any time you see a title that doesn’t conform to these principles, you’re welcome to email us and ask us to review it. Several helpful HN users do this routinely.
"We primarily just want to retain the title that was chosen by the author/publisher, because it’s their work and they are entitled to have such an important part of their work preserved."
Nobody said the title had to be deleted. But when it doesn't convey WHAT the "thing" is, it needs augmentation. Currently on page 4 there's an example that not only conveys nothing, but DOESN'T respect the actual title you find on the linked page. The HN post is entitled merely "tunni.gg".
But if you click on that, you get to a page that says, "Expose localhost to the internet." But the poster couldn't be bothered to put that important and interesting information in the title. Instead, the title is worthless.
You see plenty of similarly and intentionally obscure titles on HN daily. Try calling them out and see what happens.
> people who point out obscure titles are downvoted in most cases, and eventually shadow-banned
Nothing like this happens! Nobody gets banned for pointing out anything about titles. People only get banned ("shadow" or otherwise) for serial abuse or trolling (and only after multiple warnings), or for spamming. Comments only get downvoted if more people disagree than agree with the title suggestion or the way it's suggested. It's no big deal. It's how opinions are expressed and debated on HN.
> The HN post is entitled merely "tunni.gg".
That's Tunnl.gg [1], and it would have been fine for the page's heading to be added to the HN title (that routinely happens when software projects on Github are submitted). It's also not terrible for just the project name to be there, because the name of the project (a variant of the word "Tunnel") hints at what it is. But we're not dogmatic about it, and anybody could have emailed us (hn@ycombinator.com) to suggest a better title we would have given it due consideration and replied appreciatively. We do that multiple times each day.
> You see plenty of similarly and intentionally obscure titles on HN daily. Try calling them out and see what happens.
“Intentionally obscure” isn't the right framing. Maybe we don't always want to clobber people over the head with obviousness. The joy of surprising discovery is an important part of the HN experience.
But the key principles – (a) respect the original work of the author/publisher and (b) don't mislead or disrespect the HN audience with clickbait or false information – have proven to be the most stable and defensible over time. There's still plenty of room for discernment in the way those principles are applied on a case-by-case basis.
Direct experience.
"AV1 open video codec now powers 30% of Netflix viewing, adds HDR10+ and film grain synthesis"
Re: HDR - not the same thing. HDR has been around for decades and every TV in every electronics store blasts you with HDR10 demos. It's well known. AV1 is extremely niche and deserves 2 words to describe it.
It's fine that you haven't heard of it before (you're one of today's lucky 10,000!) but it really isn't that niche. YouTube and Netflix (from TFA) also started switching to AV1 several years ago, so I would expect it to have similar name recognition to VP9 or WebM at this point. My only interaction with video codecs is having to futz around with ffmpeg to get stuff to play on my TV, and I heard about AV1 a year or two before it was published.
One word, or acronym, just isn't enough to describe anything on this modern world.
you might not know what AV1 is, and that's fine, but the headline doesn't need to contain all the background information it is possible to know about a topic. if you need clarification, click the link.
I'm not trying to be elitist, but this is "Hacker News", not CNN or BBC. It should be safe to assume some level of computer literacy.
And now HN administration tend to editorialize in their own way.
We generally try to remove numbers from titles, because numbers tend to make a title more baity than it would otherwise be, and quite often (e.g., when reporting benchmark test results) a number is cherry-picked or dialed up for maximum baitiness. In this case, the number isn't exaggerated, but any number tends to grab the eye more than words, so it's just our convention to remove number-based titles where we can.
The thing with this title is that the number isn't primarily what the article is about, and in fact it under-sells what the article really is, which is a quite-interesting narrative of Netflix's journey from H.264/AVC, to the initial adoption of AV1 on Android in 2020, to where it is now: 30% adoption across the board.
When we assess that an article's original title is baity or misleading, we try to find a subtitle or a verbatim sentence in the article that is sufficiently representative of the content.
The title I chose is a subtitle, but I didn't take enough care to ensure it was adequately representative. I've now chosen a different subtitle which I do think is the most accurate representation of what the whole article is about.