Netflix's bet on advanced video encoding
theverge.com
theverge.com
What's the point of all that, if I can shoot 4K on a $2,000 camera and it's going to get compressed heavily anyway.
I actually think streaming would be best for lower budget productions, I'm not fully engaged when I watch something on Netflix anyway. It's kind of strange to see them spend $200 million or so on a movie knowing that I'm doing a million other things while it's on.
Just noting that the official guide[0] lists a number of cheaper cameras, including one that retails for $2k (the Panasonic BGH1).
0: https://partnerhelp.netflixstudios.com/hc/en-us/articles/360...
I'm more disappointed my cheaper Canon isn't eligible. When I think about it, there's no reason every single streaming movie even needs to be in 4K. It's going to be compressed heavily anyway .
I agree. I'd rather have better audio and actors not whisper their lines, forcing you to either turn on CC or turn up the volume. I wouldn't want to purposefully stream in 4k anyway - I don't really have a reason.
The content I watch on YT is, IMO, much more interesting than the netflix shows anyway, and recorded with much less expensive gear.
Variety's writeup has a bit more technical detail on it - https://variety.com/2016/digital/news/netflix-meridian-imf-t... .
It's pretty clever, many of the shots seem clearly designed to stress-test common encoder issues and failures -- they put thought and care into it, and it shows.
Or you could use a 4K stream from Netflix (only available to premium premium subs) and encode it down to HD and likely get a better image.
This is absolutely true for all the streaming platforms. My Jellyfin/Plex servers with pirated content are set up in a fairly ridiculous way: I use sshfs across multiple seed boxes located on different continents to "merge" the filesystems in a way that is transparent-enough to Jellyfin that one server will serve up all the content.
Even with this somewhat-absurd daisy-chaining, I can serve much higher bitrates to my home than Amazon, Disney, Netflix, Hulu, etc. The encoding quality is much better and even completely video-naive guests who know nothing about technology notice the difference and comment "Wow, the picture quality is so good!", especially with anything like Dune 2 or Planet Earth. Even when the video gets transcoded on the fly for device-compatibility, while it's degraded, it ends up still being better visual quality than the major streaming services.
The streaming platforms are definitely optimizing for minimizing their costs rather than serving decent bitrates that the user-agent can handle.
If you really care about visual quality, I do reccomend trying to get a membership to PassThePopcorn, or rip content from BluRays yourself, which you can rent from libraries. Ripping is really not much effort once you learn enough to do it once or twice.
I'll start using the major streaming services again when I can actually receive not-mangled visual quality from them.
Added to that, you can comfortably overload an ISP's peering and send infrastructure costs rocketing.
Practicalities, things might work on an individual scale, but when you have millions upon millions of users, it creates problems.
Even if you look at torrent content, you can find a given movie anywhere from a few gb (1080p) to a few dozen gb in size at 4k with higher relative throughput and quality. Not to mention the encoder settings.
Personally, I don't go for the highest quality encodes because I don't see the difference well enough. I tend to favor x265 over x264 only because the blur degredation is less annoying than blotchiness imo.
When you are streaming, the level of encoding will swap out, even at the same resolution a number of times depending on what you're getting. You may well do better with a 1080p stream over a 4k stream depending on your screen and device's upscaling behavior. 4k takes at least 4x the bandwidth, and has tighter latency needs. There are others that can go over this and explain it much better than I can, but it's not malicious, it's partly cost sure, but it's also a matter of what your connection can handle.
Edit: Netflix changed this sometime in the past half years: https://help.netflix.com/en/node/30081
Whether it's because Netflix locks the good high-bitrate encodes to the most expensive subscription plans many people aren't on, or because their "Roku 4K" or "Fire TV 4K" or whatever is actually negotiating the 1080p-H.264-low-bitrate stream, or because their internet connection latency was auto-detected too low, or their device doesn't have enough memory to buffer nicely or whatever.
I bet Netflix is absolutely capable of delivering something really nice. But in my experience, Netflix quality as delivered looks about on-par with "YouTube free 1080p" -- so much so, that even plugging in a 1080p-only-Blu-Ray, is a huge improvement even non-techie folks can notice.
I‘d choose the blu-ray any day.
The optimistic take here is that the purpose of this encoding work is to make Netflix accessible consumers in ALL countries- not consumers with beefy internet.
IMO the realistic take is that Netflix knows it can do this while saving $$$ serving lower quality video to consumers who cant tell the difference.
I got into programming/software by encoding my cough well-sourced cough movies/TV shows to MP4 for my iPod video.
Far be it from me, maybe it's insufferably geeky detail, but the slow decade-long march described as "gee each movie is different" and "gee each scene is different" followed by Herculean work of FAANGers insufficiently appreciated by creative types was solved by VBR years upon years earlier. (VBR = variable bit rate)
Once you're getting to "we'll use ML as a VBR algorithm!", that's original, but the problems described and solution was understandable and solvable by a 18 year old non-programmer in 2007 with free software.
VBR wasn't some niche thing either, it's a very very obvious optimization I've never seen a codec miss, from MP3 audio to MP4 video. There's no caveats here or haughtiness or flippant "Dropbox is rsync + my weekend" dismissiveness on my part. It wasn't news to _anyone_, it's a very obvious optimization that was applied by everyone
I'd be veeeeery curious if there was much contribution here beyond using x264, occasionally with patches, and then engineering a pipeline around it
I kind of look at this in the same way as the fact that so many movies or TV shows can’t help but eventually have a plot where someone makes a movie or a TV show, maybe at a stretch a stage production. And, somehow, all the characters know a whole lot about stage productions.
Even better they „researched” better metrics like pVMAF so they can again show how good they are, in theory.
Or are they just aggressively searching for corners to cut to save bits?
I can't wrap my mind around this article and exactly what's going on because AFAIK VBR has been default ~forever. You don't even need multipass for that because you just have to look ahead N frames, it was received wisdom, statistically backed, as far back as 2007 that multipass was a waste of time because it was extremely marginally, like .0001%, better than just doing VBR, and using CBR was really dumb and wrong unless you had very specific edge case technical needs. I can't even remember what, maybe poorly designed initial H.264 decoders or people feeding into legacy broadcast system
Little more context here: https://news.ycombinator.com/item?id=40770734
i.e. slice it not just into scenes but also into objects and do bitrate on that level. i.e. Face and objects in foreground get more. It seems we now have pretty small models that can do that sort of stuff (see Apple & MS ones recently) so should be feasible at scale.
I'd imagine you can also train an LLM on patterns that encoders choke on...chequered patterns etc.
Encoders have been doing that for decades at this point.
AI-related models are actively being researched for multiple applications in video encoding. From generating predicted frames, to generating fine details, choosing tools and parameters for each frame, narrowing the search space, etc.
https://deeplobe.ai/exploring-object-detection-applications-...
Literally getting closer to understanding the context of the scene and what matters to a human rather than understanding the pixels
https://deeplobe.ai/exploring-object-detection-applications-...
If the model can figure out the key objects in the scene on a bit more deeper level then that can drive what parts the encoder needs to spend more bits on
I'm not sure if they've actually shipped anything yet, though.
[0] With blackjack, and hookers: https://blog.chiariglione.org/the-mpai-framework-licence-app...
From the eyes of the viewer: Attention-aware video compression for improved low bitrate video https://medium.com/the-svt-tech-blog/from-the-eyes-of-the-vi...
Did you ever test this on a larger scale? And do you know if it is only applicable to one encoder?
- Netflix will only send you "high quality" (over 720p) streams on certain browsers (IE on windows, Safari on OSX, ???? on Linux) that support Encrypted Media Extensions.
- Netflix also auto-scales the quality they are sending based on their understanding of your connection.
- Also apparently they do per-title re-encoding passes.
This all combines, in my experience, to on average the worst streaming quality overall and also the most opaque. There are debug modes you can enable to see some of this, but generally it's very hard to tell what quality you are looking at and what is preventing you from getting a nicer quality. I also find that Netflix's "low profile" content (i.e. not max quality) looks bad - i.e. the 720p stream looks quite bad in addition to being low resolution.
Netflix dropping to potato quality after a few minutes is infuriating. Refresh the browser and quality goes back up for a few minutes before it drops again. My wife often doesn’t notice it; it drives me nuts. My theory is that they try to save bandwidth by dropping quality after some time and hoping you don’t notice.
They won't send you anything at all unless you have EME, coupled to a supported DRM engine. Getting >1080p requires a "more secure" DRM engine (e.g. Widevine L1, PlayReady SL3000)
* https://support.google.com/youtube/answer/3306741
* https://support.google.com/youtube/thread/89050028/my-movie-...
How do they maintain the quality long enough to rip the content? When I used Netflix years ago, I could barely keep a constant 1080p stream at times.
They both (afaik) do hybrid releases by taking e.g. audio or DV metadata from one service and merging it with another source.
Funnily enough, even for UHD remuxes, P2P groups are top-tier (like Wildcat and Framestor).
Also they often contain every subtitle and audio track (often 40 subtitle tracks!), which would be either a lot of work or not possible at all if they couldn't extract the stream properly. I suppose they might be able to get those from the low quality unencrypted streams but it just seems implausible.
They release shows pretty much immediately as soon as they are on netflix, don't they?
It seems improbable to me that they are systematically recording and reencoding every show manually, although I guess it could be possible in theory.
A friend of a friend tells me that they've never seen UI elements or quality drops on any scene rips.
All signs seem to point to the encryption being broken in my opinion.
Are subtitle tracks even encrypted? Unlike video, there's no secure path (eg. HDCP) for subtitle (or even audio) tracks, so they're presumably trivial to rip.
I believe this specific method is outdated, but my understanding is that current methods also bypass drm instead of using screen capture.
[0] https://googleprojectzero.blogspot.com/2017/07/trust-issues-...
[1] https://nvidia.custhelp.com/app/answers/detail/a_id/4682/~/s...
I’ve got a bookshelf computer hooked up to a 52” 1080p screen in my living room, nothing fancy, and I’ve never noticed any issues at all with quality, or buffering times. I just open the video through the website in chrome, and play it, and it’s fine.
[1] https://chromewebstore.google.com/search/netflix%20higher%20...
They would be much more concerned about the quality on their app on smartphones, tablets and smart TVs.
This is the future. You can multiply a pixel as many times as you want.
They really managed to squeeze the sound quality to a minimum.
It's not only netflix that does this, all of the streaming services do (though some of them limit to 1080p instead of 720p).
Such an utterly pointless anti-piracy measure... every show ends up on torrents in 4k regardless, only serves to punish those actually paying for the service.
Rapidly oscillating patches, blurry smears, it’s only Netflix (other video streaming services are fine) its getting quite annoying, a few times it’s been so severe I thought it was a problem with my projector
Enshittification continues relentlessly.
It's just rampant monopolization.
Enshittification makes it sound like it's a natural expected process, for companies to buy their way into dominance the market, destroy competition, then deceive and underserve customers while raking in undeserved profits.
It also fails to lay the blame on Congress and the Administration's doorsteps, where it belongs.
I feel the correct frame is the general category of "market power" (including, but not limited to, the "anti-competitive practices" that forms the basis of current regulatory policy).
In the case of streaming, the closest analogue to monopolization is vertical integration: the custom-produced content which is only accessible on a particular streaming platform (sometimes including no releases on physical media).
On the one hand, Netflix could be considered to have a monopoly over the distribution of "Squid Game" (and Prime over "The Boys", Apple over "Silo", etc), meaning they can raise prices, not for the product itself, but for the "club goods" distribution platform, which the consumer may not be interested in otherwise.
On the other hand, entertainment/culture are not the same kind of "inelastic demand" necessity as telephone service in the past, or internet service now. And when we look at actual outcomes, I'd say quality for vertically-integrated productions are better on average than older business models; and while prices are going up, I wouldn't call them ridiculous price-gouging either (in addition to competing with each other, the streaming services are also competing with piracy and password-sharing).
Anyway, I think it's absolutely the case that enshittification isn't inevitable, and we should be wary of succumbing to learned helplessness in the face of capital consolidation. I would claim it's a "natural expected process" of the shareholder corporation, and the perverse incentives of short-term profits and stock prices, even when it comes at the expense of long-term brand equity. But solving that general problem is probably out of scope for "the bitrate is too damn low and the subscription fees are too damn high".
I'd love to see more regulation in the space, but realistically it'd be tricky (I don't expect any Doctorow-approved abolition of DRM, or mandatory interoperability, anytime soon). One could perhaps establish streaming-quality standards, where a company couldn't advertise "4K" unless it hit some threshold. But when it comes to the issue of streaming quality in particular, I suspect the tragic reality is that the majority of users simply won't notice or care.
German Court Fines Netflix €7.05 Million for Continued Infringement of Broadcom HEVC Patent (2023)
https://www.broadcom.com/company/news/product-releases/61711
Are we surprised?
You'd think netflix could do better than the ad-hoc groups of individuals who do it for free in their spare time.
I would have thought Netflix could add an x265, VP9 or AV1 stream for the best quality streams though, like youtube does.
Netflix has never streamed 4K content in h264 for example.
Then they have no excuse.
Netflix is not focusing on quality but saving their delivery cost on bandwidth, which may hurt them later surprisingly.
Well, that't the problem, isn't it ?
Netflix has to squeeze a (bigger and bigger) profit for every byte transferred over the network.
In many occasions, Ad-hoc groups of individuals who do it for free in their spare time provide the best service.
It's because their interests align perfectly with individuals, and not corporations looking to enshittify and squeeze every dollar out of people.
we've already crossed the line where it creates a garbage viewing experience for the end users, with pixel porridge everywhere and absolutely horrendous visuals in the darks and shadows for every movie and series
netflix engineers are failing (on purpose) to create a decent viewing experience for their users
Every Netflix show I've seen in the past year (n=3) has had these crazy panning drone shots of forests in winter, with really high f#'s, graphics of a million particles exploding, shots of ocean waves at dusk... basically, they are shooting video encoding stress tests and then encoding them very poorly. The result looks like dogshit.
When people say "refreshing their browser" it sounds like they're watching on a PC. That's probably the worst platform to watch on these days, from a market share point of view. Just saying.