Modern codecs like AV1 can bring better quality video to the open web
blog.mozilla.org
blog.mozilla.org
The problem with H.265 is the IP situation is a mess. AV1 looks to be better in every way. Looking forward to its adoption. So far the pirate scene doesn't seem to be using it at all.
The REAL blockers are licensing and, more importantly, legacy consumer hardware. Whether you want to call the latter a speed or power problem isn't the real issue, it's that not all consumers have the dedicated hardware and the older CPU then has the speed and power problems.
I've found that I'm not able to encode video above the size of the VRAM on the GTX 1050 in my home server. It seems to spew out errors, despite the GPU's memory usage being roughly 60MB from ffmpeg
My GTX 1080 meanwhile seems to happily churn through anything and everything thrown at it
Still, the benefit to the millions of people who might download and watch a video surely outweighs the cost of the one individual computer that spent 10x the amount of CPU cycles encoding it.
That's expected on new devices at some point next year.
Pirates, quite understandably, don't care about patents or IP. They can be a good indicator of whether there is any other reason to use AV1 over H.265.
There's development happening here: https://aomedia.googlesource.com/aom/+log/refs/heads/master/...
Maybe not. There are still normative (i.e. changes affecting the spec) cases in the bug tracker:
https://bugs.chromium.org/p/aomedia/issues/list?q=label:Hotl...
Private torrent communities are already reluctant to use x265 which provides marginal gains compared to x264 at a great increase in encoding time.
A part of the process in pirate movie encoding (p2p, not really scene) is testing optimal encoder parameters (adaptive quantization strength, psychovisual rate-distortion, quantizer curve compression etc) on short clips meant to be representative of the whole movie (e.g. 60 frames every 4000 frames). If encoding a single test clip takes an hour as with x265 that sure puts a damper on things.
About the only place in movie torrent communities where x265 makes sense is 4K encodes as x265 can retain HDR metadata (h264 can too to an extent with the 2017 revision to T-REC H264 but it's hardly supported anywhere).
This may be different for pirate _streaming_ services.
This is easily explained by 1) no hardware acceleration, and 2) the reference implementation is currently intended for clarity and ease of changing, not efficiency or speed. After the bitstream solidifies, both problems will be solved.
> Private torrent communities are already reluctant to use x265 which provides marginal gains compared to x264 at a great increase in encoding time
Because there would be a clear quality loss. The websites they're ripping from (amazon, netflix, whatever) are using H.264, and to recode into H.265 would lose significant quality.
This is wrong for two reasons: the source codec doesn't matter (it gets decoded to raw yuv420 first anyway), and netflix/amazon/UHD BluRays already use HEVC for 4K which I alluded to earlier.
Pirating scene hasn't been using av1 at all because the bitstream hadn't been stabilized (an encode you made last month probably won't work on versions from this month) and, as a result of work still being done on the bitstream, no encoding hardware or software optimizations are available making it on the order of 3 magnitudes slower than existing options. Towards the start of 2019 you should start to see more hardware support pop up though.
Do the benchmarks include animation? Perhaps flat surfaces like one sees in anime compress a lot better than average in x265?
On the development and academic benchmarking side any comparison worth it's salt should have more than 5 different categories of video style. The paper linked in the article based their results on 20 different sequences, 3 or 4 of which looked to be animated.
However, all modern codecs that I know of have directional intra prediction, which should be pretty effective in anime. Basically, what it does is that the decoder is able to follow lines with a variety of angles.
There are other things that help compression in anime: static frames, large flat areas, lack of film grain in computer drawn animation.
With the supposed savings of HEVC versus h264, and the rather widespread HEVC hardware support in devices, I would have guessed that they would have embraced it at a larger scale by now.
Will be interesting to see what happens with AV1.
The MPEG LA is just a licensing administrator formed by companies owning MPEG patents, they didn't standardize anything nor are they responsible for MP4 container.
The standards working group is MPEG, and according to Wikipedia the two are not affiliated ( https://en.wikipedia.org/wiki/MPEG_LA ).
I’m also very impressed with HEVC, but mostly from the angle of UHD Blu-rays. The quality is remarkable. For the same size or slightly larger than 1080p Blu-rays (around 50GB, sometimes up to 70), you get 4k resolution (4 times the pixels as 1080p) and HDR.
I railed against "scene" groups for many years over the quality (or lack thereof) in releases they put out. To the point I simply started buying the media I wanted, ripping it myself and then converting it to HEVC
Doing as close to a 1:1 DVD rip as I can, I've shrunk things like Seinfeld from 152GB of MPEG-2 video to just 31GB in HEVC, and it still looks absolutely fantastic
HD video fares even better. The X-Files on Blu Ray is roughly 1.7 terabytes. HEVC encoded brings it down to 240GB or so
Edit: as I suspected, they are reencoding from other groups. At least they make it clear I suppose:
"Preacher.S03E04.The.Tombs.1080p.WEBRip.6CH.x265.HEVC-PSA 447mb.
Source: Preacher.S03E04.The.Tombs.1080p.AMZN.WEBRip.DDP5.1.x264-NTG | 2.38 GB"
No way that could look as good as the x264 and the video has been encoded three times, amazon->NTG->PSA.
I really wish the next gen codec, H.266 VVC could sort this royalty mess out before hand.
Why cant we keep things simple. USD $1 Per devices Royalty on Hardware encode and decode, that is roughly $3B + Royalty every year for the next 20 years.
Software Implementation should be free. So an H.266, VVC based image and video decoder could be immediately rolled out to all PC, and browser vendor could support it.
One of the problem with current MPEG codec is that they are too worry about free software implementation take over their royalty. Well we don't have huge increase in IPC anymore. We have a roadmap from TSMC that shows us the next 10 years of leading edge transistor is going to be more expensive then ever. We don't have any breakthrough / theory on hand they we could make those a lot cheaper. The days of Moore's Law are long gone.
This allow hardware maker to sell "faster" video and picture decoding. Consumers are a lot easier to spend money on physical object, then paying $x for faster software. Once they felt how slow their images and video were, it give them an incentive to upgrade, PC, Tablet, Phone etc.
Sure, that would be the optimal situation.
However, for a variaty of reasons you sometimes have to deal with overly compressed images. Adding noise on the client during play back disguises compression, making it nicer to the eye at no bandwidth cost.
Encoders generally have a few options to tune the grain that gets added back in to get it as close as possible to the original source.
A link would be nicer than an apparently very specific search term.
That is confirmed by this article: https://jmvalin.ca/papers/AV1_tools.pdf
In other words, it is complete nonsense, texture synthesis, like the way some people go for printing a digital image on a fake "canvas" for a "textured, 3D look"... or maybe one of those flickering flame light bulbs.
Can anybody here imagine an audio codec that added simulated hiss and crackles to make it sound like you were listening to a vinyl record? That's what this is.
Film is the same. Directors choose the film type and grain patterns that match the look and feel they want. A more grainy film feels darker and more serious. Splotchiness contributes to an unrefined or maybe even psychotic feel.
You don't want your encode removing that. You want the atmosphere of the visuals to match the original, because the director likely had that atmosphere in mind and made other decisions around it.
It's kitschy, ersatz, whatever you want to call it. I used to spend a lot of time with artist filmmakers (many of whom are very nostalgic about celluloid) and there is no way in hell they would accept this solution. It wouldn't meet their standards of authenticity. It also "fakes" a medium-specific property in a way that is unprecedented in audio or visual coding. Their solution would be to either code at high enough bitrate to capture the grain, or insist on screening on physical celluloid.
That might sound extreme and unrealistic—it's what artist's film people are like—and even a bit snobby. After all, a lot of the options out there for adding film grain effects (or scratches etc.) to video are not intended for use by professional filmmakers, depending on how you define professional. There is definitely an element of snobbery to the statement that film grain should never be faked, whether by compositing something onto your home video or in a hidden way inside AV1.
I hope you can see that I respect the director's prerogative in choosing grain, I just don't think this AV1 grain synthesis methodology is sound from an aesthetic/authenticity point of view. It's digital faking of an chemical/analog effect, which IMO makes it unavoidably kitsch for reasons to do with old modernist ideas of "medium specificity".
The problem with matching the grain perfectly without the randomization is that it drives the bitrate crazy. A movie with 99% fidelity grain and 0 psy-rd will have a bitrate of 30mbps+.
If you use psy-rd though, you can get to 99% fidely grain at closer to 20mbps. The two screenshots will be visually indistinguishable, even when you are rapidly switching between the source and encode, even though you know that the grain is being randomly generated you can't tell.
If I get a chance later today I'll drop some comparison screenshots for a film encode that used a lot of psy-rd, I think the results will surprise you.
And adding digital grain "back in" doesn't sound very authentic to the original analog source.
Besides, "authentic" is such an overhyped pile of poo anyway. Everything is a reproduction; what matters should be that the artistic intent remains unaltered.
I feel a ramble coming on, but I believe movies should be available in a version close to how they were viewed on theatrical release.
If you think about it, the first 100 years of this artform is always going to be special.
Yet this article from Mozilla seems like that from June the situation might have changed now that a 1.0 stable spec is out. Given that inspection for infringement has been part of the codec creation process it seems promising that they will be safe. But patents have proven themselves to be funny things.
That said, have there been any updates since June from lawyers/other blogs/etc about the patent situation yet?
The graph starts at 50%?
And they didn't mention in the Moscow State University test, AV1 was 300x to 1000x slower.
While I like competition from AV1, whatever they are doing in terms of marketing really doesn't resonate with me.
Reference implementations of codecs are always slower. AV1 hasn't had the benefit of years of optimizations yet, so this slowness is to be expected.
Plus, mobile devices will use embedded fixed-block ASICs for this, just like they already do for every existing codec. All codecs are completely impractical on mobile devices if you're basing your estimates off software numbers, even extremely well optimized software will kill your battery. So it's basically meaningless to try and extrapolate from optimized numbers, much less preliminary ones.
Besides, encoding AV1 is what's expensive, but mobile devices also do not necessarily need AV1 encode support anyway. They will need decode support. Slow encode support is going to be a problem for people creating and serving video. But better compression rates from AV1 substantially help mobile devices, too, by reducing bandwidth needs for providers. You're likely to see major video content providers taking the hit of AV1 encoding, while users only need to decode, and this will largely be how it is used.
So people aren't using Snapchat, Instagram Stories, FaceTime, IGTV ?
Because pretty sure encoding is just as important as decoding.
Assuming a gargantuan encoding slowdown, even in the face of hardware-based encoders on mobile devices (which, again: hardware decoding/encoding is the fact of the matter on how it will happen), then there's no reason to support AV1 encoders since it will hurt battery. You're better off with something else, because users care more about their battery than compression ratios or minor artifacts on their Insta videos.
Assuming no gargantuan slowdown, and devices are equipped with AV1 encoders that perform admirably, then your problem is solved. Just use the hardware encoder. (Personally, it's unclear to me how much of AV1 encode support is fundamental slowness that can't be mitigated by hardware and implementation, but I suspect a lot of this fear is tied up in early-stage numbers. I bet it can go much faster, even if it's still slower overall.)
If you're a media provider like Netflix for example, though, the bandwidth savings will utterly dwarf the cost of the more expensive encoding process. They encode once and serve thousands of times, and they don't run Netflix (the company) on old mobile phones. So they'll do it anyway, because they want their users to use AV1. It saves them bandwidth on a high quality video, and it saves users on their data plans and their battery.
This isn't really very hard to think about. Either AV1 encoding is fundamentally too inefficient for mobile, or it isn't. If it isn't, users can still get fast decoders (that wasn't the bottleneck), since the bandwidth savings are going to be highly desirable for everyone, basically, and they'll be pushed for. If they can be efficient enough, then there's no problem.
These issues unless solved relegates AV1 to being a Youtube and Netflix format and pretty much irrelevant for everyone else.
Reference implementations are not about performance, they are documentation. They are not supposed to be used for anything else but documentation.
Your conclusion is ridiculous. There is absolutely not reason assume that av1 is fundamentaly more power consuming that HEVC.
A tank uses more gasoline that a VW Beetle. This statement is true, and as irrelevant as your statement.
That is valid for all H.262 , H.263, H.264 and H.265, as well as the up coming H.266 VVC. The current Av1 isn't a documentation. Nor it is a Reference Implementation in the regards of all MEPG codec. The AV1 is built on a working, professionally made libvpx encoder built on VP9.
>There is absolutely not reason assume that av1 is fundamentaly more power consuming that HEVC.
It definitely will be more complex, both encoding and decoding. Its fundamental complexity is much higher then HEVC, how far could they optimise so they are close to 5 - 10x of HEVC is a different story.
I like it when the goalposts move in such a way that "used by services in over 50% of US households and services where 70% of all video usage come from mobile, and offers excellent bandwidth improvements for millions of users" apparently means AV1 is a failure, somehow, based on preliminary numbers for a use case (encoding) that largely isn't relevant to mobile devices (and will improve dramatically over time making encoding more usable, anyway, as all prior codecs did.)
Unfortunately for the AV1 designers, they could have realized this fatal flaw sooner, had they only consulted the grand expertise of..... a random internet forum user.
The development of the codec has been done with constant input from the hardware developers who are backing AV1 (Intel,AMD,Broadcom,NVidia etc) in order to make it effective when used with hardware acceleration. Just watch some of the developer presentations on Youtube.
Your fear that all the streaming and hardware giants would cooperately develop a codec that won't be effective on mobile hardware is unfounded.
https://medium.com/netflix-techblog/more-efficient-mobile-en...
YouTube and Netflix adopting VP9 before others doesn't make VP9 irrelevant for everyone else. The same is true for AV1.
It's maybe a little early in terms of available computing power (and how optimized encoders currently are) to think about AV1 mobile encoding.
I have an iPhone 7 which has no hardware VP9 support but VP9 video works well at 720p in VLC. I'm sure the battery won't last as long as playing H.264 video in hardware but I wouldn't call VP9 playback impractical.
My battery started at 90% and at the end of three hours it was at 55%. I think that's pretty good for software decoding.
I'm not sure which VP9 decoder VLC uses on iOS. I presume it's ffvp9:
https://blogs.gnome.org/rbultje/2014/02/22/the-worlds-fastes...
I wouldn't be so quick to dismiss it - video chat is a great use case for an improved codec.
I would highly, highly argue that they do. There is no reason in modern day that we need to all be uploading the full size of images and videos to servers to be re-compressed. It creates a significant barrier to entry for any social media platform when it should be as simple as the device itself being able to encode it, with the server checking it and creating other sizes. If AV1 becomes the great standard we can reduce a lot of costs. We already could do it with jpeg compression, but hasn't seemed to happen.
Slightly less compression rates for faster compression is perfectly acceptable. I would hope theres profiling options for that.
The trend is overwhelmingly towards mobile being the dominant platform for video. And if your videos can't play on the majority of the iOS and Android it's simply a non-starter. Look at what happened to Flash once Apple banned it on iOS.
Yes, the reference implementation of encoding is slow. Being fast is not a high priority for a reference implementation. Optimized implementations will come later. ASIC implementations will come later. The H.265 reference implementation was similarly slow, and it’s not a problem. And mobile devices primarily care about decode performance.
Please provide evidence. Because all other data has shown it to be 100x slower.
There is a reason why people are obsessing over encode speed.
Also there are already an alternative AV1 encoder in the works, RAV1E, written by Mozilla in Rust, so there will be competition which will likely speed development, and the x265 developers have stated that they will create an AV1 encoder if that's where the market goes, which at this point seems very likely given that all the major players are backing this codec, coupled with the fact that it's royalty free and thus can be implemented anywhere.
> Please provide evidence. Because all other data has shown it to be 100x slower.
I got this!
H.265/MPEG-HEVC is the latest video coding standard... it's well known that reference implementation of HEVC codec, HM, acts an important role during standardization... However, HM is far from a practical codec because of very slow coding speed even on modern multi-core computers.
https://ieeexplore.ieee.org/document/7041782/ (Dec. 2014)
Today you can achieve >200fps with CUDA-based encoders.
Not sure why this issue keeps getting brought up. It's baseless and completely untrue.
It's royalty-free, but what that really means is that contributors are licensing their AV1-connected patents to everyone for free. That's all fine and good as long as AV1 isn't found to infringe on patents owned by non-AOM members.
VP8/VP9 both were patent encumbered and required licensing from MPEG-LA. It's going to be interesting to see what happens if AV1 does take off whether the big MPEG-LA patent contributors decide to go for royalties.
> big MPEG-LA patent contributors decide to go for royalties.
There are enough big backers of AV1 to beat back racketeers who will try to stop it.
They were not. MPEG-LA tried a little FUD to slow down the adoption of VP8/VP9.
https://www.ibc.org/delivery/codec-wars-the-battle-between-h...
Does x86/x64 have any? What's the name of one of the instructions (for example)? I feel like mostly all I see is just bigger vector instructions...
With Apple the problem was not so much webm, but the vp9 codec, which is the codec most webm containers use (see youtube).
ie, codecs: av1, vp9 <> h.264, h.265 containers: webm <> mp4
I really hope an open format will win "the war", but I can't see it work at scale if it doesn't support streaming.
https://bitmovin.com/constantly-evolving-video-landscape-dis...
Twitch wants to use AV1 for their streaming. They want to use the switching frame feature of AV1:
https://www.youtube.com/watch?v=o5sJX6VA34o
And the chroma from luma feature of AV1 works well on video game content: