MPEG: A crisis, the causes and a solution
blog.chiariglione.org
blog.chiariglione.org
The majority of the work was done by the grad students of the professors who are on the standards committee. Private companies then race to patent or buy up anything that's tangentially related during the draft period, so that they can sign on and receive royalties after standardization. Many of these companies are owned by or have board members who are standards committee members. The whole thing is a giant racket.
Licensees were getting tired of this, so they started to do the same thing, and now you have multiple patent pools, no clear guidance on which ones are legit, and even fewer people who are willing to play along.
Though obviously, everything's too good to be true. Just like MPEG seems to be hitting the end, some day the AOM will hit the end, too. The world changes. Something else will emerge and impact the business model. It may be worse, or maybe it'll be a return to norms. Could just be another one of those things that cycles every so often.
With audio codecs, it doesn't look like we ever really needed much in the way of commercial entities, since Vorbis and Opus have been outperforming the competition for a long time now.
Sorry to be pedantic about one bit of an otherwise-fine post, but AAC is comparable to Opus at any given bitrate[1]. Plus, even the most esoteric flavor of AAC has the enormous advantage of 7+ years of ubiquity[2].
[1] See the multiple charts at http://opus-codec.org/comparison/, for one presumably-unbiased source.
[2] All versions of Android support HE-AACv2 playback, iOS introduced support for HE-AACv2 playback in iOS 4 (released 2010), macOS introduced support for HE-AACv2 playback with iTunes 9.2 (released 2010), Adobe Flash Player introduced support for HE-AACv2 in 2007, All versions of Windows Phone support HE-AACv2 playback, all open source players support HE-AACv2 playback via FAAD2
That being said, I still think Opus is better now, since it is more versatile. I've yet to see anyone use HE-AAC in a real-time context, unless they're using it under a different name.
Opus is definitely better, but we have basically hit the wall in audio compression (we'll need a serious breakthrough to go much farther). There also isn't as much interest in audio codec research because even very large raw audio files pale in comparison to fully compressed video.
The developers are slowly reducing the required bitrate for transparency, it wouldn't surprise me if they reach transparency at ~64kbps VBR in a year or two.
https://i.imgur.com/zxwOBMJ.png
Etc.
Discussion of 1.3-beta here:
https://hydrogenaud.io/index.php/topic,115156.0.html
Discussion of preferred bitrate and point of transparency:
https://hydrogenaud.io/index.php/topic,115386.0.html
https://hydrogenaud.io/index.php/topic,114656.0.html
I would highly recommend that you give it a go yourself. Take a FLAC source, encode Opus files yourself. The encoder is freely available from https://opus-codec.org/downloads/ and integrates nicely into Foobar2000's converter function (Opus is already there as a profile, it will ask for the location of opusenc.exe the first time you use it).
Use the ABX Comparator plugin for Foobar2000 (both available from http://foobar2000.org) and run a few trials.
After all, public listening tests are one thing, but to be absolutely sure a format/bitrate is transparent to you, individual testing is needed.
I have run a number of tests myself (of Opus and other formats) and found general transparency at ~96kbps Opus, lower for some tracks. For those tracks that weren't transparent at that bitrate, the artifacts were generally benign and not that annoying (slightly increased noise levels, mostly).
Generally Transparent = MP3 LAME @ ~ 128kbps. That is the good old standard.
There has been PR and marketing materials, much like Video Codec making claiming they are 50% better then H.264 or AVC or whatever. AAC or MP3Pro even claim to be MP3 128kbps quality at 64Kbps. Of course which never happened even after years of fine tuning.
Opus is the first and only codec in years, or decades that had better music quality then MP3 @ 96Kbps. Even AAC cant do that, at least not in majority of cases.
I have come to the conclusion that Audio compression has come to end of the S curve, with diminishing returns. It literally took us all the years till all MP3 patents has expired to get an Audio codec that is better at it with 20% less bit rate.
I very much doubt we could have 64Kbps VBR to sound better without many more breakthroughs.
Says who? And which version of LAME? That encoder has seen huge improvements over the years. We should not define audio transparency by an outdated and flawed format. We should define audio transparency by whether it is distinguishable from a lossless source or not.
But yes, a modern version of lame (3.98 or newer) will be transparent around V5 VBR, which is ~130kbps. BUT -- and this is a huge twerking 'but' -- the MP3 format suffers from pre-echo. There is nothing you can do about it. It is lessened at higher bitrates, but you can never completely get rid of it. There is also the problem of the SFB21 defect, which can severely bloat the allocated bitrate on content that is heavy in high frequencies: http://wiki.hydrogenaud.io/index.php?title=LAME_Y_switch
Newer formats such as AAC, Vorbis and Opus do not share these limitations. I don't know where you read that AAC is worse than MP3, because that is simply objectively false, as shown by every listening test you can care to dig up. Did you mistakenly use FAAC (probably the worst AAC encoder out there) or something?
https://i.imgur.com/zxwOBMJ.png
I know it's one listening test out of many, but they all agree. AAC (with a good encoder, such as Apple's) at ~96kbps is better than MP3 at ~130kbps.
FFmpeg/libav don't do HE. FDK/libaacplus are non-free. neroAacEnc always used to have the best quality, but that's shareware.
Further nitpick: the page is very misleading. Audio compression has different targets, a crucial one being music.
The Opus official page has comparatively little test references for music, both of which are at 64/96kbps for AAC [1-4], which is significantly below the general audience listening standards (160+ kpbs).
This implies that the red line on the top right of the first diagram is literally made up.
Until further blind listening tests are performed, generic conclusions such as "Opus outpeforms the competition" are unfounded.
> It attempts to summarize results [...] and (when no data exists) show anecdotal evidence
This is borderline offensive for people who are serious about researching music quality. Any anectodal evidence [in this field] is garbage.
[1] http://listening-tests.hydrogenaud.io/igorc/results.html [2] https://people.xiph.org/~greg/opus/ha2011/ [3] http://listening-test.coresv.net/results.htm [4] http://opus-codec.org/static/comparison/GoogleTest1.pdf
Also, I happen to know a few very moral and ethical business owners who would rather lose money than screw someone over or do a shady business deal. (and have done this)
So your comment is also missing a major part ofthe picture.
In places where the research is done like you said, all your points are true, but IMO institutions such as the Fraunhofer institutes are worth keeping.
Also note that I never claimed that the patent income wasn't being put to good use by funding research. I was arguing that many of the patents themselves were bullshit, and that many of the companies that own them had acquired them from universities on slightly-too-friendly terms because of the MPEG cartel.
The alternatives are government funded research, or companies like Google doing it all internally, but then Google will keep some of it as trade secrets, which is also undesirable.
I don't think there's an alternative if your goal is go keep all research publicly available and in a way that any company can easily license it.
If you act like a cartel, generate reams of questionably valid patents so that you can hand them out to your member corporations under sweetheart terms and at the expense of your licensees, you're eventually going to piss off enough people to make the music stop.
Evidently, this is what happened. The patent pool is now so large that it's really easy for other players to come in with their own, similar patents. They can use these to create new patent pools and try to take everybody hostage with higher fees (HEVC Advance), or they can build a protective moat and threaten to sue anyone who demands licensing fees from them.
Patent licensing can be made to work in a lot of cases, but these guys really dug their own graves here.
Also, I think there are a number of alternative funding routes that can be made to work. Government funding, as you mentioned, could be one of them. Terms of the funding could stipulate that the research be made publicly available and licensable under reasonable terms.
Companies can also choose to fund the research via Fraunhofer rather than doing it all internally. Not everybody has Google's resources, and there are major advantages to having improvements included in a standard that everybody can access.
Patent licenses are only a tiny fraction of their finances.
https://www.fraunhofer.de/content/dam/zv/de/publikationen/Ja...
2016 they made 134 million € from patent licenses, about 6% from their total income of 2.1 billion €.
Of which about two thirds come from either public grants or direct government funding.
But of course, government funding is today one of the largest sources still.
I simply posit that (software) patents actually aren't needed for that. Or to be more specific: economic exploitation of their patents - nowadays, I could e.g. imagine them to offer their considerable patent pools for defensive purposes, to "disarm" patent minefields in problematic fields. Of course, that would be a tightrope walk in regards to cooperation with private entities...
The use of patents by Fraunhofer is to ensure that the research is actually real-world usable (and the government isn't just wasting their grants) and that it is worth it to the industry, while at the same time ensuring that the industry helps in funding the research that it will make profit fron
And another edit: One could argue that's a side effect from the modern anglo-american tradition of patenting (that has seemingly spilled over to continental europe) that almost completele ignores "Erfindungshöhe", stripping patents of their inherent technical merit leaving exploitability as the only measurement.
The Helmholtz-, Leibniz- and Max-Planck-Societies exist to do science for science's sake. They build stellerators and map the oceans, because it provides knowledge.
The Fraunhofer soxiety exists to turn theoretical research into something companies can directly implement. That is what separates it from all the others. It is supposed to provide finished templates and blueprints with its patents, so you can license something, and have it ready to use.
That is why Fraunhofer didn't just develop an algorithm for mp3 and aac, but developed production ready encoders, decoders, hardware en- and decoders, 5.1 decoders and stereo, even simple ones for mono. They created the source code, heavily documented, and the algorithms, and blueprints for building them in hardware.
So that companies can license it, just integrate it, and are done.
That is why the Fraunhofer society exists, because our economy of small to middle businesses can not compete in R&D with giants unless we do all the R&D in such a research institution licensing ready-made solutions to your small companies.
And yes, the US-view on patents with regards to not needing implementation detail and no requirement for Schöpfungshöhe (in fact, the US system even has the opposite of that) is severely damaging this concept.
(Ironically - in contrast to defense against patent violations/litigation, where having Fraunhofer's back is certainly a big benefit! :) )
I mean, we are in a time where e.g. even Sony (of all companies!) freely licenses their LDAC codec to the AOSP. To me it seems that after decades of countless multi-front patent wars many finally start to admit they are sick and tired of it. (Esp. when looking to the rising China, which seems to do well at the innovation front while not giving much of a damn about patent protection...)
Or is this guy just an incredibly skilled troll with 30 years of experience in pulling the wool over people's eyes? I don't think so, though. He really seems to think that imposing massive amounts of friction on a system somehow makes it work better.
Wow. Just wow.
I mean, I'm not unsympathetic to the argument that patents can promote advancements by funneling money to inventors, however weak it starts sounding when you look at actual events in certain spheres. But the claim seems to be that nobody is going to bother to research improvements in an interesting, impactful technical area because they'll no longer be able to (1) invent something, (2) hire an army of lawyers, then (3) profit. Er, okay. Let the people who will only work with that sort of path go do their ICOs instead.
I suspect that he believes that the perpetual funding stream of ever-better patents is needed to fund the research for ever-better codecs (which, as you say, is disputable but not unreasonable). I also suspect that he would argue that some of the problems of the protection racket amounts to a necessary evil (a protection racket does give you a viable means to combat submarine patents). However, the AV1/HEVC situation undermines these arguments, and this is someone who realizes the gravy train is running out of rails and is desperate to keep it from derailing instead of bailing while he still can.
And doesn't Firefox have some special deal with Cisco to use their x264 decoder on the web?
HEVC is currently literally impossible to license. One of the pools hasn't decided on pricing yet. You basically have to set aside some cash and hope it's enough to back-pay once they announce the pricing. But even with the existing ones, you have to pay for encoders, decoders, and videos encoded in it. https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#P...
If you consider compression from an information theory point of view, for any given measure of "quality" there is theoretically a limit to compression. It's not clear where the limit is, or how close we are to it, but because it does exist stagnation is inevitable eventually.
As it gets harder to make gains it will become less of a business proposition and more of a research/academic thing.
Besides, for a company like Netflix or Youtube, shaving 5% off the file size without a drop in quality can literally mean millions of dollars saved per year.
Usually I'd expect someone in this position to toe the party line pretty hard, but he actually sounds reasonably level-headed to me. He even cautions the reader not to get their hopes up that anything will improve.
Fuck the Fraunhofer Institute.
There have been a lot of mistakes in the growth of the internet, but mpeg... Trying to own an entire space isn’t about making the world a better place. It’s about being in control. Me me me.
Given how big video is now, and the interplay between available bandwidth and video quality, one wonders if certain bumps in the road would have been smoothed out if Netflix or YouTube had ramped up a few years earlier.
I'm convinced many big players wanted people to think mp3 was going away to push newer, patent encumbered formats.
I don't think is correct. The driver for innovation in video compression technologies will continue to be the practical problems of encoding and distributing video. The quest for smaller video file sizes while maintaining quality is what has driven the development of VP8 to VP9 and now to AV1. Smaller video file sizes directly help reduce the costs of companies like Google, Amazon, Netflix, Hulu, Apple, etc. and I don't see that changing any time soon. There will be an AV2 eventually.
For example you need hardware acceleration for it to be a feasible alternative on mobile.
they can try to keep an advantage by not releasing their most cutting edge encoder.
Or have major platforms (realistically, Windows, macOS, iOS, and Android) ship decoders at the platform level, and have them ubiquitous enough. This is, ultimately, how we got H.264 decoders commonplace in browsers.
That said, Firefox held out for a long time with H.264, and only once it was clear it was costing them users did they give in and add H.264 to the whitelisted platform codecs exposed.
But even then MPEG (LA) didn't really make any money out of that because IIRC they pulled a hack where Cisco is providing a decoder binary, which they distribute. But the license fee has a cap which cisco was already almost at anyways, so that didn't cost cisco much.
Then they have (had planned?) reproducible compilation. So the decoder is open source, but you're not allowed to ship a binary that hasn't been compiled by cisco, but you can always verify the binary cisco provides matches the source code, as the resulting binary will be exactly the same if your compile it yourself.
Cisco essentially sponsored an open-source royalty-free decoder here.
So yeah, they gave in, but MPEG got nothing from them.
Good. Fuck MPEG LA.
OpenH264 is only used for WebRTC and, importantly, there's no AAC support (which doesn't matter for WebRTC because you can just use Opus). Firefox uses the OS platform's H.264 decoder for video element playback.
> which cisco was already almost at anyways, so that didn't cost cisco much
It apparently cost Cisco quite a lot more in licensing fees but they considered it worth it for the sake of integration with their video conferencing hardware. See the comments to this blog post from Monty Montgomery for some discussion on it:
https://xiphmont.dreamwidth.org/61694.html
https://xiphmont.dreamwidth.org/61694.html?thread=439550#cmt...
And Firefox supporting H.264 through the platform decoder came a while before OpenH264 happened.
Today, there is an advertising and entertainment industry generating huge surpluses that can also bankroll engineers to develop codecs. I’m not convinced that this is a better model.
Vorbis was released in 2002. That's 16 years ago. That's hardly 'until recently'.
Tarkin also suffered from the patent minefield everybody else put up (so they couldn't go for the obvious solutions because these were already locked down for another 15 years)
And even lossy video codecs had some OSS experimental developments in the early 2000s that admittedly never went too far - Tarkin predates VP3 open sourcing if you want to count that, and Snow got somewhat decent results if you ignore that H.264 predates it.
What do you think: How many engineers can be paid for reducing Netflix's traffic by 10% simply by these lowered traffic costs, and not counting secondary effects as a larger user base because more people can watch the videos with their available bandwidth? Also YouTube, Amazon Video, public television, ...
I have been paid by Mozilla to do this in some capacity for a decade. The budget for our entire team (which is the largest it has ever been), is less than the cost for just Mozilla to license H.264, by a non-trivial margin.
Here are some of the things that team has done:
- Made a Theora (VP3) encoder easily superior (in quality/bit) to what Youtube was shipping in Flash at the time (VP6), and (favorably) comparable with H.264 baseline.
- Developed and standardized the Opus audio codec.
- Provided infrastructure and maintainence for Xiph's older codecs (Vorbis, Speex, FLAC, etc.).
- Provided some minor assistance to VP8 and VP9 development.
- Made a from-scratch video codec (Daala) competitive with HEVC while avoiding the patents (i.e., with one hand tied behind our backs).
- Made significant contributions to a video codec superior to HEVC (AV1).
Now, maybe you want to call all of that almost nothing, but this code is in every Apple and Android handset in existence, and every current desktop browser, and used by some of the largest websites in the world (Youtube, Wikipedia).
The major concern with open source development is the free rider problem: why should Media Company X spend resources on an open source codec when Google will build one for them?
But the value current patent holders are attempting to extract from an individual licensee is larger than the cost for them to develop the technology from scratch by themselves. That value isn't created by the technology. It's created by having a common standard. And we don't need patents to incentivize that.
This is fine – in a macroeconomic sense – but of course it sucks if you're one of the companies being disrupted.
The logic of software patents systematically underestimates the existing incentive structures. That's because of the simplifications economists usually make. (See https://en.wikipedia.org/wiki/Assume_a_can_opener)
The assumptions underlying patents become less justifiable as
1) the pace of innovation in an industry increases, compounding first-mover advantage;
2) complexity increases, because integration of an invention into a viable product becomes much more difficult--often such that copyright and other IP protections alone are more than sufficient to provide a moat;
3) as financing becomes cheaper and potential markets larger the time horizon for and amount of ROI demanded by investors diminishes;
Those are especially applicable to many modern high-technology industries, software (Silicon Valley) and industrial (SpaceX).
Other industries have proven to be able to thrive without strong patent protections or even copyright protections for myriad other reasons. The fashion industry has thrived without strong patent or copyright protections, largely based on trademark and marketing. Likewise for the beverage industry and consumer foodstuffs more generally. Major automakers had poor copyright protections, and while they had stronger patent protections, as safety regulations increased they shifted to a market segmentation model such that the big auto manufacturers often freely license patents among each other long before the patent expires.
Economists, especially conservative economists, often argue against anti-trust, reasoning that as markets become larger (finance markets, consumer markets, etc), the typical lifetime of a monopoly become shorter--often shorter than the typical lifetime of anti-trust litigation.
I would argue that as markets grow larger and more fluid they're far more capable of discovering and leveraging sufficient incentives to support innovation in the absence of government-imposed monopolies. Especially in the realm of software, the circumstances where there's a legitimate market failure requiring patents are so uncommon that there's no reason to even entertain the notion of a software patent system.
In the absence of _manifest_, _systemic_ market failures, patent advocates make the same mistake Communists make--an inability to imagine how a market can support an endeavor is not evidence that a market is incapable of supporting that endeavor; it's merely evidence of one's own lack of information and imagination. And the surest way to introduce market failure is by dangling the prospect of government-imposed monopoly rents, artificially increasing the opportunity costs of discovering and leveraging all the non-obvious incentives that naturally exist or could evolve.
The value of the patent system is that it permits decoupling. Compare the Wintel monopoly to the decoupled ARM ecosystem. It wouldn’t be better if mobile chip R&D had been bankrolled by Google’s say search business or Apple’s cell phone business instead.
> The value of the patent system is that it permits
> decoupling. It wouldn’t be better if mobile chip R&D had
> been bankrolled by Google’s say search business or Apple’s
> cell phone business instead.
Your argument is a variant of the argument that by creating a transferable property right, we reduce the transaction costs of negotiating access to inventions. My point is that those transaction costs are dramatically overestimated (don't confuse complex or opaque for costly), and in any event far exceeded by the unintended costs of a patent system.
Importantly, when it comes to complex technology a patent no longer embodies the requisite know-how. People and companies don't lose much by giving away their secret sauce because very often the only people capable of applying and productizing it in a timely enough fashion to capture the lion's share of profits are the inventors themselves. Moreover, the best way for a market to discover who is best at leveraging an invention--if not the inventors--is by making it more freely accessible.
Also, the shift to SaaS and cloud computing means that complex systems never leave the control of their creators, anyhow. This is happening despite the fact that the patent system is supposed to incentivize disclosure.[1] Worse, the patent system makes it easier to stop competitors without substantial disclosure of the end-to-end systems. You can patent all the small sub-inventions, permitting you to block competitors, without every having to disclose the larger, integrated whole that makes it functionally useful in the market.
Even if the patent regime ensures that we get an MPEG codec that is 10% more efficient (or arrives 10% sooner) than an equivalent codec that would arrive in the absence of patent protections, is that worth allowing MPEG rights holders to effectively extort money from smartphone makers?
Because that's what we're talking about here. In this day and age, patents don't make innovation commercially viable; at best patents making them viable a little sooner and a little better. But the _costs_ are enormous, and I would argue far outweigh any meager benefit.
[1] Similarly, film and TV producers pursue DRM despite copyright protections, and despite the fact that DRM seemingly provides little marginal extra profits. The transactional costs to Hollywood of negotiating DRM would seem huge. But either the costs are less than they seem on the outside, or they're already making such huge profits that they can afford to try to lock down the market even more. But copyright was created to reduce the transactional costs of negotiating a locked-down distribution system. If companies are pursuing DRM anyhow, we really should question our presumptions about the necessity fo government intervention. For copyright we probably need some minimal protections (weaker than we have), but we can probably forgo patents altogether, especially in the realm of software.
Again, just because it seems that a patent regime would make a market more efficient doesn't mean that it would. Our measurements of efficiency might bear no relation to what a free market needs to maximize benefits. Communists thought they could devise a more efficient market in grain production, but obviously they missed or misjudged a few necessary incentive structures that allowed for a self-sustaining market (including secondary markets, etc) that maximized output.
There are many reasons for developing better codecs, whether they perform better on hardware, they allow for more channels in a pipe with fixed bandwidth, or optimize the viewing experience for particular types of signals.
When I read the reference post I heard the echos of rationalizations that have been used for years and years to justify applying rents against someone else's labor. I can only hope that the author is correct in that this rent seeking behavior has become unsustainable. That would be a good thing indeed.
And nothing of value was lost.
Also, that the indirect value of having an open-licensed codec to the big players in AOM is far more than the possible license revenue.
that's quite a lot of FUD. hosting providers have very aligned incentives to invest in better - and not necessarily proprietary - compression technologies, as do ISPs who have to carry the packets.
> The situation can be described as tragic.
I'd call the situation liberating. Tragic for parasitic patent aggressors, but great for actual technology progress. Choose what you prefer.
> AOM will certainly give much needed stability to the video codec market but this will come at the cost of reduced if not entirely halted technical progress. There will simply be no incentive for a company to develop new video compression technologies knowing that it assets will be thankfully – and nothing more – accepted by AOM for use in its video codec.
That's false, because instead of patent trolling, proper company would be adding resources to the common pool, benefiting in result from it. I guess the author finds the concept of collaboration to be foreign, but it actually pushes progress much faster than patent driven pocket lining.
> Can something be done?
MPEG can fix it going forward, and push for royalty free / patent unencumbered technology only.
DASH is not royalty-free: http://www.mpegla.com/main/programs/DASH/Pages/Intro.aspx
It is, the above is just a list that trolls are assembling to intimidate others. They have it on many thigns, including established royalty free codecs. If they sue someone with it and prove in court they actually own patents on DASH - then it will be another story. Until then - intimidation doesn't equal ownership.
In this industry, such intimidating tactics (very protection racket and mafia like) are commonplace. Check the story how Qualcomm and some others tried to intimidate Opus project.
That's what happened to VC-1. Microsoft tried to release VC-1 on royalty-free terms but the MPEG LA formed a patent pool and then it was no longer royalty-free.
Probably not. The better way is to do a review and have ready rebuttals of fake patent claims. I didn't exactly look into how it was done for DASH, but its big users like YouTube and Netflix should have done it.
> The MPEG LA has been successful with their other patent pools.
And completely unsuccessful with many others. There was the same type of list put out by MPEG-LA trolls for VPx, and I remember there was research published that debunked it all. So there is zero trust in anything that MPEG-LA say, but surely having concrete defense is a good thing.
No. They didn't succeed in forming a patent pool for VP8 or VP9. It was one of their "pools in formation" but was never established. The DASH patent pool is beyond that stage.
And if they'll fail, there should be some concerted effort to make a patent unencumbered alternative.
It is indeed true that the MPEG approach is dead. Once AV1[2] is out, the open, royalty-free codecs will be the better performing codecs for both audio (Opus[3]) and video.
So yes, if MPEG is dead, its components are free to join AOMedia, as long as they leave their silly RAND ideas behind.
[1] http://aomedia.org/.
> Alliance for Open Media (AOM) has occupied the space left free by MPEG: outdated video compression standard (AVC), no competitive Options 1 standard (IVC) and no usable modern standard (HEVC). AOM’s AV1 codec, due to be released soon, is claimed to perform better than HEVC and will be offered royalty free.
The primary use is to display media streams with reduced network, CPU, and battery usage. But it's a much more general field, replete with all kinds of fascinating insight about the nature of visual/aural manifestations and human perception. Anything you can remove yet still recover the relevant parts of the original signal is an opportunity to understand how our brains, and the world in general with its physical laws, work. There's a ton of interesting and useful research that could be done -- but would you really want to look into how texture is encoded, processed, and reacted to in our neurons when you have to worry about various mafia-like patent licensing consortiums? The whole licensing setup has a chilling effect on many different area of inquiry, even apart from its impact on classic compression algorithms. (Though that is harmed too -- if you come up with an incremental improvement that works really well for some type of input, and you'd like other people to build on your work, what do you do?)
well mp3 was mostly frauenhofer, not mpeg (yes they created the standard, etc. but the most patentable work was done by the frauenhofer) also the date... on 1992 it was not (yet) called mp3. it's silly to actually name it in the mpeg sense...
Edit: not a fan of mpeg, but if I would be them I would try to form a non profit and actually try to collect money like that instead of being a super duper not obvious big licensing company. I mean H.265 was released in 2013 and it's still not as widely used as it could be and better fully open source alternatives emerge quicklier than ever.
We also saw open standards emerge, like Ogg Vorbis and FLAC (although neither were widely adopted by hardware vendors compared to Apple's AAC/ALAC)
Standard MPEG-1 MPEG-2 MPEG-4 Visual MPEG-4 AVC MPEG-H HEVC
Bitrate vs previous 25% less 25% less 30% less 60% less
More advanced video codecs have higher visual quality at lower bitrates, but are more computationally expensive. All of these video codecs use the same underlying ideas of 2d transforms, intra- and inter-frame compression, motion compensation, entropy coding etc., but using more complex techniques to preserve video quality.
I wouldn't say it's a simple as "Moore's law in action".
So yeah, there were obvious gains left on the table for H.264 for complexity reasons that H.265 picked up; likewise for H.265/VP9 that AV1 picked up.
That said, there has been a lot of refinement that didn't necessarily increase complexity, and there have been various tools that saw reinvigorated research once complexity targets were raised.
I believe this is total BS. There are great open technologies(i.e http/http2) and they didn't need a extortion cartel to police anyone who dares to use them without to buy a license(if they could buy it in the first place). Codecs should be open and they can be developed/sponsored by people who benefit most(i.e. Media companies, internet companies, hardware companies etc)
To argue there will be no innovation in this field without patents is similar to saying there could be no innovation in Linux or Apache without restrictive, object code only proprietary licensing.
I'm only happy about this. And I seriously doubt this conclusion: "There will simply be no incentive for a company to develop new video compression technologies".
It seems the author of this text thinks the only way to have an incentive to develop new technology is patent fees. This is outright ridiculous.
Companies like Google or Netflix obviously have an incentive to further improve media codecs, because transmitted gigabytes translates into costs for them.
https://ngcodec.com/news/2017/10/21/why-we-are-supporting-vp...
The simplicity of VP9's licensing versus the complexity HEVC's licensing probably has a lot to do with that.
Meanwhile, nothing iOS supports VP9 HW decode. Until they come around, people could give a fuck about installed base.
Then there is the stuff this article doesn't very well measure. Like DVB-T2 in Germany using 1080p50 10 bit HEVC. That is expected to be around for a decade or more. That is a whole lot of TV settop boxes.
The amount of Android devices dwarfs iOS devices and Apple knows that it is not viable position in long term. That's why they joined AV1. People in near future would not give a fuck about the Apple, if they couldn't play their Youtube or Netflix videos in 4K on them.
1. AOM was formed in 2015, and the idea of collaborating between Mozilla and Google and others were very well known within the industry before the formation of AOM. But let's just call it 2015, so three years later in 2018, the head of MPEG, just a year ago ( 2017 ) has seen this a threat coming.
May I ask What was he doing between 2015 and 2017?
2. >all the investments (collectively hundreds of millions USD) made by the industry for the new video codec will go up in smoke and AOM’s royalty free model will spread to other business segments as well.
Collectively Hundreds of millions? If there were only 10 companies paying the max cap of 25M per year, that is $250M per year from HEVC and to MPEG LA alone, excluding the other two groups and NPEs. And that is devices maker, excluding content producers. Assuming a video codec has a life time of 10 years, that is a total of $2.5 Billion minimum. Do you not think wasting hundreds of millions to save a few billion for the industry is a very good deal?
I do not known how the US and Canadian University works, but at least what the article said is true in terms of Universities funding in EU, Cooperate Sponsor etc for the work. Well at least it was true in my era of studying.
And I do not have a solution to this problem.
And I guess I cant really blame MPEG, what was originally an outrageous amount $25M now seems fair and reasonable compared to what others were asking. It is likely the greed that every company has for their IP portfolio wanted more.
As a matter of fact only a few were made in AV1 because the two codec were so different.
UPDATE: Looking at the mailing list archive, there isn't any activity there: http://lists.xiph.org/pipermail/daala/
The code repo is also pretty quiet: https://git.xiph.org/?p=daala.git;a=shortlog
Video decoding is weird because while the software might be sound, I have no idea how the hardware supports it. I don't know if the hardware has to be specifically built to handle a family of codecs, or if it is a kernel matter or else...
The only projects I can actually think of that are based in the US and made by american authors are those by xiph.org, and they only get away with it because their entire business model is developing royalty free alternatives to MPEG codecs.
Even if multimedia patents might not affect big corporations much, they definitely seem to strongly affect the open source community. I imagine if we had similar dystopian laws here in the EU, our best and most beloved multimedia software would plain not exist.
So a good portion of the startups you're imagining die early due to a lack of investment or incorporate in the US at some point in order to have a better chance of finding investors. And then some of the rest simply aren't as growth-focused because they need to focus on short-term profitability to survive, which means they'll likely end up silently dominating a particular industry niche rather than making a big entrance on the global stage.
Software patents are only a problem if you can survive long enough to be sued. If you're not exceptionally unlucky, you're more likely to go bankrupt before that happens.
There's an interesting loophole: just also include in your invention the machine (memory + CPU) running the algorithm. Then you're patenting a device, instead of simply math.
Stallman described this method more than 10 years ago. I was hoping this wasn't true, until the (french) company I work in successfully used it to patent an underlying algorithm from the software I had written.
Maybe such a patent would be debunked in an european court? I'd be glad if this were the case.
I worked in video for a several years - sorry to hear about your issues.
Good.
“Can something be done?”
Yep. Search for a new job!
In the past it was common that researchers would go to research institutes such as the Fraunhofer institutes, and do research there. The research would be published, patented, and licensed. From the fees, the Fraunhofer institutes fund themselves.
Now this is being replaced by companies doing the research in-house, where it's often kept as trade secret, or, if it's patented, never licensed to competitors. This limits innovation.
I'm not sure how we can protect the German research societies, but it'd be a shame to see this continue. Research done by companies is inherently unfair, and gives them a competitive advantage.
In this case, of course, Google opened their codecs, but for example with Waymo's self-driving technology, that won't happen. Ideally, every carmaker could license the best selfdriving technology as soon as it exists.
This is relevant to this discussion because the Fraunhofer group has previously created mp3 and AAC, and those patents have funded hundreds of further openly available research projects