I appreciate that it costs money and time to develop these algorithms, but when you're backed by multi-billion dollar "partners from industry including Apple, Ericsson, Intel, Huawei, Microsoft, Qualcomm, and Sony" perhaps they could swallow the costs? It is 2020 after all.
In the early days of MP3, all MP3 rippers and players were built off of their implementation.
Hardware and software companies had to license in order to play MP3 files. As such there was not native support for MP3's for quite some time.
In the late 90's right around the explosion of MP3's on the internet, Fraunhofer was going after companies for doing so.
In my humble opinion, that license mess set back innovation in the portable audio space by a good 5 years.
Seeing all this, I'm convinced that copyright in general and patent system in particular does more harm than good by slowing down the technical progress of the humanity as a whole for the sake of some already rich people becoming a bit richer.
The initial idea behind patent system was sensible, but the way it's abused now... I mean, it could work in today's world as intended if patents lasted a year or two, not what is effectively eternity.
There are plenty of societies that don't respect intellectual property and copyright. And those societies don't innovate at the rate as those who do.
There are certainly abuses in the copyright, trademark, and patent systems. But throwing out the baby with the bathwater is not the answer. Identifying the abuses and improving the system is the answer.
Perhaps that progress won't happen at the rate you'd prefer, but it's significantly better than burning the whole thing to the ground.
Changing patent length (pc's suggestion) is hardly seems like burning the whole thing to the ground.
In the US, if I understand correctly, there are ways to defend against patent abuse, though it typically involves costly legal fees. Fees many cannot justify in spending.
Addressing patent challenges may be a way to protect innovators, both those who should gain reward for their innovations, and those who seek to grain reward by building upon innovations.
That's the key: In the U.S. Good luck stopping someone infringing on your patient in a good number of other countries.
What it does do though is allow someone to start from zero, and catch up to the rest of the competition very fast and cheap. They can then offer their "product" cheaper in order to gain market share. As long as they are getting/keeping customers, there's no need to innovate. You can have a viable business without spending tons of cash on R&D, and if you're making money that way, who cares?
*I am no way endorsing this kind of business model, but it exists and does well.
For one, this claim suffers from a correlation/causation issue. But also, do you have an actual citation for research which shows this is true?
Comparable examples being ?
That's not necessarily so - for example, during the industrial revolution, Germany largely overtook the British in mechanical engineering skill during a period where the British had copyright but before the Germans got it: https://www.wired.com/2010/08/copyright-germany-britain/
Also, it's quite possible the causation goes the other way: A society might not succeed at innovation because of IP laws - it's just as possible that because a society has succeeded at innovation, it passes IP legislation, in order to 'kick away the ladder'. But just like regulatory capture shows in general, legislation that helps yesterday's winners seek rent is not necessarily the same (and is often in fact the opposite) of legislation to help tomorrow's winners see the light of day.
That sounds nice and reasonable, in theory.
In practice, all the money is behind expanding the copyright and patent systems. When is the last time the duration of copyright terms was shortened? When is the last time the patent system was adapted to be less draconian and less protective of those poor, poor multinational corporations that somehow end up holding all those patents?
Spoiler alert - that has happened exactly never. Instead, all we get is 'harmonization' which always means extending terms and giving those laws more teeth, to match the strictest law implemented anywhere in the world.
Steamboat Willy's copyright expires January 1st, 2024. How much do you want to bet that Disney will be pushing for another copyright term extension before then?
> Perhaps that progress won't happen at the rate you'd prefer, but it's significantly better than burning the whole thing to the ground.
The current systems only ever get stricter. Where are the much shorter terms? Where is the recognition that cooperation and remixing fosters innovation and progress and as such should be encouraged, not punished? Where is the PTO following the actual law that says math and logic (i.e. software) are not patentable? Etc, etc, etc.
The abuses have been very well documented over the past several decades. There has been zero progress on incremental improvements. Tell me again how you propose we improve the system without a drastic overhaul?
I'm not 100% sure about this. Just look at how hard China laughs at intellectual property and copyright in general and tell me if you still think the same.
Not being burdened by expensive license fees is a competitive advantage.
But then again, also not having to care about workers right is...
This needs to vary based on field. Some areas, like drugs, take forever to get to market and have exorbitant development costs to recover. (Though, there, other abuses need to get fixed, like renewing patent lifetime with slightly different applications or formulation.
Unfortunately, most companies require that any patent awarded to an engineer in their employ is automatically assigned to the company.
Get rid of that loophole and employees will be able to license their patents as they see fit. Of course this is fraught with practical difficulties, but some kind of compromise could be reached.
Never happen, obviously.
Or did it push it forward? If not for the licensing at the time, would it have been developed? Would it be allowed to be used by anyone just paying for the technology?
Fraunhofer has a budget of over 2 billion Euros, and 30% of their money comes from public funding. They run over 70 institutes, so they do much more than this.
The root problem here is that nowadays public research is funded with industry money, which means there has to be a return on investment, hence the patents. In fact, this has metastasized into universities being graded by their patent portfolio volume. So I would expect there to be patents even if 100% of the funding came from the tax payer.
It would have been possible to do the whole process just with public money and zero patents. In fact, I would love it if some research team collated all the patent tax payments across the population of Germany and compare the bottom line cost for the country.
I wager it would have been cheaper without patents, too.
Fraunhofer gets roughly 30% of its funding from public sources, the remainder is raised on a per-project basis. It's a fair assumption that those industry partners provided some funds towards the development here. Maybe they even covered all the payroll costs for the involved scientists for the duration of the project.
And yet more income means more money for other research projects. Maybe ones that are not as commercially interesting, or for which a partner decides to terminate a contract rather unexpectedly. While I am also a fan of OSS and would love for work like this to either have no patents or a liberal patent grant, I can also appreciate the desire to fund your research institute.
Why would the taxpayer fund anything that isn’t open and free. Crazy.
The argument that anything funded by tax money must be open is a very fair stance. Though the line gets very blurry when you mix various sources of funding like this. To the best of my recollection I have yet to be paid from any public funding (rather than project specific funding raised from the industry, for example).
Personally, I have no qualms with the funding model, but other points of view are presumably equally valid.
At least the patent licenses usually used with MPEG mean that private use of open source implementations is free.
Why is that?
Now compound this by the fact that a) trying to make an exhaustive patent search to get a verifiable claim that you don't infringe on any patent is very problematic b) known patent pools like MPEG-LA are known not to cover everything.
So you can make a reasonable bet that you avoid infringing patents by avoiding patents from MPEG-LA and few other better known groups, but you can't actually guarantee that you're not infringing on any patents.
This resolves, sort of, into a game of chicken and depends heavily on whether a lesser known patent holder decides it's worth it to bother executing against you... but even if they don't, unless they come out with a royalty-free license, the possibility of patent is a Damocles' Sword hanging over your codec.
Is there even such a thing?
Isn't the problem that one has to actually go to court to get the answer to this question?
In fact, I heard more than once that current advice is to explicitly avoid searching :/
The way to get around this is to exist in the EU and avoid providing anything to the US.
Care to try your luck?
I don't think they'll be successful. The Alliance for Open Media was careful to avoid potential patent problems during AV1 development. So, unless AOMedia seriously failed in that effort, AV1 will be alright.
The patent system was not designed to have every tiny little technique patented, and this is its failure mode.
In the USA. Wilful and unwilful infringement of the patent costs the same in Europe.
They can be reasonably sure they do not infringe known patents from certain Patent Pools and patents declared as part of MPEG-LA bundles. They can't provide reasonable data that they do not infringe on any submarine patent, something that killed 3 attempts by MPEG-LA to provide a royalty-free codec for the web - all that was required to kill it was a note from a company that they "might" have patents covering things they tried to release, or that they decided not to allow royalty-free license for their known patent.
Meanwhile patent search is complex enough that it's unreasonable to impossible to make a statement that you definitely don't infringe any unless you keep clear of anything invented within last ~20 years.
That's true of every new video codec. It didn't stop the use of H.264 or VP9 or even HEVC.
Multiple companies have now rolled out AV1 into production. We'll see what happens.
What's wrong is claiming that AV1, VP9, VP8 are "patent-free". They are not.
The existence of Theora, VP8, VP9, and now AV1 seems to contradict that theory.
You could argue that they infringe on some unknown patents, but that is also arguably true of patent cabals like MPEG (you just hope that the cabal is big enough that there aren't any patentholders lurking outside). The only difference is that with a patent cabal you have the fun of having to obey the restrictions of everyone who showed up with a possibly-related-in-some-way patent and joined the cabal.
Not to mention that it isn't necessary for a patent pool to be a cabal. AOMedia has a similar structure to a patent cabal except it doesn't act like a cabal (its patent pool is royalty-free in the style of the W3C). So even if the argument is that a patent pool is a good idea (and video codecs cannot be developed without them), there isn't a justification behind turning the patent pool into a cabal.
> At least the patent licenses usually used with MPEG mean that private use of open source implementations is free.
You say that, but there's a reason why some distributions (openSUSE for one) still can't ship x264 (even though the code itself is free software). Not to mention the need for Cisco's OpenH264 in Firefox (which you cannot recompile or modify otherwise you lose the patent rights to use it). The existence of the MPEG patent cabal isn't a good thing, and any minor concessions you get from them do not justify their actions.
And retaliation clauses are present in basically every free software license that has clauses dealing with patents (including Apache-2.0).
I'm more and more fond of calling it all a big game of chicken, a MAD without nukes,
The video patents aren't just "patent troll" patents, either. They are highly enforceable, and were registered by corporations like Ampex.
I have been trying to write a simple app to stream RTSP (security cameras), and that has been a pain.
I need to basically use either proprietary (paid) or GPL software to do it.
Video software is not for the faint of heart. Much as I grouse about the licensing, I am not about to develop my own codec.
I did write this one app, which is an ffmpeg wrapper, to convert RTSP to HLS (Which is not -currently- suitable for realtime streaming): https://github.com/RiftValleySoftware/RVS_MediaServer
It's GPL, because I need to use the GPL ffmpeg H.264 codec.
edit: Your project reminds me, https://github.com/arut/nginx-rtmp-module is super worth checking out and might be helpful to you.
One of the frustrating things about implementing video software, even licensing it, is pretty much everything out there is an expression of ffmpeg, which is a really, really good system, but does have some baggage.
> It's GPL, because I need to use the GPL ffmpeg H.264 codec.
I don't understand the problem?
I can see the problem with people patenting things and preventing you from writing your own implementation, but it seems you just want other people to do the hard work of implementing it so you can wrap a skin around it and do what?
It was a statement of fact.
I've been writing open-source software for well over 20 years (actually, well over 30 years –Where does the time go?).
I think I understand the issues involved.
Apparently, they aren't. There are at least 2 other patent pools that claim patents for HEVC, and I think I saw 3 in some other article before:
https://streaminglearningcenter.com/codecs/hevc-ip-mess-wors...
Ultimately, it's a question of how much you're gonna risk to get where you want, and how much power/influence/wealth you can bring to squash a possible lawsuit.
To completely avoid risk your only choice is to to use old technology where all the patents have expired (20 years in US), like MPEG-2. The next lowest risk is to use H.264 and VP9 which have been out for a while and whose the patent pools have stabilized over the years (and the original parts of the standard will have their patents expire soon - but not some of the newer profiles). After that I would argue that AV1 is less risky that H.265 and H.266, as a lot of work was put into intentionally avoiding patented technology that was not part of the pool, and no one outside the pool has yet made patent claims against it.
EVC baseline is basically that, only using technology based on H.264 that are already expired or soon to be expired and patented techniques from companies that are giving it away to the standard.
In some way EVC is even more exciting than VVC.
"1.3. Defensive Termination. If any Licensee, its Affiliates, or its agents initiates patent litigation or files, maintains, or voluntarily participates in a lawsuit against another entity or any person asserting that any Implementation infringes Necessary Claims, any patent licenses granted under this License directly to the Licensee are immediately terminated as of the date of the initiation of action unless 1) that suit was in response to a corresponding suit regarding an Implementation first brought against an initiating entity, or 2) that suit was brought to enforce the terms of this License (including intervention in a third-party action by a Licensee)."
This makes it much harder for practicing entities or their licensees to assert claims against other practicing entities over the formats.
Edited because I didn't know that some European countries accept software patents.
For example, many countries in EU do not allow patents on software, but that's not something you can claim to be true for all of them - at least before Brexit, since iirc UK was pretty happy to provide software patents.
Then there's a case where if you're really willing you can, as far as I understand, force a patent dispute through WTO, with possibility of patent valid in USA being executed for example in Poland, despite the fact that the patent is invalid in Poland (it doesn't matter if your software is part of physical solution in Poland, algorithms of any kind are not patentable).
The question is, does a "practicing entity" holding the patent cares enough to go through the hardest route to get the patent executed using WTO as a forum? One needs to compare costs and benefits. It's why patent trolling involved pretty much few counties in Texas, because that's where the costs were lowest compared to benefits.
https://en.wikipedia.org/wiki/Software_patents_under_the_Eur...
Not a private organization.
Look at all those European patents in the MPEG-LA license pool!
Wanna take the risk? You might end-up winning the lawsuit, but at this moment, there's a good chance for you to be already out-of-business.
- in Europe: http://www.bailii.org/ew/cases/EWCA/Civ/2002/1702.html
- in US: https://web.archive.org/web/20061205050434/http://eolas.com/...
But as I said, if your startup is being sued by Dolby, whether the enforcement is successful or not is actually irrelevant. Showing that your work doesn't infringe a patent, or that Dolby's patent is invalid, is a money and time-consuming process (unsurprisingly, patents are not generally written to facilitate re-implementation or defense).
(Moreover, in the US, in some cases, the patent owner might even get a preliminary injunction ( https://www.tms.org/pubs/journals/jom/matters/matters-9712.h... ), which might seriously and immediately harm your business. I don't know if such a thing exists in Europe).
Big tech companies like Dolby and IBM use a preventive racket-looking technique ; it involves trying to sell to potential infringers a "protective" subscription, but there's no preliminary analysis of whether there actually is any patent being infringed.
During broadcasting tech events like IBC or NAB, Dolby actually sends people to other company's booths for this ; and there's a famous story about IBM against small-at-this-time SUN : https://www.forbes.com/asap/2002/0624/044.html , whose gist is:
> "OK," [the IBM lawyer] said, "maybe you don't infringe these seven patents. But we have 10,000 U.S. patents.
> Do you really want us to go back to Armonk [IBM headquarters in New York] and find seven patents you do infringe?
> Or do you want to make this easy and just pay us $20 million?"
This is a quirk of some UK courts, where you can literally just start a case to ask a question on some detail of the law and get an answer.
The question was:
> "Is it a defence to the claim under s.60(2) of the Patents Act 1977, if otherwise good, that the host computer claimed in the patent in suit is not present in the UK, but is connected to the rest of the apparatus claimed in the patent."
From Wikipedia:
> Questions of validity were never considered by the court.
> Edited because I didn't know that some European countries accept software patents.
European patents are granted at the European Patent Office (individual european countries also have their own patent offices, whose patents can only be enforced in their home country).
You can, but fraunhofer certainly isn't trying!
> At least the patent licenses usually used with MPEG mean that private use of open source implementations is free.
0_o that is not at all the truth.
A GPL implementation doesn't guarantee a patent grant. Even if you wrote the code yourself, even just for your personal use, your own work could still be illegal to use due to lacking a patent license from the original patent holders.
Be careful about using implementations of H.26x codecs, because in countries that recognize software patents the code may be illegal to use, regardless whether you've got a license for the code or not. Even when a FLOSS license says something about patent grants, it's still meaningless if the code author didn't own the patents to grant.
I think a more subtle "open-sourcing" of this IP could still be possible, though. Maybe one that still requires that large corporate players that are going to sell their derivative products, acquire a license the traditional way (this is, after all, what the contributors to the codec's patent-pool and R&D efforts based their relative-R&D-labor-contribution negotiations around: that each contributor would end up paying for the devices of theirs that run the codec.)
Maybe there could be a foundation created under the stewardship of the patent-pool itself, which nominally pays the same per-seat/per-device licensing costs as every other member, but where this money doesn't come from revenue but rather is donated by those other members; and where this foundation then grants open-source projects an automatic but non-transferrable license to use the technology.
So, for example, a directly open-source project (e.g. ffmpeg) would be granted an automatic license (for its direct individual users); but that license wouldn't transfer to software that embeds it. Instead, other open-source software (e.g. Handbrake, youtube-dl, etc.) that embeds ffmpeg would acquire its own automatic license (and thus be its own line-item under the foundation); while closed-source software that embeds ffmpeg would be stuck needing a commercial license.
Is there already a scheme that works like this?
So yes you are correct, but in effect it might as well be patent free as far as a 3rd party end user is concerned as they have been provided what is in effect a safe harbour.
Does it mean money, or not? Because if not, then Fraunhofer does not exist.
But there is absolutely a problem here, because said 'mega businesses' actually should have a strategic imperative to want to make internet technologies more widespread.
Why on earth would MS want to limit their main line of business for a tiny big of IP related revenue?
It would seem to me, that G, MS, Huawei and all of the various patent holders should be trying their best to remove any and all barriers to adoption. There are enough bureaucratic hurdles in the way to worry about, let alone legal concerns.
Even if MS or whoever had to buy out some laggard IP owners who didn't want to play ball, it would probably still make sense for them.
Fraunhofer or anyone else are not in that situation, but the behemoths running vast surpluses are, it just seems shortsighted for them to hamstring any of this.
Both h264 and h265 have these implementations, I think FFMPEG library has both under the terms of GPLv2.
The decoders are almost completely useless. The video codec, at least the decoder, needs to be in the hardware, not in software.
Mobile devices just don’t have the resources to run the decoders on CPU. The code works on PC but consumes too much electricity and thermal budget. Even GPGPUs are not good enough for the job, couple generations ago AMD tried to use shader cores for video codecs, didn’t work well enough and they switched to dedicated silicon like the rest of them.
You'd be surprised at how often these are used.
Even Raspberry Pi has a hardware decoder: https://github.com/Const-me/Vrmac/tree/master/VrmacVideo
Linux Kernel support for HEVC is WIP but I’m pretty sure they’ll integrate eventually.
is this because of licensing/copyright?
But the very first Pi already had hardware h264 decoder (and even encoder!) which didn’t need any extra keys to work. No idea how they did it, maybe the license was included in the $25 price. Pi 1 was launched in 2013, at that time h264 has been already widespread, while mpeg-2 use was declining.
I think that’s why they did not include the license. It increases price for all users but only useful for very few of them, who connected a USB DVD or BluRay drives to their Pi-s.
This is also why you see articles from time to time highlight a stupid patent as if it's an outrage that the patent office allowed it. It's not the patent offices job to enforce patents. You can literally go ahead and patent swinging on a swing (1) and the patent office would approve it if the paper work is in order. The media would then likely pick up on this with outrage as if that's an enforceable patent. The truth is that it's simply not the patent offices job. Patents are meant to be enforced by courts.
Actually, it is (at least in the US). USPTO can deny patents on the basis on nonpatentability, and its general refusal to do so post-State St. decision is often cited as one of the problems of the modern patent system.
Broadly speaking, however, if the argument is that software patents are invalid in Europe because they'll be found so by the courts, it should be noted that SCOTUS is actually pretty likely to rule software unpatentable were it to hear a software patent case. A little background is in order:
In Parker v Flook (1978), SCOTUS said that mathematical algorithms (i.e., basically software) is unpatentable. In Diamond v Diehr (1981), they said that part of the patent being software doesn't make the entire thing invalid. The big decision is State St (1998), which is a CAFC decision holding that anything was patentable so long as it produced a "useful, concrete, tangible" result and basically broke the patent office. When SCOTUS decided Bilski v Kappos (2008), they emphatically (and unanimously!) called out State St as wrong, but declined to endorse any guidelines as to what the limits of patentability should be. Later Mayo (2012) and Alice (2014) decisions again unanimously and unambiguously laid out what wasn't patentable: natural processes, and "do it on a computer" steps.
A few years ago, we had a patent attorney at work tell us (paraphrasing somewhat) that Alice made it really hard to figure out how to write a software patent that wouldn't be invalidated. Their continued existence (and pretense to their enforceability) is less because it's secure and more because no one wants to spend the money to litigate it to the highest level (see also the Google v Oracle case, which is exactly the sort of thing a software patent case history would entail).
https://aomedia.org/membership/members/
AV1 decoding has already been in Chrome and Firefox since at least a year ago. We're just waiting for hardware decoding and encoding support now, which should start appearing this year.
The next version of Chrome will also support the AV1-based AVIF image format this month:
https://chromium-review.googlesource.com/c/chromium/src/+/22...
YouTube, Netflix, and Amazon/Twitch are also likely to not support VVC (and some of them don't support h.265 either) for their streaming services.
FWIW, Firefox supports it already (behind the image.avif.enabled knob).
They should swallow so people again come back and say they are doing it to kill competition? Just received bill from doctor's visit for hundreds of dollar, I guess things are not gonna be free in 2020 after all.