Why Debian returned to FFmpeg
lwn.net
lwn.net
I was disappointed of how aggressive the libav guys where during the fork. They changed the content of ffmpeg.org (which they had legitimate access to) to claim that libav was now the new name of the project. Then they replaced the ffmpeg package in Debian with their own version.
It's entirely possible to maintain both packages in Debian with the alternatives framework. This whole false dichotomy and calling ffmpeg the fork is not helpful in my opinion. So I'm glad that ffmpeg is getting it's package back. Forks are good and should stand on their own merit instead of doing politics like that.
"This program is not developed anymore and is only provided for compatibility. Use avconv instead."
Five minutes googling and I realized there had been a (somewhat acrimonious) fork. So my interpretation was quite natural: the libav developers and/or the Ubuntu maintainers were being dicks to the original project. If they really wanted to be clear they could have said something like:
"This is a deprecated wrapper around avconv, part of libav. It is not part of the ffmpeg project. See <url> for details."
How hard would that have been?
"The ffmpeg program is only provided for script compatibility and will be removed in a future release. It has been deprecated in the Libav project to allow for incompatible command line syntax improvements in its replacement called avconv (see Changelog for details). Please use avconv instead."
libav is not the successor to ffmpeg. libav is a fork. How'd Debian move so quickly to a(n) (immature) fork?
Politics.
Basically it was a big split down the middle, the entire project split in half.
Not to kick 'em while they're down but it certainly wasn't on technical merit.
As I say to people this is like the Catholic/Protestant split, both sides think they are the one-true-ffmpeg but both have valid claims. Doesn't change the fact it's a disastrous situation and both projects are not in particularly good states.
ffmpeg is the one-true-ffmpeg. If libav surpassed ffmpeg, we'd have libav. The one-true-libav. libav doesn't have a valid claim; It's a fork second to ffmpeg.
Somebody put the cart before the horse. Happy to see it fixed.
Gentoo initially followed the mainstream but changed the default flag to ffmpeg a few months ago.
Probably pretty rough on the ego.
The Debian security team vetoed this option, which is why the recent discussion was choosing between, 1) bring back ffmpeg and remove libav, or 2) keep libav and do not bring back ffmpeg (option 1 was chosen). They don't want to support security patches for both, and for all combinations of software that might use either one.
Do you know if contrib packages are also getting security updates ? If no then why didn't they just move the ffmpeg package there. To me it felt like the really wanted the ffmpeg package to be virtual and pointing to libav.
On what grounds?
There appear to also be issues with frequent soname bumps: https://lists.debian.org/debian-devel/2014/02/msg00651.html
It isn't quite as simple as this when we are talking about libraries... Which one is used by other packages to build against?
Whichever is used, has to become the runtime dependency.
It is, but it is also a big maintenance burden, in particular for the security team.
I tried this criteria, I divide the committers in three groups:
A) Michael Niedermayer
B) People with more commits in FFmpeg, i.e. Clément Bœsch, James Almer Carl, Eugen Hoyos, ...
C) People with more commits in libav, i.e. Vittorio Giovara, Martin Storsjö, Anton Khirnov, ...
(I supouse there are other commits from people outside the published table, but they are few.)
Statistics:
libav FFmpeg
A) 46 1831
B) 37 1071
C) 1074 856
Tot: 1157 3758
Complete table:
Developer libav FFmpeg
---------------------------------
Michael Niedermayer 46 1831
---------------------------------
Clément Bœsch 179
James Almer 155
Carl Eugen Hoyos 150
Andreas Cadhalpun 21 114
Lukasz Marek 98
Paul B Mahol 93
Ronald S. Bultje 85
wm4 16 83
Christophe Gisquet 66
Benoit Fouet 48
>>>Subtotal 37 1071
---------------------------------
Vittorio Giovara 294 294
Martin Storsjö 253 252
Anton Khirnov 206 197
Luca Barbato 131 113
Diego Biurrun 72
Rémi Denis-Courmont 32
Hendrik Leppkes 17
Himangi Saraogi 16
Gabriel Dume 16
Federico Tomassetti 14
Peter Meerwald 12
Janne Grunau 11
>>>Subtotal 1074 856
------------------------------------
>>>>>>Total 1157 3758But since Michael alone seems to "outdevelop" all others, ffmpeg comes ahead in terms of features and supported codecs, so people prefer ffmpeg. Since i don't know the ffmpeg APIs or the code quality, i don't know if it is true that ffmpeg has the worse design and development process. If it is true and Michael leaves it sounds like a lot work to take over. On the other hand... i understand that libav was already a fork of ffmpeg.. So some people did take over the code in the past.
Just restating what the article says.
Let's put Michael Niedermayer in his own group "A".
Clément Bœsch has no commits in libav and 179 in FFmpeg. So I guess he's in the FFmpeg team.
Andreas Cadhalpun has 21 in libav and 114 in FFmpeg . So I guess he's in the FFmpeg team.
I put all the persons who have more commits in FFmpeg than in libav in the FFmpeg team. Let's call this team "B". The classification is clear, because in all the cases the difference is big.
Vittorio Giovara has 294 commits in libav and 294 in FFmpeg. It's a tie! It's more difficult to guess, but I put him in the libav team. (This choice makes the bus factor bigger.)
Martin Storsjö has 253 in libav and 252 in FFmpeg. It's also difficult to guess, but I put him in the libav team.
I put all the persons who have more commits in libav in FFmpeg in the libav team. Let's call this team "C". Some cases are difficult to classify, but I choose the classification that makes the bus factor bigger.
In the table:
libav FFmpeg
A) 46 1831
B) 37 1071
C) 1074 856
Tot: 1157 3758
The persons in group B wrote 1071 commits for FFmpeg and 37 of them were copied to libav
The persons in group C wrote 1074 commits for libav and 856 of them were copied to FFmpeg.
The moral is that with the data of the table it's not clear that the group B is much smaller than group C. Only that group A is bigger than both!
I can't comment on code quality, project management, etc and frankly I don't care. If one library causes problems to every bit of my system, it should stay off until it is on par with the library it tries to replace —provided that both libraries use a free software compatible license.
Given the new status quo, my system refused to update since it would pull both libav due to the new profile and ffmpeg due to my use flags. As the Gentoo devs made this new choice, I thought I should embrace it, so I followed the steps to switch to libav. This created a nightmare of dependency issues that took me months to solve, especially given my limited time and that this happened on my main machine. I finally switched back to ffmpeg but it seemed to me that only when it —libav— was removed from Gentoo's default desktop profile my problems really stopped.
Gentoo has supported both for as long as its included libav at all.
Gentoo has supported both for as long as its included libav at all.
This kind of behavior is lethal to open source projects, end users will end up fragmented across the two versions, package maintainers for the various distros have to make tough choices and everybody loses.
Forking a project should be reserved for when a project becomes - by some objective criteria - abandoned and there is enough support for the project to continue. Then and only then should a project with a large amount of traction be forked.
Forks are both the greatest thing in the open source world and the most annoying thing at the same time, they have the potential to rescue projects but more often than not they're used to kill projects.
Really happy to see ffmpeg back, and I hope that the people that the majority of the contributors to libav switches to directly contribute to ffmpeg so that there will be an end to all the duplicated effort and wasted resources.
What about Hudson/Jenkins and OpenOffice/LibreOffice? And egcs?
I think criticizing forking is misguided; the fork is a consequence of the problem, not the cause, and it can often be the best resolution of a bad situation.
Consider what would have happened if it was impossible to fork ffmpeg: the developers who created libav would still have a problem with how ffmpeg was developed, and they could very well simply leave it. That would mean a loss to ffmpeg itself (which used the libav patches), a loss of the libav developers, and for what?
The whole contributor base – everyone contributing to ffmpeg – tried to get the project leader out of the project, but he refused. So these people forked ffmpeg – or rather, they took all the repos (which they owned), took all the CI servers (which they owned) and would have also taken the name ffmpeg, but the project leader owned that name.
It’s more of "one asshole vs. all the people writing the code he gives away" than anything else.
The project leader is also a contributor, so clearly not 'everyone'.
> So these people forked ffmpeg – or rather, they took all the repos (which they owned), took all the CI servers (which they owned) and would have also taken the name ffmpeg, but the project leader owned that name.
There are people that consider Linus to be 'an asshole' and I'm really happy that he at least has the Linux name as a trademark.
To me, looking from the outside in and reading about the whole sage they're all assholes. You keep your dirty laundry inside and you do what you can to present a unified front to the world. As soon as you fail to do that you are failing your project in the most important sense that you can fail it: continuity.
I see the issue, I – as user – prefer ffmpeg, but I – as dev – prefer libav. I’d love to somehow see both sides working together, but it sadly just won’t happen.
This is weird to hear as a consumer of these libraries. When people ask why I prefer one or the other for my own use cases, I tell them that FFmpeg has the better API and the better format support (specifically vastly more pixel formats/depths in its lossless codecs). But regarding API, at least the parts that concern me, FFmpeg is a bit fuller and requires less boilerplate. libav* have a large surface area, so even minor affordances like avformat_alloc_output_context2 and avcodec_find_best_pix_fmt_of_list are helpful.
To be fair to Libav, due to being "downstream" FFmpeg has benefited greatly from their improvements, e.g. AVBuffer and the redone AVFrame management on top of it. They absolutely deserve credit for improving the API. But FFmpeg's API being effectively a superset of Libav, as a plain old user of the libraries it doesn't really make sense to target the latter.
That's to say nothing about the politics or people involved in the projects, it's just a matter of practicality.
Example: ffmpeg -i doom4_trailer.mkv -acodec copy -vcodec copy doom4_trailer.mp4
Pretty simple, right?
libav:
>[mp4 @ 0xa33800] Application provided invalid, non monotonically increasing dts to muxer in stream 0: -83 >= -83
>av_interleaved_write_frame(): Invalid argument
(premature exit)
In the end I stopped using both programs. One was aparently unsafe and not installable without breaking things, and the other reinstall itself constantly, acts in a questionable way, and did not solve my problems. I guess that for Debian this was a false step that consumed time and resources but did more damage than good. Libav probably damaged also the image and dominant position of VLC. After several errors and troubles since them, its current luster is not the same as in the first years in my opinion.
The great thing about open source is that even if they step away it's not like the code is going anywhere. To an end user of ffmpeg it would still continue to function.
I personally had no idea ffmpeg was forked to libav until one day I tried to install it in Ubuntu and was like...wtf. Then I installed libav and went about my day.
Libav:
$ git shortlog -ns --no-merges v11..HEAD | head -n15
FFmpeg:
$ git shortlog -ns --no-merges n2.4..HEAD | head -n15
[1] https://lwn.net/Articles/650830/ffmpeg has maintained compatibility, making it really easy for downstream users and distro maintainers to keep going, whilst continuously adding new features and improvements, including aggressively merging those placed in libav (I guess as long as they aren't deletions or cleanups?).
I feel like there's a lot to learn in this whole drama and that it hasn't been very deeply explored. libav team originally claimed that ffmpeg's leadership had gone dark/fallen off the map, but they sure came back quickly to express discontentment at the mutiny. libav team undoubtedly has talented people working on it (wasn't DarkShikari on the libav side of the schism?) and libav/ffmpeg share a lot of goals. You'd think they could come to some type of compromise less onerous than this entire saga has been. IMO it's a failure from all sides that this fork was even a thing. A phased, unified, slow, and well-managed release process like Python 2 -> Python 3 might have made all of this unnecessary.
This is at total nit, but I really doubt emulating the Python 2 -> Python 3 release process would go high on anybody's list of successful project evolutions. Given that Python 3 was officially released in 2008, it's worth keeping in mind that as recently as 2014 most new projects were still being created in Python 2.x[1]
1. http://www.i-programmer.info/news/98-languages/8269-python-2...
That said, given that python 2.7 is being continued to be maintained - but not with new features - for such a long time, and letting python 3 evolve quite quickly until it's now actually beginning (finally, around 3.5ish) to have features which are really appealing to many users (the async stuff, type hinting, etc), I don't think they've done too badly.
They could have made it quicker to happen, but since python2.7 is still a very good language, and works stably on practially every OS you could want, why try to force a change until people are ready?
People bitch about it [the changeover] a lot - not because it's terrible, but because it's lasting such a long time.
AFAIK, NO mainstream OS is shipping python3 as the main python, or even as a default installed python, yet.
Most big libs work in python3 now.
I reckon in 5 years time python3 should be the default install, with 2.7 as the legacy extra package for people who need it.
One way that could have been helpful (and may even be possible?????) would be a -2k flag for python3 which runs in python2.7 compatibility mode, or something. I dunno.
IIRC, Arch Linux does. Bleeding Edge. From the Arch Wiki:
>To install the latest version of Python 3, install the `python` package from the official repositories.
It's still nice to have Python3 be the default as a Python3 developer.
$ pacaur -Qi python Name : python Version : 3.4.3-2
Python 3 adoption was a little slower than may have been ideal, but I personally think it was a very successful way to implement breaking changes, especially when it could've been a libav/ffmpeg-style mess.
Given that the schism was based on politics and philosophy, I doubt that a compromise would have been possible.
ffmpeg moves fast and sometimes gets sloppy, and was hard to version for distros. libav wanted slow and careful and clean and wanted to be distro friendly.
Clearly libav tried to Do It Right(tm), but it looks like they ended up on the losing side of the fork. Forks only work when you can get the majority of developers to come along with you.
Meanwhile ffmpeg merged basically everything from libav. Furthermore ffmpeg made an effort to provide compatibility, even for stuff where libav intentionally broke compatibility.
So basically ffmpeg benefitted from all the work put into libav but not the other way around. Seems to me that they brought this upon themselves.
[1]: http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html
While the two projects have chosen somewhat different approaches to developing the software, these differences were not what lead to the fork, and neither project particularly resembles pre-fork FFmpeg.
659 contributors. There are 14 people with over 1000 commits, 34 with over a hundred. 4 people with over 100K additions/deletions.
Q: Does FFmpeg use patented algorithms?
A: We do not know, we are not lawyers so we are not qualified to answer this. Also we have never read patents to implement any part of FFmpeg, so even if we were qualified we could not answer it as we do not know what is patented. Furthermore the sheer number of software patents makes it impossible to read them all so no one (lawyer or not) could answer such a question with a definite no, those who do lie. What we do know is that various standards FFmpeg supports contain vague hints that any conforming implementation might be subject to some patent rights in some jurisdictions
IPC/ECLA (and probably whatever the USPTO use, USPC [IIRC]) allow for quite fine classification so it's not necessary to "read them all". The point is in there somewhere, and valid, but this phrasing seems just a little too flippant - to the point of deception.
I don't doubt that there is a patent attorney/researcher/engineer in a company somewhere sufficiently on top of their game to say truthfully they don't infringe on any patents with their primary technology, patents valid in their jurisdiction, but it would certainly be a rare thing.
Unfortunately software patents, as implemented in the U.S. especially, are a morass full of landmines and predators, and offering little of value. By and large the patent holders (both legitimate and illegitimate) tend to stick to trying to extract money from rich players (which is mostly the large companies that do have the resources to care about patents) and/or making patent-exchange deals to protect themselves. Unless you have a lot of money to throw around, it's a game you can't win, and any attempt to deal with the system at face-value is a pointless waste of scarce resources.
"Ignoring Patents", Mark A. Lemley
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=999961
What's going on here? The answer, I think, is quite simple: both researchers and companies in component industries simply ignore patents. Virtually everyone does it. They do it at all stages of endeavor. From the perspective of an outsider to the patent system, this is a remarkable fact. And yet it may be what prevents the patent system from crushing innovation in component industries like IT. Ignoring patents, then, may be a "workaround" that allows the innovation system to function in the face of overbroad patent protection.
Thanks!
It made me wonder in my own professional work: should I also focus more on getting things done instead of trying to find the best solution? Will that approach give better results in the end? It's hard to tell if libav is more maintainable in the end, especially if no one will use it in the next few years, which might be the result of Debian backing down on libav.
Still, the fact that ffmpeg has more features and its devs fix bugs and security issues more quickly seem to suggest that ffmpeg is more maintainable, even if the libav maintainer claims otherwise.
---
In the end it's better to work on a (perhaps) less maintainable product that people use compared to a very maintainable product that no one uses.
As an SDET who has dealt with code perfectionist, please just focus on getting things done and features shipped. Assuming that bugs aren't life & death in your product like car software to deploy airbags, or huge consequences/delays like a land rover destine for Mars missing its mark and flying off into space. Worry about all that perfectionist stuff later, when you're sure it actually matters. Until then, as long as your code isn't a hideous abomination, it's fine. Just ship it. If you have passionate QA/SDETs, they'll find the critical bugs and do performance tests if needed. This is especially true if the product had a good spec that it was developed around.
That said, don't take any shortcuts when it comes to security!
I'm a perfectionist, too, but I'm also lazy. As a result I hate the idea of working hard on some beautiful piece of code that no one ends up using.
This question can only be answered in the context of the project's life cycle. Is it in the early stages? Are there many downstream users relying on its stability? How clearly is the distinction between internals and interface communicated to downstream users? How technical are the end users?
> the fact that ffmpeg has more features and its devs fix bugs and security issues more quickly seem to suggest that ffmpeg is more maintainable
Or it could just be that ffmpeg has people who are willing to dedicate a much larger amount of time to their maintainership duties. Often differences like this have much more to do with nontechnical causes like corporate support.
From personal experience: yes. The problem is the perfect solution is usually only obvious in hindsight, after you've done the work, so the quicker you get there the quicker you can get to where it "should" be.
Given how there appear to be no problems adding new features to ffmpeg, any technical debt the project has incurred seems to be within manageable range.
It doesn't look at all, as if libav is better, by any reasonable metric.
The output in the short term is that libav is riddled with bugs, the hope is that in the long term this will pay off. Unfortunately in the interim ffmpeg seems to have propelled ahead.
I don't need menu support or parsing or anything ... I just wonder, can I point it at a DVD .iso and it will play video ?
On a side note, ffplay has a really neat feature: while playing an audio file using ffplay, it displays a spectrogram by default.
And clicking in the spectrogram window (launched ffmpeg from Gnome terminal so extra graphic window appears) jumps the playback to the appropriate % of the track.