All the giant companies used ffmpeg (2020)
twitter.com
twitter.com
One day, he began working on certain audio/video encoding problems that needed to be solved for our project. He chose FFmpeg as the primary tool around which he began building his solution. After having spent several days with FFmpeg, figuring out and documenting various ways of using FFmpeg to solve all our audio/video encoding problems, one day, when we went out for lunch, he declared to the team,
"FFmpeg is awesome! If you give me a shoe, I will turn it into a video file!"
It’s not actually true, but it is story true.
Glad to see this phrase from The Things They Carried outlive the book.
None.
If an old photo is what you've got, make the best at extracting from it.
The OP makes clear that this was the only viable way of extracting information. 1 >>> 0.
Also, as a sufficiently early recording (anything before the mid-1940s), the medium would have been shellac or wax rather than vinyl. Shellac especially is immensely fragile and prone to shatter when handling.
I'm not telling you what you meant.
I'm telling you what I heard, and, in reponse to your implied question, why I'd responded as I had.
I worked on an earlier version of this project where we actually had to play the albums but we also took high res (300 dpi tifs) of the individual records with the hope that we'd never have to physically touch them again. So I'll claim contributing to this effort. ;)
Paragraph 2 of the link.
Every time an archivist touches the media - no matter gloves, a clean environment, the care applied, etc - there's a risk of altering or even damaging it. In our case, the eventual goal was to keep the ever-deteriorating originals in nitrogen-filled vaults.
Therefore, if you can avoid having to touch it often (or ever), then you've managed to protect it better, probably longer, and more cheaply.
Static, high-resolution photography (or other imaging) is about the least-invasive, lowest-risk options available.
“The paraphrase of Gödel's Theorem says that for any record player, there are records which it cannot play because they will cause its indirect self-destruction.”
―Douglas R. Hofstadter, Gödel, Escher, Bach
ffmpeg is doing all the heavy lifting there. VLC mostly uses libraries from ffmpeg.
I have no idea why people are so impressed by VLC. Anything built on libavcodec can play all the same files, but they won't have VLC's deficiencies in performance, quality, reliability, or usability. ffmpeg is a great project and VLC an incompetent one riding its coattails.
Like when Apple added ExFat support and Windows couldn't handle it, because Microsoft only wrote the spec. They didn't follow it.
I know and respect Fabrice from in person interactions. He is exactly who he claims to be. Maybe an interesting question, instead of this current line of discussion, is to find out what led to his successes?
Could it be replicated?
I would love to work more on open source.
If it only works for small companies or end users, maybe it can be dual-licensed or mixed-licensed, offering the basic software for free but selling closed-source add-ons for expanded functionalities.
While I think the GP was a joke, you ask an interesting question that I tried to answer 12 days ago on this very site. [1]
tl;dr: I'm not fast, but I'm getting faster because I understand my entire codebase. I think that may be a good portion of Mr. Bellard's success, or if it is not, it may be the best way for mere mortals to emulate his success.
FFmpeg doesn't have any dependencies does it? You are probably onto something.
I studied a bit of a partial implementation of an OpenGL software renderer written by him. I even found a bug I could fix!
Actually his first projects were not all that advanced, they are things "normal people" could do. I think key lies down to 3 points:
- Start simple: do not try advanced projects at first, start slow, plan, don't write the first function before you know what will be the last. Use this for many small but increasingly complex projects to gain experience.
- Understand what you do: most of the most notorious Bellard's projects are complex but based on many simple smaller moving parts and based on simple, understandable concepts. If the first point is respected, this one comes naturally.
- Be regular: do what you need/want but do it regularly. If you look at what Bellard does in a single day, it doesn't seem that different from what most people can do in a single day. But he does it with incredible regularity.
Now, in Bellard's case there's just on more special thing: he is a gifted mathematician. That certainly helps being a good developer and planner too, but turning into a mathematician of his level also takes another level of discipline.
This is some seriously deep meta-heuristics. Very insightful. Thank you.
This is a true counter-standpoint to all the agile cargo cult that has been developed in the last years... Not to be taken as 100% dogma, like everything...
2) More money than God.
Also helps: zero fucks to give about maintaining the damn thing.
[1] https://en.wikipedia.org/wiki/Stanis%C5%82aw_Lem#Philip_K._D...
"Oh here's a list of my projects."
>FFMPEG is a media processing framework. It's great for simple media processing jobs (builds a graph, runs the graph). A build of "FFMPEG" includes that media processing framework plus dozens of open source libraries (filters) that do the actual work (x265, x265, libav, etc).
>So "FFMPEG" gets a lot of well deserved credit (it's the framework that lots of applications and well-known services run), but the core FFMPEG is probably a tiny fraction of the development man-hours in a build of FFMPEG. It's all the linked libraries that make it useful.
It is also worth stating this is from 2020. I am not sure why this is submitted now.
The most well-known 3rd party libs provide a few (albeit commonly used) video encoders. Of course, having interfaces for these external libs is what allows media operations to be consolidated into a single process workflow. As standalone tools, x264/x265 would be more cumbersome to integrate into a pipeline.
I came across it a day or two ago in a reply to @Suhail's thread of donating to OSS projects[0]. I suspect that's where the submitter saw it as well.
https://ffmpeg.org/consulting.html https://ffmpeg.org/donations.html
I have designed and build with my team VOD service for leading local entertainment TV provider in 2009.
We were amazed at provided flexibility by ffmpeg. In no time we were able to hook up our back-end logic and go.
We opted to "risk" with Nginx and Jquery and delivered successfully on the front-end.
The funny thing is that at the time a lot of commercially available "solutions" were a total joke and costed an arm and a leg.
Looked through their copyright notices and it turns out they were using ffmpeg. It's really everywhere.
If you've ever heard a Twitch streamer, especially people playing 8-bit games or chiptune-ish music, and wondered why the audio sounded a bit dodgy, it's ffmpeg. AAC at 160kbps (the streaming standard) is supposed to sound better than that. OBS will use FDK if available, and on macOS and Windows it'll use CoreAudio if installed, but you need to install an old MSI pulled from an old version of iTunes to get that on Windows... so the vast majority of streamers on Windows are stuck with ffmpeg-AAC.
If you're using ffmpeg on Linux, install libfdk and use libfdk_aac; if you're on macOS, use aac_at instead. Those are much better encoders and ffmpeg can use them as external libraries.
Note that this isn't a criticism of ffmpeg; lack of quality in some codecs is a direct result of lack of funding (though I do fault them for, at one time, claiming the built in AAC encoder was good enough to be made the default and competitive with FDK; it wasn't and still isn't, but I think they reverted that claim at some point).
(source: I help run a couple anime DJ restreaming-based events and fixing the AAC issue is a big red warning item in my set-up guide; I once got a double transcode feed where both steps were using ffmpeg-AAC and I could instantly tell something was wrong)
Hey, do you happen to know if aac_at is encoding at the slowest speed? (Most compression efficiency but longest encoding time). I can't figure out for sure, so I keep bouncing audio through afconvert since it lets me set `-q 127`, and it's a pain, I'd rather do one step.
ffmpeg maps it like this:
q = 127 - q * 9;
So `-q:a 0` should do the same thing as passing 127 to AT.
There is another property though; an actual control of codec effort (`kAudioConverterCodecQuality`). That one should be accessible as `aac_at_quality` and maps 0 to 96, so there's no way to go up to the max of 127... But Apple defines "High" as 96, so I get the feeling there won't be a big difference between that and max, especially for an audio codec.
> That one should be accessible as `aac_at_quality` and maps 0 to 96, so there's no way to go up to the max of 127.
Is this a bug I should report to ffmpeg?
I think windows might have a built in AAC encoder in media foundation? Wonder if that's true and if it's any good.
OBS doesn't even let you pick streaming codecs. It just assumes H.264 and AAC. You can get it to stream in other codecs by abusing the "record" function with an output to stream instead of filename, but it's a hack...
I use ffmmpeg all the time on my M1 Max and if there are parameters that I should always be using, I would like to know :)
Having spent a lot of time using ffmpeg for transcoding, it's one of those things I recognized because I'd often forget to set the correct settings and got similar results, as well.
It's gotten better in the past few years, though, but there's still a lot of legacy codebases that are useless today because they won't build against modern ffmpeg any more (even though the rest of it is fine).
Do you have examples to the contrary from the past few years?
I say this as someone who has extended and maintained a libav* language binding with similarly terrible documentation. I tried to write more examples for people. But the API surface is vast and resources are extremely scarce. Sometimes you just have to dig into actual production code of FFmpeg (the CLI side), mpv, etc. to see how other people use it.
I learned to use libav by trial and error over years, and now I'm at the point where I can just look at the headers for a refresher whenever I use it.
At a very broad level:
- Encoding and decoding are both pipelines
- Codecs and formats are both part of a pipeline
- An instance of a codec or a format is called a _context_
- The two basic operations for any context are "put data in" and "pull data out"
Putting this all together, a lot of tutorials present this as a case of "Just push the input data in, then pull the output data out!" In my experience this is backwards. This leads to "pipeline buckling" where, e.g., you pull a compressed packet from the demuxer and try to push it into the decoder, and the decoder says "hey, slow down, my buffers are full". You have to just awkwardly stick that packet in your pocket until the decoder's buffers are dry again.
I have always had better luck pulling data. e.g. If you're decoding, first try to pull a decompressed YUV frame from the decoder. If that fails, then try to get a new packet from the demuxer. Only if that fails (and you're not using the default file I/O), then feed the demuxer another buffer. This way it never buckles. It results in a few redundant "Can you give me anything?" checks when warming up the pipeline, but you never have buffers awkwardly outliving a loop iteration. If a buffer enters your control, it's always because you already found an empty spot in the next stage to immediately dump it into.
My workaround to avoid having to buffer a potentially infinite amount of data was to instantiate the demuxer twice, once for A and once for V. Then they can run out of sync with each other in terms of the muxing, and you can keep them in sync in terms of presentation timestamps.
Your best bet is to just call ffmpeg to do what you need and not have to worry about the weirdness of the libav stuff.
Also, stuff like parsing a buffer of memory requires you to write a file driver since the code for libav is tightly coupled to ffmpeg which only handles files and not buffers.
I'm just not sure what kind of performance overhead it will have writing to a ramdisk vs. using a buffer, as it's a non-negligible amount of memory to read/write so many extra times.
I'd take a look at https://github.com/FFmpeg/FFmpeg/blob/master/tools/target_de...
In io_seek I'd also clamp pos at the end to be between 0 and the filesize. That might be needed for some files. It doesn't hurt though.
>I'm thinking that for long-term maintainability it might be better to use the CLI anyways
It's definitely easier that way. You don't have to deal with gotchas like having to set some random flags or needing to specify a filename. It's just so easy to write code that fails on 1% of files which ffmpeg can actually decode.
(The transcoding was for transforming DVB-T captures to multi-bitrate HLS and MS Smooth Streaming formats as part of a Cloud PVR. It was a separate backup pipeline in case the commercial streaming pipeline failed, which tended to happen a lot initially.)
Why I ask is that pretty much the only reason I have gstreamer installed on my pesonal laptop is to satisfy certain GNOME dependencies (due to GNOME, understandably, dogfooding its own stuff). Everything else (including apps I use daily e.g. mpv, Firefox, etc) depends on ffmpeg.
$ sudo pacman -Rs gstreamer
checking dependencies...
error: failed to prepare transaction (could not satisfy dependencies)
:: removing gstreamer breaks dependency 'gstreamer' required by cheese
:: removing gstreamer breaks dependency 'gstreamer' required by gnome-shell
:: removing gstreamer breaks dependency 'gstreamer' required by gst-plugins-base-libs
:: removing gstreamer breaks dependency 'gstreamer' required by libcheese
:: removing gstreamer breaks dependency 'gstreamer' required by webkit2gtk
:: removing gstreamer breaks dependency 'gstreamer' required by webkit2gtk-4.1Debian is not the best maintained distro.
But it generally gives me no trouble.
The core media pipeline is almost all gstreamer, ffmpeg, and just because I think it’ll surprise some people, Wayland.
I know Handbrake supports it, as I used it for my productions in the fast. And I think handbrake uses ffmpeg, maybe?
I never got particularly good at that.
There are people that have made entire careers of tuning FFMPEG.
I liked his making a $1K donation.
FTFA! In all seriousness though; there isn't an aspect of computing that hasn't been improved by F(L)OSS. (What's the L stand for?)
Gratuituous could be used for free of charge since it's what we used in latin languages (which I suppose is the same root as gracious - at least in formal french we replace gratuit with gracieux to mean the same thing).
So I wonder if English, tired of borrowing a strange word, will use gracious or gratuitous some day to denote free of charge ! That would maybe avoid all these questions of what libre means !
I rarely use FLOSS as, floss is a known word for flossing teeth and for the dance move. Libre just sounds cool compared to floss.
Mmmm.. I'd have to disagree, based on my professional experience. FLOSS hasn't really improved business software. User interfaces, Groupware, Enterprise software, Tax software, Government software, Industrial software, Telecom, POS, EHR, etc. It's hardly ever written as FLOSS because it takes a lot of time and effort, and devs don't find it rewarding to write that stuff in their free time (most OSS is written because the developer just needed that code). When there is a FLOSS solution, it's not an improvement over the proprietary alternative.
Even in "IT", there is a huge dearth of quality FLOSS used to build... anything. There's lots of one-off projects that are good for one thing or another, but very few large robust solutions, and components never seem to mix together unless they have "custom integrations" to tie them to other systems. Microsoft will forever be the king of Enterprise software integration and business productivity tools, and each industry has its own "business software" titan. SaaS and PaaS are also more closed-source than open.
When there is some FLOSS that does something difficult like video rendering, or running distributed applications, the entire industry uses it. But for the bulk of the business world's needs, often the only solutions are proprietary.
But yeah, for simple video tasks, I either use ffmpeg or avidemux, davinci resolve crashed at first use.
There's also a python module that wraps the filter syntax I think, because writing a ffmpeg command is not very accessible.
It is really crazy when freely accessible software can get developers more reputation and money, rather than pissing code away for code that they don't own anymore.
Proprietary code gets the job done, but free software helps the whole community of developers.
https://streaminglearningcenter.com/blogs/youtube-uses-ffmpe... https://news.ycombinator.com/item?id=2831292
Let's take a look at the history of ffmpeg created by hackers for hackers.
Do you remember how many times you saw strange squares on the screen when rewinding a YouTube video or DVD MPEG2, or random freezes when streaming Twitch when you think it is a video stream error. But no. Most of these are programming errors of the ffmpeg hacker backend, which has many pointer errors. It is difficult to recovery state from a failure of any level due handling C and pointer errors. The global use of ffmpeg is caused by the lack of FOSS alternatives, but not by the quality of the software. It was cheaper for corporations to fix bugs and hire a developer like me than to develop their own video codec or system from scratch. All alternatives were closed source and required of tones of money per tech licenses like MPEG2. Of course, over the years, the ffmpeg ecosystem has improved and many bugs have been fixed, but this has not changed the general situation with existing and potential bugs and vulnerabilities by design. Therefore, in different video applications, you can observe random bugs and signal drops every day. Nice try John ID_AA_Carmack
Your blocky errors in video playback aren't caused by a pointer going wild due to bad pointer math in ffmpeg. They're caused by bad or corrupted input or an improper seek to a non IDR point in the file. ffmpeg is rock solid for playback of standard corruption-free h.264 streams. This idea that any time you see corrupted video it's a (potentially exploitable) bug in ffmpeg is just not true. Nobody is getting "signal drops" due to ffmpeg problems.
Now if you start fuzzing ffmpeg and giving it wildly noncompliant and broken input, yes, you're going to run into memory errors at some point - which is why there are companies doing just that these days, to find and fix those security flaws. Most of those bugs aren't even in popular codecs (since those are extremely well tested), but rather in the very long tail of obscure formats that ffmpeg supports (try counting how many obscure video game formats it supports some time!), and therefore in codepaths that nobody watching YouTube or Twitch is ever going to hit.
Patents have absolutely nothing to do with this; they don't affect the security of the resulting code.
Nobody can write code for all the formats ffmpeg supports and do it safely, in C. Nobody. You think audits work? Tell that to Infineon, who shipped a broken RSA key generator implementation. In a cryptography product. Which was audited privately. And that's a critical part of the code that should've had many eyes on it.
Perfect memory safety for C parsers for hundreds of formats? Yeah, not going to happen. It has nothing to do with the code quality of ffmpeg. It's just not humanly possible to get this right in every instance.
That's what theorem provers and model driven development are for.
I wish these tools could be used more. There should also be automated completeness checkers for requirements and specs - a lot of errors, incompatibilities and security issues are the result of ambiguous, contradictory, or incomplete specs and requirements.
That's building a safe programming language on top of an unsafe one. At that point you might as well just use a safe programming language.
That won't fix logic errors, but the vast majority of security issues are safety problems, especially in this kind of codebase.
It might prove that all bytes of the output were written to at least once, that dereferenced pointers point toward either the input or some other valid structure, and other properties like this.
They found issues in code I wrote...
(I work for Google but do not speak for it)
I wrote a thing to proxy a twitch livestream and return it as lazy animated gif, so it could be embedded into, like, old-school messageboards and proprietary chat clients that allow hotlinking images. At the core it was feeding a .m3u8 URL into ffmpeg and asking it for a gif. When it turned out that gif was kind of awful for that use-case, I just asked ffmpeg to give me an mjpeg instead, problem solved.
https://twitter.com/ID_AA_Carmack/status/1258610970349441024