FFmpeg 7.0
ffmpeg.org
ffmpeg.org
20 years later it's still a goto. Great tool.
& $ffmpegPath -i $_.FullName -r 23.976 -vf scale=1280:720 -c:v libsvtav1 -pix_fmt yuv420p10le -crf 30 -preset 10 -g 300 -c:a libopus -b:a 96k -ac 2 -c:s copy -map 0 $destPathI convert much to 720p PS3 compliant H264, for maximum device compatibility. I take an external drive with these files with me when I travel, and in 99% of the cases plugging it into hotel TVs just works.
At the very least, 1080p instead of 720p.
Lets write a new video compression algorithm that is super efficient - great
this lets us compress movies so they can fit on cheap CD's, instead of DVDs. - great
We can now give those CD's away with movies on them - great And then every time someone puts one in a DivX player they can pay us to watch/rent it, instead of having to drive to blockbuster - wait, what?
Its easy, we'll just use a phone line that everyone has right near their entertainment center in their living room to phone home at night and send the data of what movies you watched and how many times. - what are they smoking?
Afaict these aren't related.
DivX the video codec started out as an unlicensed hacked version of Microsoft’s MPEG-4 v3 codec binary. Since it wasn’t a commercial product and was legally dubious, the author called it DivX ;-) with the smiley in the name.
When it became unexpectedly popular during the dot-com boom time, someone of course set up a DivX company that dropped the smiley, eventually rewrote the codec, and presumably acquired the trademark from the defunct DIVX (or just took it over if the registration expired, I don’t know).
and then, iirc, this is where xvid come into being. I think it was the same codec just re-written and given back to the opensource world, hence the reason for naming it "divx" spelled backwards.
This is one part of 2000s tech I'm happy to have mostly forgotten about.
You could encode a CD sized video file, burn it, and watch it.
People who have always lived in a world with great software like VLC and MPV and ffmpeg underestimate how hard it was to actually play a video file on your computer back in 2000.
We transformed a relatively decent desktop into a ffmpeg transcoding machine, which would monitor files incoming from a samba share and it would output the converted file into another samba share.
It was just a bunch of scripts and cron jobs but it worked much better than I anticipated and it was mostly maintenance-free.
Would def be interesting if someone could write up a history of the project so far. I wonder how much industry input into the OSS commits there are (like MS/IBM into linux, postgres, etc)
I agree with you in general, but it still annoys the crap out of me that QuickTime Player on macOS still won't load completely valid videos.
My basic "QuickTime compliant" options are:
-tag:v hvc1 -c:v libx265 -crf 28 -c:a eac3
Of course for everything else there's VLC.
Being cheeky of course here. FFmpeg is great. An AI assistant was what I needed to execute my ~12 FFmpeg commands per year though, with ease and speed.
I mean I've only done it once with ffmpeg but it felt so good
yt-dlp -f "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best"
--restrict-filenames
--output '%(title)s.%(ext)s'
--ignore-errors
--embed-subs
--all-subs
--sub-langs all,-live_chat
--compat-options no-live-chat
--convert-subs srt
--format 'vcodec=av01/bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best'
--convert-thumbnails jpg
--embed-thumbnail
--audio-quality 0
--add-metadata
--xattrs
--xattr-set-filesize
--prefer-free-formats
--geo-bypass
--no-mark-watched
--console-title
--no-warnings
--sponsorblock-remove all
--compat-options embed-thumbnail-atomicparsley
yt-dlp really is a great tool too, I do hope it keeps working for a long time
> prefer-free-formats
Are you entirely sure you know what you are doing?
--format 'vcodec=av01/bestvideo*+bestaudio/best'
...Whoa. Thanks for making me double-check!
Somebody down-voted you, I seem to recall seeing earlier, sorry about that. Your comment was valuable, though I'd definitely use more productive tone next time.
And finally, after some more tests it turned out that some videos have almost 2x bigger .webm variants than .mp4 so I had to extend my yt-dlp config like so (only showing the prioritization options here, my previous full config is still there upthread):
--format 'vcodec=av01/bestvideo*+bestaudio/best'
--format-sort 'quality,res,fps,hdr:12,vcodec:av01,channels,acodec,abr,asr,proto,ext,hasaud,source,id'
--prefer-free-formats
1. Prioritizes AV1 (because it's gaining more and more hardware support, is very efficient and is not license-encumbered)
2. Prefers the free formats
3. ...but still uses a non-free format if it's smaller.
Or at least that's what I believed in after testing on 7 videos that have obvious winners either way.
1. ffmpeg exposes all of its options through the CLI, and there are a lot of options. So it's probably always going to be completely undiscoverable. It really needs a GUI to be usable, but that's a project in itself (I guess the project is Handbrake).
2. They probably didn't put a lot of work into the UX of the CLI since it's an open source project.
3. Backwards compatibility.
I don't buy this. A TUI is completely possible, e.g. LazyGit, Htop, etc, countless tools indicate you can get pane-based UIs going in the terminal; the FFmpeg team has simply never made such a thing a priority.
But even without a TUI, the most basic use-cases are well-known after over 15+ years of existence, simple prompt-based wizards, i.e. "git add -p", should be offered; again, a matter of priority rather than being intractable.
A year ago or so someone posted a subscription service they made for composing the pipeline graph visually
Sure. A TUI is basically a poor man's GUI.
> simple prompt-based wizards, i.e. "git add -p", should be offered
I agree. Wouldn't be hard. I think you may as well use Handbrake though.
There's a million terrible websites that offer file conversion services. They're ad-ridden, with god-knows-what privacy/security postures. There's little reason for users to need to upload their files to a third-party when they can do it locally. But getting them to download fiddly technical software is tough - and they're right to mistrust it.
So, there's a WASM version of ffmpeg, already working and hosted at Netlify [1]. It downloads the WASM bundle to your browser and you can run conversions/transformations as you wish, in your browser. Sandboxed and pretty performant too!
If this tool a) was updated regularly b) had a nicer, non-CLI UI for everyday users and c) was available at an easily-Googlable domain name - it would solve all the problems I mentioned above.
Browsers are annoying. They constrict the designer, they constrict the user, they're overly complicated, slow, bloated... I don't know why people keep pushing them to do things they are bad at.
I wish 20 years ago we'd made a concerted effort to make Java suck less. We'd have the universal applications everyone wants but nobody wants to put effort into. But the web was new-ish, and people didn't realize that hypertext document viewers would become an entire application platform and mini OS.
What I'd really like to see is something like FlatPak, but for all platforms. Basically it would be containerized GUI apps, but one repository for one app that serves all platforms. On Android, MacOS, Windows, etc, you would run your "flatpak add https://some/repository/ my-app && flatpak pull my-app && flakpak run my-app" (but in a GUI, like an App Store you control). And that would pull the image for your platform and run it. Since it's containerized, you get all the dependencies, it's multi-arch, & you control how it executes in a sandbox. You could use the same programming language per platform, or different languages; same widgets, different widgets; it wouldn't matter because each platform just downloads and runs an image for that platform. This wouldn't stop us from having/making "a better Java", but it would make it easier to support all platforms, distribute applications securely, update them, run them in a sandbox, etc. Imagine being able to ship a single app to Windows and iOS users that's just a shell script and 'xdialog'. Or if you prefer, a single Go app. Or a sprawling Python or Node.js app. Whatever you want. The user gets a single way to install and run an app on any platform, and developers can support multiple platforms any way they want. No more "how do I develop for iOS vs Windows"; just write your app and push your container.
Many users are in environments where its not possible to download new software (schools, work places, universities).
The browser has its disadvantages, but it is the most widely-deployed sandboxed execution environment providing incredibly easy distribution of software.
If you search for "mov to mp4", Handbrake is NOWHERE to see. The 10th result for me is a Cloudflare article explaining what's the difference between mov and mp4. The ~20th result is a book called Business Funding For Dummies (no shit). Handbrake is after these. The legend says I'm still scrolling trying to find where it is.
How is an average user supposed to know Handbrake or this FFmpeg WASM site?
They're right to distrust apps that run in their browsers too, but that hasn't stopped anybody. These days everyone is scared to death of an .exe but will happily execute whatever random code a stranger on the internet comes up with if they only have to click on a link to run it on their devices. Warnings that WASM is a malware author's dream weren't enough (https://www.crowdstrike.com/blog/ecriminals-increasingly-use...) and browser sandbox escapes happen all the time but nobody seems to care. I can't even just pick on WASM, JS isn't much better and even CSS/HTML alone is getting complex enough that it can be used maliciously.
Far far rarer than downloading a virus.
> WASM is a malware author's dream weren't enough (https://www.crowdstrike.com/blog/ecriminals-increasingly-use...)
Because it can run a crypto miner, really?
Using it for obfuscation doesn't really change anything. A coder that wants to obfuscate things could already run their own interpreter and/or use asm.js.
The presets are useful but when I'm converting an old WMV or some other ancient format I want to know that I'm not leaving anything behind.
> https://handbrake.fr/docs/en/1.3.0/technical/source-formats....
"One of HandBrake’s strengths is its ability to open a wide variety of video formats. HandBrake uses FFmpeg under the hood and generally can open whatever FFmpeg will, in addition to disc-based formats like DVD and Blu-ray."
The options Handbrake exposes are essentially the ffmpeg flags. The built in presets in Handbrake are generally pretty sensible IMO and I've rarely had to deviate.
Of course, a website would be better for smaller conversion jobs (in case you have browser restarts or whatnot). Desktop apps can block computer restarts to a greater degree than websites.
No matter how nice you make it, it will probably still lose the SEO battle against the shitty ad-ladden sites fighting to win top place for google searches of "convert X to mp3"/etc.
I had to merge a bunch of PDFs for a rental application recently and it was painful. Having to upload very sensitive docs, every site being a funnel to their paid version, etc.
FFMPEG isn't complicated (its as complicated as any other CLI tool), it's that video encoding/decoding specifically is a hard problem space that you have to explicitly learn to better understand what ffmpeg can do. I think if someone spent an hour learning about video codecs, bitrates, and container formats, they would immediately feel "better" at ffmpeg despite not learning more about the tool itself.
A week or so is not a lot of time.
Keeping the affected code visible somewhere could be useful for research purposes, but you don't want it where people or automations might unwittingly use it. If the official sources where the only place this could be found then it might be reasonable to expect them to put up a side copy for this reason, but given how many forks and other copies there will be out there I don't think this is necessary and they are better off working on removing known compromises (and attempting to verify there are no others that were slipped in) to return things to a good state.
right now I'm sure it's a temporary measure, to limit the downloading of sources.
but I really worry that later this will become normalized first, after every exposed hack withrdraw source availability for a little bit aftewards, just while 'they' check for other attacks or whatever
later on, it'll take longer and longer to put the source back up. but let's hope this is merely my overactive paranoia and everything will be fine open source is still ok.
There is value in making sure (potentially) compromised code doesn't just get used normally, but I agree that shouldn't mean totally blocking access to it in most cases.
Why would one want a "Safe FFMpeg Wrapper"?
Edit: Got it, provides a rust API library to ffmpeg. Thanks @CJefferson.
What’s the alternative? I could wrap the C API, and then try to make a nice rust interface from that, but then that’s exactly what this package does, so I don’t want to repeat the work.
I often just exec ffmpeg from whatever language I'm using (as a command line thing). Not very ergonomic, but the nice thing is that it's 1:1 with all examples and other uses of ffmpeg. But I guess it depends on how deep into ffmpeg you're doing stuff. Mine is mostly to point it at doing something non advanced with a file and that's it.
However, it seems that many programs opt to instead shell out to the ffmpeg CLI. I think it’s usually simpler than linking against the library and to avoid licensing issues. But there are some cases where the CLI doesn’t cut it.
What I do is take several diverse short video segments, like 100, concatenate them into 4 segments (example 23+24+26+27 since they have diverse lengths) and then xstack them into a 2-by-2 mosaic video.
Before, I was doing it in a single stage, but now, after some advice, I do it in 5 stages: 4 concatenate stages and 1 xstack stage.
I have not profiled/timed it so see which is faster, but it works pretty well, although I often have a lot of different weird warnings.
Just out of curiosity.. what use do you have for a 2-by-2 mosaic video?
(I'm not a MacPorts maintainer, but I've been burnt by ffmpeg API changes a couple times myself before).
Not at all the same as running a 4.x release.
brew tap bsiegel/tap
Also, it seems there is currently one in progress to drop the "6" qualifier on the ffmpeg binaries <https://github.com/macports/macports-ports/pull/23315/files> so it'll be fascinating to see if any new ffmpeg7 then subsequently puts the "7" back, beginning the cycle again
I'm making some youtube videos where I play through Demon's Souls flipping a coin to decide to equip items or not, and I wanted to have an onscreen coin flip animation and sound effect. With some effort, I created a transparent set of frames for the animation. Then with ffmpeg's filter_complex I was able to add the image sequence as a video stream, overlay it over the original video, and add a sound effect. That's on top of the existing subtitles, audio channel merging, and video resizing/compression. All in a single (long!) ffmpeg cli command.
ffmpeg is one of the true wonders of FOSS.
I'm not saying the CLI is easy to learn, but once you do learn it, you have a lot of power at your fingertips.
Well if you do want to do any notable editing, it should be easy to make proxy videos to solve that problem. A few clicks to enable, even.
*for some codecs, and with limitations.
https://forumsbtvd.org.br/tv3_0/
And their video codec testing.
https://forumsbtvd.org.br/wp-content/uploads/2024/03/SBTVD-T...
https://prdatsc.wpenginepowered.com/wp-content/uploads/2024/...
If you really want to give VVC a try, better stay with version 6.1.1 as it's the last one which has patches for enabling VVdec. You won't be able to apply them to version 7.0/git master:
https://github.com/fraunhoferhhi/vvenc/wiki/FFmpeg-Integrati...
Use ChatGPT to help you find the right command for your need.
I would love a “unless you’re a pro with hyper specific needs, forget these 90% of arguments and only use this 10% in this way” type of guide.
That’s tame for FFmpeg, likely just specifying a bunch of encoder parameters the simpler command left out for defaults, maybe with some input/output streams explicitly spelled out. If you want to look at really incomprehensible FFmpeg commands, try anything with filtergraphs.
That would give you hallucinated commands, not commands that actually exist or make sense. Better read documentation or ask experienced humans.
However the wiki pages can be quite good if they cover your use case and the real strength is so many examples online. Even if many of the example command lines feel like they have been cargo-culted through the years and no one actually understands what exactly they do.
I don't like interacting with command line parameters in general, it feels clunky to me, but I don't think there's a point in arguing about it since it is more of a personal preference
But then again, my experience is that ChatGPT is dreadful at everything but the simplest anything.
The fact that these glorified Markov chains manage to fool people into thinking they posses some kind of actual intelligence or ability to reason baffles me.
Have you tried Claude 3 Opus (or even just Claude 3 Sonnet)?
I know they use some of the same encoding libraries.
The ffmpeg command line utility is "just" an interface to those libraries.
Handbrake doesn't do any of that. You can't even drag a bunch of audio files on to handbrake because handbrake doesn't do audio, while ffmpeg is great for encoding audio.
The fact that Handbrake doesn't expose the same features as the ffmpeg CLI tool is frankly irrelevant.
It's not just relevant, it's the whole thing. They asked for an ffmpeg GUI and someone recommended a GUI that doesn't use ffmpeg and doesn't do what ffmpeg can do. ffmpeg can not only rewire channels, it can stream video, capture video from the screen, capture video from a tv tuner, overlay text etc.
Also the libraries you listed are part of the ffmpeg project. They come from ffmpeg.
Also they asked for a GUI to ffmpeg, not necessarily a GUI to the command line tool.
Gah. I give up.
> someone recommended a GUI that doesn't use ffmpeg
Handbrake uses ffmpeg.
But I recognize your username. I don't remember from where but I remember reading or having a conversation with you which went nowhere. I think I'm done.
It's using ffmpeg.
That's the problem.
The thing I have a problem with, specifically, is your statement that Handbrake doesn't use ffmpeg. That statement is incorrect.
I have no problem with the statement "Handbrake doesn't make a good ffmpeg GUI, because it only exposes a small part of what ffmpeg can do". That part is totally 100% fine.
I have a problem with the statement "Handbrake doesn't use ffmpeg", which you claimed here: https://news.ycombinator.com/item?id=39943516.
And I have a problem with the response "ffmpeg is a lot more than just a wrapper around the libraries" (https://news.ycombinator.com/item?id=39941964), given the fact that ffmpeg project is the libraries and that the ffmpeg CLI tool is just an interface to them. "ffmpeg is more than a wrapper around the libraries" is strictly speaking true (because ffmpeg is both the libraries themselves and the "wrapper" ffmpeg command line tool), but it doesn't make sense as a response in context.
Do you understand? Or do I need to break it down further?
Don’t think anything exists like XLD for FFMPEG video where you can just drop a file in set the quality and codec and get the exact same dimensiond file out every time.
ChatGPT can help with learning a lot now but the mailing lists are incredible sources of kind and wonderful (and incredibly knowledgeable) people… go there!
Handbrake, Permute are super as mentioned… I’ve put down a couple to add to the list.)
Also it's not like apt, all it really does is download and run an MSI with a bit of fancy glue around it (which is why it can break so badly), you could probably install the latest version of this by manually pointing it at the right URL, so it's more annoying for it to be substantially behind.
Crazy how much DirectX (DXVA) support got added.
- DXV DXT1 encoder
- LEAD MCMP decoder
- EVC decoding using external library libxevd
- EVC encoding using external library libxeve
- QOA decoder and demuxer
- aap filter
- demuxing, decoding, filtering, encoding, and muxing in the
- ffmpeg CLI now all run in parallel
- enable gdigrab device to grab a window using the hwnd=HANDLER syntax
- IAMF raw demuxer and muxer
- D3D12VA hardware accelerated H264, HEVC, VP9, AV1, MPEG-2 and VC1 decoding
- tiltandshift filter
- qrencode filter and qrencodesrc source
- quirc filter
- lavu/eval: introduce randomi() function in expressions
- VVC decoder (experimental)
- fsync filter
- Raw Captions with Time (RCWT) closed caption muxer
- ffmpeg CLI -bsf option may now be used for input as well as output
- ffmpeg CLI options may now be used as -/opt <path>, which is equivalent
- to -opt <contents of file <path>>
- showinfo bitstream filter
- a C11-compliant compiler is now required; note that this requirement
- will be bumped to C17 in the near future, so consider updating your
- build environment if it lacks C17 support
- Change the default bitrate control method from VBR to CQP for QSV encoders.
- removed deprecated ffmpeg CLI options -psnr and -map_channel
- DVD-Video demuxer, powered by libdvdnav and libdvdread
- ffprobe -show_stream_groups option
- ffprobe (with -export_side_data film_grain) now prints film grain metadata
- AEA muxer
- ffmpeg CLI loopback decoders
- Support PacketTypeMetadata of PacketType in enhanced flv format
- ffplay with hwaccel decoding support (depends on vulkan renderer via libplacebo)
- dnn filter libtorch backend
- Android content URIs protocol
- AOMedia Film Grain Synthesis 1 (AFGS1)
- RISC-V optimizations for AAC, FLAC, JPEG-2000, LPC, RV4.0, SVQ, VC1, VP8, and more
- Loongarch optimizations for HEVC decoding
- Important AArch64 optimizations for HEVC
- IAMF support inside MP4/ISOBMFF
- Support for HEIF/AVIF still images and tiled still images
- Dolby Vision profile 10 support in AV1
- Support for Ambient Viewing Environment metadata in MP4/ISOBMFF
- HDR10 metadata passthrough when encoding with libx264, libx265, and libsvtav1
I think I read about this a few months ago but don't remember the details. What exactly does this do? Does it result in faster encoding/decoding if you have multiple filter graphs (for example a single cmd line that transcodes to new audio, extracts image, creates a low res)
> - ffmpeg CLI loopback decoders
No idea what this is...
Edit: threading => https://ffmpeg.org//index.html#cli_threading, loopback => https://ffmpeg.org/ffmpeg.html#Loopback-decoders
Loopback decoders are a nice concept. So could I use this to create a single ffmpeg command to extract images periodically (say 1/s) and then merge them into a horizontal strip (using the loopback decoder for this part)?
CGPT said: ffmpeg -i input.mp4 -vf "fps=1,tile=3x1" -frames:v 1 output_stitched.png
Gemini: ffmpeg -i input_video.mp4 -vf "fps=1,scale=220:-1" -c:v png output.png
I wonder if this also means that Chrome and Edge will be able to use this acceleration for their ffmpeg backend (instead of relying on MediaFoundation)?
> dnn filter libtorch backend
What's ffmpeg's plan regarding ML based filters? When looking through the filter documentation it seems like filters use three different backends: tensorflow, torch, and openvino. Doesn't seem optimal, is there any discussion about consolidating on one backend?
ML filters need model files, and the filters take a path to a model file as one of their arguments. This makes them really difficult to use, if you're lucky you can find a suitable model and download somewhere, otherwise you need to find a separate model training project and dataset and run that first. Are there any plans on streamlining ML filters and model handling for ffmpeg? Maybe a model file repository with an option of installing these in an official models path on the system?
Most image and video research use ML now, but I don't get the impression that ffmpeg tries to integrate the modern technologies well yet. Being able to do for instance spatial and temporal super resolution using standard ffmpeg filters would be a big improvement, and I think things like automatic subtitles using whisper would be a good fit too. But it should start with a coherent ML strategy regarding inference backend and model management.
I think I did updated my code with the new channel layout API. But it was a year ago at least. There is another API which is supposed to change, the seeking API but I wonder if it is now stable enough to be used.