Hacker News Reader

Comment by aargh_aargh | Hacker News Reader

Specification of ffmpeg commands can get quite complex, especially once you get to filters.

I never understood why it hasn't grown a DSL to specify the input/output/transformation. Could be JSON-based for what I care. Then we could have nice things like easy programmatic generation of these scripts and schema verification (for each particular ffmpeg version).

Also, when I looked at how LosslessCut (an Electron app) does the timeline preview thumbnails, it just calls the ffmpeg CLI app however many times it needs (once per thumbnail). With the number of heavy users of ffmpeg, I'm wondering how nobody has yet started a project to run ffmpeg in-memory as a server and avoid the process startup cost as I'm sure LosslessCut isn't the only one who does this.

View on HN

What do you mean? FFMpeg supports filters via its own DSL; see https://ffmpeg.guide/ or https://ffmpeg.org/ffmpeg-filters.html

It's not quite JSON and I have to look up the syntax every time I use it, but it's well-defined.

The general syntax is:

    [input1][input2]... filterchain [output1];
    [input3] filterchain [output2];
    ...

to specify chains of filters that link input streams to output streams.

For example, the following:

    ffmpeg -i INPUT -vf "
      split [main][tmp];
      [tmp] crop=iw:ih/2:0:0, vflip [flip];
      [main][flip] overlay=0:H/2
    " OUTPUT

is doing something like this (python pseudocode):

    main,tmp = input.split(2)
    flip = (tmp
       .crop(iw, ih/2, 0, 0)
       .vflip())
    out = overlay([main, flip], 0, H/2)

Here's a more complicated example (visualization on https://ffmpeg.guide/graph/demo ):

    ffmpeg -i ./long-video.mp4 
    -i ./background-music.mp3
    -filter_complex "[0:v]trim=duration=30[out0];
      [0:a]atrim=duration=15[out1];
      [out1][1:a]
         amix=inputs=2:duration=longest:
         dropout_transition=2:weights=1 1:
         normalize=1
      [out2]"
     -map "[out0]" -map "[out2]"
    ./shorter-video.mp4

Pseudocode:

    # input files (-i options)
    in0 = read("./long_video.mp4")
    in1 = read("./background_music.mp3")
    # filter graph
    out0 = in0.video.trim(duration=30)
    out1 = in0.audio.trim(duration=15)
    out2 = amix([out1, in1.audio],
        duration="longest", dropout_transition=2, 
        weights=[1,1], normalize=1)
    # output stream mapping
    out_stream[0] = out0
    out_stream[1] = out2

That'd be a great solution, I'm using ffmpeg in my personal media server app to generate thumbnails and transcode media on the fly and I was wondering that too.

There are also a few things that I had to patch on top of ffmpeg (for example it doesn't do well with generating singular HLS packets on demand). Would be nice to have a pluggable architecture that can link multiple things together like that.

Might implement this myself if I end up having the time for that.

I only found this project focused on streaming but it's unmaintained (some summer of code or thesis project, IIRC):

https://trac.ffmpeg.org/wiki/ffserver

Interesting that it uses a similar configuration syntax to Apache HTTPd.

Seems like writing a script wrapper around ffmpeg could easily solve the first problem. All it needs to do is to read in JSON/YAML/TOML/etc. then parse them into command line equivalents and call ffmpeg. That is a kind of script pretty much any coding LLM could generate.

I was watching a streamer trying to use ffmpeg in a streaming configuration and it didn't seem to work very well in that case. However, if someone was belligerent enough, I would guess it would be possible to use libavformat/libavcodec directly to parse out frames from video. For example, the streamer "sphaerophoria" posted a short series where he built a video editor from scratch [1]. I believe the code he wrote could be used as the basis for a custom-built editor to extract thumbnails from videos.

1. https://www.youtube.com/watch?v=l_hD99zpPBY&ab_channel=sphae...

If they are such a heavy user why don't they just use libavcodec directly?

Reply on news.ycombinator.com