Process videos in browser using FFmpeg compiled to JavaScript | Hacker News Reader

Process videos in browser using FFmpeg compiled to JavaScript

bgrins.github.io

165 pointspkorzeniewski49 comments

This is fantastic. Mad props to Brian for this.

While I think some of the posters here have valid points regarding performance, merely being able to do this in a browser makes this capability available to a ton of people who would otherwise be intimidated with the process of installing or compiling FFMPEG.

This is a long way from an alert() box, which in the mid-90s the only thing I saw Javascript good for.... aside from making pointless in-page clocks. Wow.

I think this is pretty neat demonstration of the maturity of Emscripten. This reveals a lot of interesting avenues one could take with legacy software in the browser to me that I hadn't thought about before. Many kudos to the creators of this project who took the time and energy to show the rest of us what's possible with a little creativity. Keep on hackin'!

Thanks! It will be interesting to see how much it will speed up as JS runtimes get faster, and as we are able to apply additional optimizations with Emscripten.

In particular, asm.js is explicitly disabled right now because of the need for memory growth. In the meantime, I'm planning on having a build with it enabled just to benchmark times on smaller files (where the growth is not needed).

It's been a while since I did any contributions to libavcodec, but I'm always surprised when someone uses the encoder parts for anything.

They've had no algorithm development for years and nobody has bothered to tune them; it's more like a disassembled Lego box than an encoder.

Of course, if you include FAAC, LAME, or x264 it should be fine. But native libavcodec/x264 with their asm disabled is already considered too slow for use, so running them under a VM must limit it even more.

(In here goes the usual statement that OpenCL is not useful for this kind of work, and SIMD or direct asm is what's needed.)

So we took a performance-sensitive embarrasingly-parallel problem and made it run on a slow single-threaded language?

Sounds like fun ;)

:) well, technically they did use a web worker in their demo, so it's on a background thread, at least.

Computation intensive applications can be done in multithread by easily using web workers. It is been around for a long time. See the following site for a quick benchmark. [1]

[1]: http://pmav.eu/stuff/javascript-webworkers/

Here are my results: 8 Workers, Test: 2^1024000000 mod 97777 = 20631, 1515 ms 4 Workers, Test: 2^1024000000 mod 97777 = 20631, 2899 ms 2 Workers, Test: 2^1024000000 mod 97777 = 20631, 5773 ms 1 Worker, Test: 2^1024000000 mod 97777 = 20631, 11508 ms

That doesn't work with Emscriptem though because web workers don't support shared state. [1]

If they manage to move FFMPEG encoding into multiple web workers I'd love to read their writeup though!

[1] https://github.com/kripken/emscripten/issues/953

Or you could think of it as taking a compute-intensive problem and distributing it widely.

Or taking a bandwidth-bound problem and completely eliminating that bottleneck.

If you've got 1000 people converting videos at once, the embarrassingly parallel isn't going to help you much on the server.

This does look like fun, and really a nice validation for emscripten, and there's a ton of practical application.

True; this seems a bit silly at first glance, but imagine how much money a YouTube-like site could save if videos were converted in the user's browser before upload, rather than by the server.

It's also an excellent new benchmark for JavaScript VMs. If we can get near-native performance on this it'll enable a whole new class of application.

So you're saying that people should never do anything unless it has an immediate useful result?

I said I thought it was fun! I probably should've made up and added /!s after all :)

If you combine fixed-point arithmetics, tiled memory access patterns, asm.js, and dynamic image processing pipeline compilation (the way that, for example, VirtualDub did it), you'll probably discover that the "slow single-threaded language" probably isn't slow at all for image processing tasks. (And WebCL is still on the horizon.)

Video encoding/decoding are in fact not embarassingly parallel. Lots of modern codecs have complex interdependencies between frames and sections of frames that make it harder to do things in parallel, as I understand it.

It might not be truly embarrassingly parallel anymore but H264 and VP8, the web's bread-and-butter codecs, both support multithreaded encoding and will comfortably max out all your cores.

This is really great. I wish I was reading this headline back in 2009 when I was working with video encoding.

If file size to download is an issue, why not build a "stripped down" version which works with just a couple of file format inputs and one output that uses a GPL codec?

I don't actually think file size to download is much of an issue - the current build is a little over 6mb gzipped, which isn't much bigger than most Youtube videos.

For building a "stripped down" version, there is a config flag "--disable-everything" that can be used, and I will investigate it a little more. We probably won't end up hosting the compiled file, just out of the interest of repo-size, but I can certainly build a "build-minimal" script.

Neat!

If this could encode an MPEG in realtime (it should be fast enough on Desktop PCs and browsers with asm.js support, right?), you could send WebCam video via WebSockets to a server, distribute it and decode again with jsmpeg[1]. Sort of like the ghetto version of WebRTC, but without all the codec hassles :)

[1] https://github.com/phoboslab/jsmpeg

It's single-threaded and asm.js still has overhead, so I doubt you're going to approach the performance you get from ffmpeg compiled natively for your machine. I expect native ffmpeg uses SIMD, too.

Regardless, there are some neat possibilities.

Heh this is awesome. Anything that avoids a round-trip to server usually is much easier to implement quickly in admin forms that use PaaS app servers...

I've been trying to get away from an old ActiveX solution for client side video playback, and this seems like it could be an excellent option. Likely going to build something based on this.

I wonder if you could also drop the size of the library significantly by dropping everything from FFmpeg except for what is needed for playback.

Playback is done through regular browser capabilities.

Wow. very cool. love the demos!

Hi guys. I have been also working on emscripten ffmpeg port, but for audio. Check out my converter at http://quick-apps.com/audio . The performance in FF is really good! I must say that native apps in browser are really cool technology playgrounds.

Neat! Did you play with using ffmpeg on audio at all? I saw you had mp4 and mpeg codecs - did you experiment with anything that would allow mp3 like libmp3lame? I have an audio processing tool that I'd love to move into the browser.

Audio processing works, but certain codecs do not come bundled with it. It should be possible to configure it with --enable-libmp3lame if you had the source alongside and had everything set up correctly. We didn't have time to try this, but the closest thing I found similar to this when researching was: https://github.com/manuels/unix-toolbox.js-imagemagick. You can see that libpng, libjpeg, etc are all being included in the output.

If you'd like to give this a shot, definitely open up an issue on Github and we can talk more about the details!

Hmm, does this solve the browser audio support issue? ;)

Use Opus, but if browser lacks support, render to Vorbis, RIFF Wave or MPEG Audio Layer 3. Obviously this isn't a serious suggestion, but I really wonder if it could work.

I was thinking libav has replaces FFmpeg? Anyway, this project is very cool. I could see web workers working a mini video grid kind of system perhaps.

We did get libav working, but it was much less stable than FFmpeg, and we couldn't get swscaler working (which meant a lot of videos would not work)

There shouldn't be that much difference between the two forks. Which parts were failing?

For swscale, was it compile problems or something later?

Also, if you have a way to run libav's "make fate" (regression tests), do you know which ones fail?

I'm not sure if we can private message on here, but if you would email me at aaronm67@gmail.com, or contact us on the github project, I can let you know our progress and where we got caught up.

No, check out http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html.

While Libav left with the server, bug tracker, etc., the FFmpeg project was restoring itself, with the help of some VideoLAN people (the FFmpeg sources are now hosted on git.videolan.org). Michael also started to merge the Libav changes back into FFmpeg every 1-2 day, with a lot of forgotten, previously rejected, sometimes controversial features, or in stand-by such as ffmpeg-mt.

The main point here is that the fork stimulated the competition, and FFmpeg became a way better (IMHO) and more complete project. Also, I must say the leader's attitude completely changed, in a very good way. This is certainly one of the most positive thing that came out of that war.

The mpv developers have written a really very good rundown of the current situation: https://github.com/mpv-player/mpv/wiki/FFmpeg-versus-Libav

That's the most impressive thing I've read on HN in a long time.

It shows how far the technologies have come. Truly a remarkable achievement.

I guess this a testament to the power of source code translation, but it's a very impractical hack overall.

I disagree completely. It is amazing that you can now encode videos, create screenshots etc without any additional software required.

Well, I'd hardly call this something usable in a production environment. It's a lot slower to say the least...

Still, I applaud the hacking :)

Some things are just better left to the desktop and not the web, I'm afraid. FFmpeg is a part of most standard desktop Linux distributions, and can easily be downloaded for other systems. Apparently installing software locally is a horrifying predicament now.

Encoding videos on the desktop with the native FFmpeg code will not only yield better performance, but I can't think of a scenario why you'd want to use as a web-based service.

Create screenshots? Really? What a complete triviality. I guess using standard software and hitting the PrtSc key is too difficult in comparison to using the browser for everything. Making screencasts is a different issue, but also one you'd do locally.

Sorry if I'm not so passionate.

> Some things are just better left to the desktop and not the web

That list changes every year, too. As JS runtimes get faster, more and more is now possible in the browser that would've been fully impractical just a few years ago. In just a few more years, I predict that stuff like video processing won't be so far-fetched either.

http://www.easyprintscreen.com saves me time firing up photoshop.

(shameless plug) more editing tools coming soon :)

Funnily enough I'm using a color pallete plugin written by @bgrins who is working on the OP project.

This looks great, but what I really meant in the original comment is creating screenshots from a video.

Porting software to multiple platforms can be a pain, and a source of bugs. If there's an update it has to manually installed by each person. Then you have to remember where you put it, or which one is in your $PATH. And if you want to send it to someone else, you have to find the right version for their machine and figure out where they can download it, or how to copy it over to them. <- These problems are all solved by having a single URL you can visit or share, where the latest version of the single codebase runs every time.

Honestly, I think this project was to show what could be done all inside the browser. This is a very cool idea, and is extremely impressive for being done in a weekend. Don't be that guy--maybe you should find some passion.

What's the practical performance? Encoding a 24-minute episode with high quality seconds is a 6-hour overnight job (at least on my machine); if this is fast enough to finish in one night then ok, but I don't fancy leaving a browser running for a week.

In our testing, in the latest version of Chrome, there is about a 75-80% performance hit. So a 6-hour native encoding would take a little over a day to compile (we're working on getting more accurate benchmarks)

You're right, you're not going to use this to encode movies or very large media files, but for encoding shorter clips, the performance isn't unreasonable.

Perhaps if you have something that can intelligently split the job into segments and communicate with other nodes over webRTC then you could have a distributed encoder that any spare machine with a browser can join.

I think that "completely new and innovative" approaches are preceded by toy projects that many consider to be impractical or simply hacks. I think that this is a pretty cool demonstration for what it is.

Ah yes, the obligatory pessimistic/negative HN comment. I can think of some things an "ffmpeg.js" would be good for. Maybe not super-high-performance, but the masses don't care. Keep in mind ffmpeg does more than just videos. It processes images & audio files too; depending on what was --enabled during compile.

- Upload a short video-clip and turn it into a looping gif without the webapp's backend doing the work?

- Take an already existing looping gif and append/truncate it

- Add watermark

- Make a ringtone from your music-in-the-cloud?(This is an idea I've been thinking about for awhile, btw)

- It'd be kinda cool to see a program like, say, Adobe After Effects or Photoshop re-implemented in a browser. This happened: http://pixlr.com/editor/

- ffmpeg also supports streaming to twitch.tv; it's the backend to http://www.ffsplit.com/. What if you could stream straight from your browser? Some HTML5 canvas window streamed via "ffmpeg.js" to some other place? Access your webcam/microphone maybe?

Reply on news.ycombinator.com