Video Rendering with Node.js and FFmpeg | Hacker News Reader

Video Rendering with Node.js and FFmpeg

creatomate.com

46 pointsfagnerbrack9 comments

"Thinking quickly, I made a megaphone by taking a squirrel and tying it to a megaphone I had."

Seriously: This is an interesting article because it's about more than just rendering video, it's about creating a new video procedurally and then rendering it with that pre-existing rendering program. It would be nice if the title reflected that.

Oddly enough, I've done something pretty similar, but using Python + PIL instead: https://github.com/notpushkin/glitzz

Two improvements you can borrow:

1. FFmpeg can read frames from HTTP URL, which you can (ab)use to generate frames on the fly without hitting the disk

2. If you do that, you can also switch from PNG to BMP, since CPU is now your bottleneck

Another way to do it on the fly is to feed image data into stdin of ffmpeg. I haven't tried with BMP with that method, so the intermediary encoding might not be spared.

Im gonna need to dive further into this, I do js canvas and webgl rendering to videos with the chrome/puppeteer via 'puppeteer-screen-recorder'.

But they do at times cause some issues. Latest 3d renders have been causing memory issues that I think would be solved with bigger boxes but havent need to investigate for a little while.

Have thought about it just outputting frames and then later having ffmpeg sticth them into a video, but havent gotten around to really testing it.

Im guessing this limited to 2d canvas, but excited to check it out. Thanks!

> Im guessing this limited to 2d canvas

WebGL/3D works fine, just with some additional dependencies (e.g. mesa drivers) and a little more setup in Nodejs to create the context and copy the framebuffer to node-canvas to do the image encoding.

Here's a little 3D animation I've rendered using a similar technique (plus WebGL) in a docker container:

https://www.carvana.com/shareyourcarvana/MjM2MTc2OTpBbGV4YW5...

The main thing to watch out for is whether you need specific WebGL extensions that might not be supported. Array instancing is the main one I use, which is supported.

Worth also looking at Remotion[0] if you want to do motion graphics via Javascript

0: https://www.remotion.dev/

Also Motion Canvas: https://motioncanvas.io

Would this also be possible with the vlc WASM project?

https://code.videolan.org/jbk/vlc.js

> There is no better way to convey your message than by using video

If you start your article with something so blatantly untrue why should I believe the rest?

Reply on news.ycombinator.com