Show HN: A stupid website for animating images on top of videos (HTML5 and WASM) | Hacker News Reader

Show HN: A stupid website for animating images on top of videos (HTML5 and WASM)

madeitfor.fun

161 pointsTrevorSundberg49 comments

The original inspiration for this was a very old video from 2007, the 300 PG version: https://www.youtube.com/watch?v=gNqiSkd1M6k

I had originally named this project Cake Town as an homage to a scene in that video, however the domain cake.town was taken so I chose https://madeitfor.fun/ instead :)

The goal was to be able to animate images and gifs over a video with controls that worked easily on both mobile and desktop, all on a website. I started to realize that with more things being compiled to WebAssembly, I could take advantage of video encoding directly on the page (no server encoding required). The web is truly a powerful platform!

This is really awesome -- was interesting to see FFmpeg being used fully client-side.

I had previously used this ffmpeg WASM port (https://github.com/Kagami/ffmpeg.js) and hadn't seen the one you're using before (https://github.com/ffmpegjs/ffmpeg.js). Any thoughts on their pros & cons?

Also, I'm the maintainer of the awesome-ffmpeg list - mind if I add this to there?

Absolutely, feel free to add it!

Overall ffmpegjs/ffmpeg.js has been really nice. I would honestly prefer less API: they wrapped everything into a worker and made their own API calls, but I'd prefer the raw Emscripten filesystem and no worker/proxying. Then maybe a second library that wraps it up would be nice for ease of use.

I ran into one issue where the memory being used by ffmpeg was too great for running on an Android Pixel 3a (malloc failed). I split up the video encoding into 30 frame chunks. I don't think this was related to ffmpegjs/ffmpeg.js however.

Did you noticed a performance difference between when encoding 30 frame chunks vs whole file (barring zero completion equals divid-by zero)?

This is wonderfully executed hack, for any feedback all I would have are nits. Well done, you have made my Friday.

This is anything but stupid. Live motion tracking, ffmpeg browser encoding, plus it's open source - so a great resource to learn!

What more can you ask for? It's incredible work. Keep it up!

This is incredible. The humble little meteor icon does motion-tracking!

How is that even possible in the browser?!

Awesome!, If you want to try without the ffmpeg dependency in the future, you might have some luck with the Canvas captureStream API [1] and the MediaStream Recording API

[1] https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasE...

[2] https://developer.mozilla.org/en-US/docs/Web/API/MediaStream...

Great suggestion! I could use this with requestFrame()!

https://developer.mozilla.org/en-US/docs/Web/API/CanvasCaptu...

I think I will make this change because the ffmpeg legality seems hairy. Thank you!

This is very well done, intuitive to use, and fast - I'm impressed! Only tweak I would want is a way to move the ends of a selected timeline segment, at least for mobile.

Good suggestion. What would you think about a marker that sits above the timeline that you can drag?

That sounds perfect. Would it be on both sides of the time segment (2 handles)?

Indeed, I made a bug to track it for myself: https://github.com/TrevorSundberg/madeitforfun/issues/2

Really cool. Would this work with live video streams? Sports feeds?

Any idea how far away we are from doing this kind of stuff - https://vimeo.com/hawkeyeinnovations - in the browser. What would it take?

Edit - better example clip - https://vimeo.com/402159917

It could certainly be adapted to work on live streams, but I'm not sure what kind of setup they have to track the ball in that example. Seems decently complicated!

Yup looks complicated. I think they are using multiple cameras to work things out.

Who knows, one day maybe the browser will handle multiple vid feeds and the compute. Atleast thats what your site made me imagine.

Try the "track motion" feature, mind blowing doing that in the browser.

Be careful how you use ffmpeg. ffmpeg is LGPLed. you can call it like a DLL. You can not run some compiler/transpiler on top of it that combines it with other non GPLable code. AFAICT you must keep the LGPLed code a separate file and connect at runtime. Which means sticking in npm and running a standard bundler over it so it gets all put into a single file would either make your code illegal or require you to also release your app as LGPL (assuming the 650 other deps you probably pulled in were LGPL compatible)

This is a good point, actually this might be what ffmpeg.js is already doing. I was wondering why they went out of the way to run the worker and pull ffmpeg code from unpkg.com, but I suppose that makes sense now!

Thanks for the tip, I'm certainly no legal expert!

If you use the GPL license what codec are you using when encoding?

If I remember right the H264 encoder is not in the GPL version.

I am using H264, but I'm now considering other options!

This is the standard for GPL licenses and your focus on this particular aspect of the license amounts to FUD. LGPL, GPL and many other copyleft licenses are in common usage across the ecosystem, including the Linux kernel.

From Wikipedia[1]:

"... The license allows developers and companies to use and integrate a software component released under the LGPL into their own (even proprietary) software without being required by the terms of a strong copyleft license to release the source code of their own components. ... "

" ... For proprietary software, code under the LGPL is usually used in the form of a shared library, so that there is a clear separation between the proprietary and LGPL components. ..."

[1] https://en.wikipedia.org/wiki/GNU_Lesser_General_Public_Lice...

I am not spreading FUD and I am not dissing the license. FFmeg is great software. My only point is the standard build process for JavaScript versions will go against the license.

If you just do the equivalent of

    npm install --save ffmpeg.js
    webpack

You've may have just built something that must be LGPLed. That's a good thing to know so you can take steps to honor the license and either LGPL your stuff or keep ffmeg out of your build process and just link it at runtime as the license requires

LGPL does not require dynamic linking; what it requires is to distribute the application in a form which allows the user to replace the LGPL-covered parts. Dynamic linking is one way to accomplish this, but not the only one.

Do you have links to shipping examples of these other methods short of making your app open source?

This is what the internet needs more of

This is really cool! It’d be even better if you added “playsinline” attributes to your video elements so your videos didn’t play in fullscreen on iPhones.

Will do!

Done! I took a bit of a shotgun approach because I don't have an iOS device to test, and the documentation on playsinline seems lacking (or at least non-standard). Hope that works!

This is perfect for me, I took a funny video last night with my phone and I can edit it with this site and save it for later. Great job!

This is actually a little bit mindblowing. What are some other WASM ports for a new gen of client side apps?

https://archive.org/details/sg_Sonic_the_Hedgehog_Rev_1_1991...

This is super cool. Very impressive project. I had not realized ffmpeg could run client side. This opens up some really interesting possibilities for media editing and sharing (imagine if video sharing platforms enabled people to edit/annotate videos in the browser before publishing)

For sure! Although this project is using a worker for ffmpeg, currently it only runs one which means it's like you're running encoding on a single core machine, but with even more JavaScript overhead.

You could spin up simultaneous workers but you'd have to figure out how to break up what you're doing into separable parts. The file system is not shared between workers so you'd need to either send all the data back over or do something clever. This would be very doable, but I don't want to overload the browser especially on mobile.

Looking forward to when web threads are safe again... SharedArrayBuffer ;)

Wow this is absolutely fantastic. I'd really love to read a walk through of how you built this.

This is brilliant - I'm amazed how humble you are in calling it a stupid website. I've always wanted to do "something" with video development, just because it's always been a massive black hole for me. Any tips?

What aspect do you want to get into? Making plugins for post production? Developing new codecs? Streaming? Broadcast? Apps?

There's lots of work doing less sexy things like building workflows for transforming existing video libraries to something to use in various OTT platforms. Think B&W films or older TV programs. Understanding old video formats and how to prep them for modern platforms is something that not everyone does well. Of course that would require deep dive into the history of Film->Video.

There's also lots of options on developing supporting software to do non-sexy things like storing descriptive metadata and making it available. Transforming closed captions, subtitles and other ancillary file formats that support video.

I would definitely recommend trying something with JavaScript/HMTL5 videos. Doing anything native has always been decently complicated and there's a lot of setup work involved. If you want to get your toes wet, writing basic filters is fun :)

Very impressed by the motion tracking! I was able to track the head of the bunny with an object with 2 clicks and withing 20s.

Shared anim: https://madeitfor.fun/?data=eNq9XF2PVLkR%2FSutiVazq4Cxy3aVjc...

Haha thanks for sharing, it's the first time I've seen anyone use it and share the link so I'm pretty excited!

Great job. Buttery smooth even on a low end mobile device.

Really cool indeed. Congratulations.

Is this working on Safari? I got nothing other than a Side Bar and Black Screen

I haven't tested in Safari since I don't own any Apple devices :(

However, I'll happily take fixes for it!

You should add a comment to the thread giving the backstory of how you came to work on this, and explaining what's different about it. That tends to seed discussion in a good direction.

Other Show HN tips here: https://news.ycombinator.com/item?id=22336638

Thanks for the tip, I'll try and write something up :)

Looks like you did well!

I really want to think of something to use this for now, it's really good!

https://madeitfor.fun/?data=eNrtWVlv3MgR%2FiuEgo3WiIbqi30YEI...

I laughed way too hard at this

Reply on news.ycombinator.com