Cool! I am the maintainer of a project that does very similar things. Maybe this is relevant for people trying to compose videos in JS or TS.
https://github.com/redotvideo/revideo
It's based on the browser canvas and allows for instant preview of compositions. We recently rebuilt our renderer on top of the web-codecs API which makes it super fast. Would love some feedback!
Thanks! We're using a server side ffmpeg process for the encoding right now, so running a server is required for rendering. We are exploring fully client-side rendering too but I'm hesitant to use WebCodecs because of the limited compatibility with many browsers. Ffmpeg-Wasm does look promising so I'll give that a closer look very soon. I'll also check out your project, thanks for sharing.
If you mean Moviepy, then yes, there are some advantages. As I mentioned in another comment, you can ship Revideo as part of your website and run it inside the browser. One of the reasons we struggled with Moviepy was that we couldn't preview our changes in real time which made the developer experience a bit tedious. We also like the syntax of our generator functions better, but that is of course subjective.
Thank you, I'll try to disable it! I noticed the little yanks too but didn't know where to look and therefore didn't take the time to try and get rid of it.
It really depends on the complexity of the scene. We're only getting started on optimizing rendering and I'm convinced that we can get it to be significantly more performant/cheaper than Remotion. Right now we're faster in some cases and slightly slower in others. Feel free to contact me at konsti at re.video. I would love to understand your use-case and see if and how we could help you.
We haven't played around with this yet but this is possible. There are people working on this stuff in the Motion Canvas community, which is the animation library we're based on. It should be pretty straight forward to adapt this for Revideo.
https://github.com/motion-canvas/examples/tree/master/exampl...
So far, we only have a prebuilt Player component for React, but it is definitely possible to integrate Revideo into a Vue project with a bit of hacking. I'd love to better understand your requirements and help you get started. I already did this for React so porting this over to another framework like Vue should be straight forward - feel free to message me in our Discord server (invite link is in our repository) or email me at konsti at re.video.
We tried to reach out to Jacob but unfortunately didn't get a response from him. In general, Revideo and Motion Canvas have very different goals and we were very reluctant to go the route of forking his work. We are always open to contribute changes back into Motion Canvas if that is wanted, so far we haven't seen that to be the case.
Thank you! As of now, many people use Revideo to programatically create Youtube videos (primarily entertainment content, think of AI generated memes and Youtube Shorts) and for use cases in video marketing (think automatically generating and A/B testing ads from a product portfolio). As the project becomes more mature and better, we want to enable developers to build more complex, and especially interactive video editors using the framework.
I'm glad you raise the point of reusability and people building abstractions - what I find really exciting about code as an interface for video editing (rather than GUI) is that it makes it much easier to create and share isolated, reusable components and functions. Especially as we are open source, I hope that many developers will build abstractions and share them on Github
Out of curiosity, what do you mean by "Video Creation Studio"? Would you add a visual interface for editing to Revideo?
Hi Mokshith, we talked about releasing a project using the Sieve API and Revideo before! This is definitely something we still want to do. I'll reach out!
The fact that it is JS actually implies more than just being written in a different language. You can ship Revideo as part of your website and run it inside the browser. One of the reasons we struggled with Moviepy was that we couldn't preview our changes in real-time which made the dev-experience a little tedious.
Hey! I'm not really familiar with either one so take this with a grain of salt. On the first glance, it doesn't seem like these projects are designed to be run in the browser. We're mainly focussing on the use-case of using Revideo as a part of web-apps (we're building on top of the HTML canvas API which makes it really easy to preview instantly). There might be some use-cases where a lower level implementation is more suitable.
Yes, that's us! We started out in the open source LLM space, but realized that most companies didn't really have any real use cases for custom fine-tuned LLMs. Maybe we were too early, but everyone we talked to was best served by just using OpenAI. Late last year, we started looking at other areas and became fascinated with AI-generated social media content, which sent us down a rabbit hole that led us to build Revideo. It's been a very long and painful process, but we now have a lot of really cool and smart users building really useful stuff, which is something we haven't experienced with any of our previous pivots.
Certainly! You can pretty much build full video editors with Revideo. There are many companies with video-editing at the core of their product that could have been built with us (think veed.io, invideo.io or the editor part of www.synthesia.io). We're thinking about building a template in this direction so people have a starting point when building these kinds of apps. In terms of video automation, we've seen people build apps for marketing and enterprise learning. One marketing tech company uses Revideo to generate video ads from their customers product catalogue.
Encoding the video in the browser is on the roadmap, but currently we stream the frames one by one to an ffmpeg backend process and concatenate them into a video there. This has been very reliable and fast so far. I've only just started looking at WebCodec, but it seems to be more challenging, especially as most of the APIs aren't supported by some of the mainstream browsers, so as you said, some fallback will be necessary.
The main use case is not necessarily about AI, but about putting all sorts of assets together into a video. However, we definitely see a lot of people using AI-generated voices and images/videos for their videos, which we think is really cool. We will probably integrate these APIs more into Revideo in the future.
I agree! We were definitely motivated by the emergence of AI tools for video when building Revideo - as mentioned in the post, we think that a lot of video creation can be automated using AI. Currently, there are probably some higher-priority challenges related to the core rendering library that we need to solve, but we’ve definitely already thought about building a universal client library for common AI services that are useful for generating videos (e.g. text-to-speech, text-to-image, text-to-video)
This is unfortunately not something that is currently on the roadmap. Revideo mainly runs inside the browser which is why we built it entirely in Typescript. You can check out moviepy (https://github.com/Zulko/moviepy) for a similar project written in Python.
Yes, exactly! We're also big fans of the generator function model.
In Remotion, you get the current frame of the animation through a React hook called useCurrentFrame(). You then use React to build the UI based on the return value of the hook.
By structuring it as a generator function, later parts of the function are for later parts of the animation, which makes it a bit easier to read and write. We found this to be a little more intuitive. Ultimately it probably comes down to personal preference.
LoRA blew me away the first time I looked into it. Especially since you can host many LoRA adapters at once for a fraction of the cost of hosting an entire model by sharing the base between the adapters. I built a little tool to make LoRA fine-tuning easier. The adapters export to Huggingface. You can check it out here: https://app.haven.run
I wanted to fine-tune the recently released Llama 2. To my surprise there weren’t any ready-to-use packages out there (even for the original llama) that I could just give a JSON file with conversations to train on. It took me forever to get this right so I decided to turn this into a pip package so others don’t have to go through the same pain. Enjoy!