TokenFlow: Consistent diffusion features for consistent video editing | Hacker News Reader

TokenFlow: Consistent diffusion features for consistent video editing

diffusion-tokenflow.github.io

173 pointsputtycat23 comments

long long ago i had some fun stitching frames into a panorama from some really old footage (by hand). The most hilarious trick was to upscale the footage a lot so that frames could be visually aligned, made transparent and improve the resolution. The grainy snow turned into a sharp picture with new previously invisible details

"code coming soon" I really really hope this code will be ready is setup and run. My experience with ML/AI research GitHub code is pretty bad. Most of the time you have to guess the right dependencies (and versions).

yeah it's especially annoying on which CUDA version you have to guess to make it work. Sometimes projects don't even produce conda environments properly.

I would prefer incomplete or barely working code over no code.

GitHub is the perfect place to post incomplete code, because someone else can contribute fixes.

If you're a scientist, please don't hold back publishing code just because someone else demands a certain code quality.

95% of problems would be solved by locking the entire dependency tree.

The more I see things like this, the more I wonder why there is no good video upscaler yet. Something that can really upscale old DVD or even VHS-quality video.

Topaz Video Enhance AI has barely advanced in the past few years, old versions sometimes even produce better results than new versions. And even then, some of the enhancements can be achieved with some carefully chosen non-AI filters.

We're getting there. Temporally stable video generation is not trivial and is only now slowly solved. Upscaling of single images in a generative fashion (as in, actually adding in details) works really well, you can create images with an incredible amount of detail. Doing the same for video is around the corner.

This instagram account makes some really neat stuff leaning into temporal inconsistencies

https://www.instagram.com/never_ever_never_land/

This webpage caused my laptop (4 GB) to OOM. Really cool though!

My browser tries to load all the videos at once...can't watch any of them yet.

The videos are working now! Wow! Getting closer to perfection!

In some styles this is good enough!

ohhhh, why not try CDN to serve the video

they messed up AWS access to the videos, as of time of writing

they should upload to ipfs, pin with filecoin, and let a CDN like cloudfront’s serve the video

Or... Just upload to YouTube and embed it.

And YouTube will deal with transcoding to different resolutions for different devices, auto select what quality to send depending on bandwidth, ensure it works on all popular platforms, and all the other bits of video hosting that these researchers didn't want to do.

I was assuming they had a reason for not doing the simple thing and offered something similar

Reply on news.ycombinator.com