I created this Hor/Vertical scanline driven paintbrush video renderer “Splash”

Splash is an alternative approach to traditional top-left to bottom-right scanline image/frame rendering. It sorts the pixels of a frame in order of significance, and paints them guided by weighted rows and columns. The longer the rendering, the higher the detail, making it highly tolerant against data corruption. A full frame is lossless, truncated frames are lossy. Splash shines in it’s lossyness.

Inspiration: The fractal zoomer rendering engine of my other project [https://rockingship.github.io/jsFractalZoom] wondering how it would be with real-world imagery instead of procedural generated content, and how I believe my brain perceives movement: you pay attention first to areas with heavy contrast changes because they are most likely caused by moving things.

Explanation by example:

STARSHIPS:

14 second clip with high contrasts and lots of movement in different directions. Top video show frames from the a 1212x510 input source and rendered top-left zig-zag to bottom-right. Each frame contains 618120 pixels, Splash takes 618120 self-chosen pixels spread over 6 frames and paints them in the bottom video. Top and bottom both have identical pixel data rates.

[https://rockingship.github.io/splash-media/starships/starshi...]

GTA:

Lets explore the extremes by assuming there is data corruption and only the first 1% of the frame was recoverable. Left is Splash and what it’s continuous paintbrush rendered. Right would be the traditional first 9 scanlines of the input making if effectively unrecoverable. To give traditional at least a fighting chance, the frame is downscaled 100 times to 90x50 and upscaled again to original 900x506.

[https://rockingship.github.io/splash-media/gta/gta-sbs-1800x...]

FFMPEG:

The project includes a patch for ffmpeg [https://rockingship.github.io/splash/20230313-splash_codec.p...].