Hacker News Reader

Comment by ks2048 | Hacker News Reader

ks2048

> This is a solved problem: Rsync does it.

Can you explain what the solution is? I don't mean the details of the rsync algorithm, but rather what it would like like from the users' perspective. What files are on your local filesystem when you do a "git clone"?

View on HN

When you do a shallow clone, no files would be present. However when doing a full clone you’ll get a full copy of each version of each blob, and what is being suggested is treat each revision as an rsync operation upon the last. And the more times you muck with a file, which can happen a lot both with assets and if you check in your deps to get exact snapshotting of code, that’s a lot of big file churn.

The overwhelming majority of large assets (images, audio, video) will receive near-zero benefit from using the rsync algorithm. The formats generally have massive byte-level differences even after small “tweaks” to a file.

Video might be strictly out of scope for git, consider that not even youtube allows 'updating' a video.

This will sound absolutely insane, but maybe the source code for the video should be a script? Then the process of building produces a video which is a release artifact?

This is relatively niche, but that's a thing for anime fan-encodes. Some groups publish their vapoursynth scripts, allow you to produce the same re-encoding (given you have the same source video). e.g.:

* https://github.com/LightArrowsEXE/Encoding-Projects

* https://github.com/Beatrice-Raws/encode-scripts

Hm, the video itself would probably be referenced by an indexable identifier like "Anime X Season 1 Chapter 5", and provisioning of the actual video would be up to the builder to get (probably from some torrent network or from DVD although no one will do that)

That is nowhere near practical for even basic use cases like a website or a mobile app.

Isn't it? In practice it means that the "video" should live outside of the git repo, you could just download it from an external repo, and you always have the script to recreate it if it ever goes down.

For example:

PromotionalDemo.mp4.script

"Make a video 10 seconds long showcasing the video, a voice in off should say 'We can click here if we want to do this, or click there if we want to go there'. 1024*768 resolution. Male voice. Perky attitude"

> This will sound absolutely insane, but maybe the source code for the video should be a script? Then the process of building produces a video which is a release artifact?

It already kinda is, but that just means you now need access to all the raw footage, and rendering a video file in high quality & good compression takes a long time.

https://en.wikipedia.org/wiki/Edit_decision_list

Which is the problem only if you see of "building" as something that should be instantaneous or take a couple of hours tops.

This is similar to replicability in science, there's some experiments that are inmensely expensive to replicate, like LHC, but it still IS replicable technically.

I see, I think in that case the raw video format would still be the source code along with the EDL. What I'm suggesting is that the raw footage would still be an output from the source code that would be the script and the filming plans.

Silly idea, but it's worth thinking about this stuff in an era where the line between source code and target code is being blurred with prompts.

A lot of video editing includes splicing/deleting some footage, rather than full video rework. rsync, with its rolling hash approach, can work wonders for this use-case.

Reply on news.ycombinator.com