Hacker News Reader

Comment by goerz | Hacker News Reader

You can still think of them as snapshots. Git just does compression on the entire folder of snapshots, including de-duplication of data that doesn't change between snapshots.

In fact, when I teach git to students, I don't even bother with the trees/blobs, which in my view are just an implementation detail. I just tell them to think of git zipping up their working directory together with some metadata (commit message, reference to parents), and putting that zip file into its own "compressed" storage inside the .git directory. That seems to be sufficient for a good mental model of how to work with git (independently of the git's somewhat baroque command line interface, which just takes getting used to)

View on HN

This is the thing though. You're talking about snapshots which actually have duplication removed... in my mind this really fits more with the 'diff' model. I've already done the exploratory diving-into-git-internals thing years ago, so I could develop a better understanding of how things actually work.

But for newcomers who want to understand how git is working, it really makes more sense to tell them it's 'like a diff. Not exactly under the hood, but think of it like a diff for now'. This is what I've been telling people as I've mentored a number of people in getting acquainted with git over the years, and if they're curious enough to look under the hood, they'll get a better understanding of the internals.

As a programmer, what you're working with is essentially the diff. This is the easiest way to think about things initially. The fact that git is storing blobs under the hood, shallowly deduplicating blobs but still storing large chunks separately that may contain duplicate data, until it generates packfiles which do a deeper deduplication/compression, is really not that helpful. Telling people it's more like zipping is a bit disingenuous because it doesn't really explain how things are compressed more efficiently over the course of many changes.

If I have a 1MB code file and make 1000 commits of one-line changes then sure, git is initially storing large blobs representing those, but then will compress over the change set when it generates the packfile.

Compared to making a zip of the file for every change (say these are 100KB compressed) and now you have people thinking the 1000 one-line changes generate 100MB in the .git directory.

You may think that a 1MB file with many smaller changes is a fabricated example, but consider that dependency lockfiles (package-lock.json I'm looking at you) can easily grow to this size, and contain this many changes.

Not really: if you do a checkout of a snapshot into an empty directory, you expect the entire state at the time of the snapshot, not just the diffs.

It may depend on the background of who you're talking to. Programmers may be very comfortable with diffs, but non-programmers (in my case, physics graduate students) usually aren't. On the other hand, everybody is familiar with snapshots: even high school student will end up with "report_v1.docx", "report_v2.docx", etc, which are snapshots at the file level (and work reasonably well as long as you have a consistent scheme and don't need branches). I've also routinely seen less-technical people organize their research / paper writing by making a weekly snapshot of their work folder ("project-2020-04-1"). Telling these people that git basically does the same thing for them automatically with a tree-like "labeling scheme" that allows for branches tends to go over quite well, in my experience. For actually programmers, I'd be inclined to give them a more technical introduction to git's internals. I'd still point out that git stores compressed snapshots, not diffs (especially if they're older and may have previous SVN experience)

Those non-programmers are likely going to have a worse understanding of what is happening when you zip/compress something anyway, but I concede this is probably the most straightforward path if they have some understanding of what a zip is, and can't understand what a diff is. But even then I question if they should be using git, since `git diff`, `git show`, basically everything git exposes, is going to show them diffs.

A storage with pure diff would be impossible to recover if you get a error in any commit. It would also be much slower to examine the data, and newer version control do not use pure diff.

The version control system Mercurial had description about these problems on the homepage, "behind the scense", which was good reading.

I am not sure if GIT is the best solution, but at least a "pure snapshot" is okey, but where a diff storage must in practise include some snapshot logic as well.

Diff based, but with snapshot "control frame" every N commits, like video?

Obviously joking though.

As a programmer I care about diffs only when I am comparing two versions. A commit creates a new version. "Snapshot" is a distraction.

Also (for less-technical audiences), I don't exactly dwell on the de-duplication. It's just "Git makes snapshots and puts them into .git in some efficient way. Don't worry about it. Or, if you want the details, read the Git SCM book."

The diff mental model doesn't work for things like `git checkout <commit>`.

I actually haven't had a problem with this, though perhaps it's because I understand what's happening at a deeper level. You're generally referencing commits which exist somewhere in this family of commits you can view with `git log --graph`. You can easily think of checkout as the path of diffs to get there. Files at commits are still whole objects, mentally, but the thing we care about as programmers working with multiple versions are the diffs.

I have had it break down a bit more when working with stash though, because now the object you're referencing can exist outside of that graph-like commit family.

The "snapshots which are stored as deltas, if that works" part is unrelated to the diffs the git porcelain generates for you when you do a git-diff or git-show. The former is purely an implementation detail of the storage (albeit an important one), while the latter is entirely virtual, calculated from the snapshots every single time you view the data. That's why operations like git-diff and git-blame can take some time on large trees or histories (and why e.g. git-blame has various options to tweak how it tracks files across revisions, because that is not something git does), while git-log is fast.

So it only stores the difference between the two snapshots? ;)

The implementation-specific compression doesn't store deltas or diffs, it stores unique blocks of text.

Git allows for shallow clones, which would be impossible if the protocol or implementation were based solely around diffs.

No. If it chooses to compress the commits, which it if I remember correctly not does automatically for each commit, but rather occasionally as a larger step, it uses the difference to whatever it deems to be a good candidate, if it finds one. E.g. if you have a file in commit A, change it massively in later commit B, and then on a different branch create commit C that also changes the file to one very similar to the one in B, git might very well compress C by storing the difference from B to C, despite those having no direct relationship in the commit graph. It can also choose to not use a delta to a different version entirely, and this is 100% an internal implementation detail of the storage system in git (afaik one of those implementation details is that it prefers candidates that are in the same commit chain, but it doesn't have to - and it can easily jump multiple commits if that works better). If you ask git to show you a diff to the previous commit, it does not pull a diff from storage, but pulls two file versions from its storage backend (which if deltas have been used to store will resolve those) and diffs them.

No, it stores an entirely new set of references to objects, as well as some of those objects themselves (any that are not identical to previously stored objects).

You cannot look at a commit on its own and know exactly how it's different from the previous commit, but you do have the complete new state. You have to look at the parent commit's references and do an object-by-object comparison to identify exact changes. On the other hand, when you look at a diff, you can see exactly what has changed, but you cannot produce the version that came before without also having a complete copy of the current version.

I don't know that you need to teach them any of that. Version control is an abstraction. I have no clue what happens under the hood and I don't care.

To some extent, this is true. I don't feel the need to totally understand gits packing logic or the specific mechanics of the various diff/merge algorithms.

But some knowledge of how/why your tools work the way they do can be very helpful.

Some knowledge of a tools internal working can be fundamental to efficient use of that tool. At the very least it can allow you to understand or derive your useful interactions with that tool rather than simply memorize how it is used.

Reply on news.ycombinator.com