Hacker News Reader

Comment by DocTomoe | Hacker News Reader

[flagged]

While it's correct that each Git commit is a snapshot of the worktree, and not a diff, Git uses copy-on-write / structural sharing, which means that generally speaking adding a commit to your repository is a very cheap operation storage wise.

Recommended reading if you're interested in Git's data model, it's pretty easy to understand compared to Git's UI: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects

That's my favorite fun fact about git: the diff you see isn't really a diff under the hood, but in fact it kind of is if you look one more layer under.

What? That's not how Git works, even ignoring packfiles.

(And yes I have no idea what Github's backend does, but I can't imagine "duplicate all data for every commit" would be a feasible implementation strategy.)

Many people have mentioned that this is incorrect, but I wanted to ask why you thought this, and why you thought it so confidently?

The Linux kernel repo contains 1.1 million commits and 80k files on master. That would mean a naive total of 88,000,000,000 files being stored as a “full copy of the repository”.

Does this pass the smell test?

"Git stores a copy of the repo" is not an entirely unreasonable inference to draw from "git does not store deltas".

I guess that’s my question: is it? Does that inference pass any kind of smell test with basically any non-toy repo?

I think one can (and should) look at some real repos to conclude it's not that simple, but if you're simply told it doesn't store diffs, what else would you think it stores?

I would think that either it stores a volume of data that would make it impossible to use… or it it doesn’t.

And if it isn’t impossible to use then I wouldn’t post about how it is.

That's not how git works. Unless every commit changes every single file in the repo (without ever reverting to a previous state). Most repos don't work that way.

22 million commits is a lot though; the author states that the repo was over 8GB "last they checked", and the entire thing looked to be an experiment to determine "Github's or git's breaking point" - I guess they found it.

hopefully this is a sarcastic comment. T_T

Many people have mentioned that this is incorrect, but I wanted to ask why you thought this?

The Linux kernel repo contains 1.1 million commits and 80k files on master. That would mean a naive total of 88,000,000,000 files being stored as a “full copy of the repository”.

Does this pass the smell test?

i remember reading that too https://willi.am/blog/2014/10/14/for-the-last-time-git-store...

Assuming it's correct, I think this answer explains it well: https://stackoverflow.com/a/31996121/283879

Basically, yes it may write snapshots per-file (never per-commit) locally but there is a separate routine to transparently repack the whole thing with deltas.

as pointed out in the sibling comments, that's not right, but also it can't possibly be right: torvalds/linux has about 1.2M commits, and according to https://cdn.kernel.org/pub/linux/kernel/v6.x/, the current Linux kernel source size is 131M after compression, so (assuming the kernel size has grown linearly with each commit) each maintainer would need to have 75 TB of storage on their personal machine?

Reply on news.ycombinator.com