Why Git Is Hard
roadrunnertwice.dreamwidth.org
roadrunnertwice.dreamwidth.org
Hm.
I love Git because its core is so simple it fits on a napkin [1]. From such a simple mental model, everything else flows. It's an extremely simple mental model because it compresses so well: from only the first principles, I can derive the rest.
Any other computer system, I try to dig and find its first principles, but nothing is quite so clean, so I have to keep in mind a bunch of exceptions and add-on warts to understand them, a larger mental load.
That said, I seem to often be the one around my teams most comfortable around Git. I can't explain it. Perhaps I've erased my initial struggle to understand it. Perhaps others don't conceive of Git as a datastructure...
[1] files=blobs (named after the SHA1 hash of its content), dirs=trees=collections of blobs+trees (named after the SHA1 hash of its content), a commit=top-level tree + parent commit(s) + commit comment (named after the SHA1 hash of its content). A branch is a reference is a pointer to the latest commit in a series. Creating a new commit adds to the top and updates the pointer.
> Perhaps others don't conceive of Git as a datastructure
The problem is that git doesn't really look much like a datastructure at first glance; it's ostensibly a shell command that you invoke, which makes people look at it in terms of inputs and outputs rather than spending time learning the details of all of the parts that aren't obviously in plain sight.
> A branch is a reference is a pointer to the latest commit in a series. Creating a new commit adds to the top and updates the pointer.
I think this is a pretty good demonstration of the issue I'm talking about; I've talked to engineers who totally understand git's underlying model but still are incredibly resistant to this definition of a branch, even though it's the accurate one. The idea of a branch conceptually being the entire chain of commits from the most recent one to the initial one is so pervasive that even the term "branch" heavily implies it (when arguably the more accurate analogy would be to call them "leaves"). When the definition of one of the most fundamental concepts defies people's common intuition, the finer details of advanced operations on them like rebasing don't have a chance of being widely understood. Git's failing isn't that it's complex, it's that the only effective logical unit to think about the system is the entire thing, and that lack of abstraction is the alien model to the way we're encouraged to think of things as programmers.
Their IDEs can show them the graph you're talking about in a much richer form, lets them navigate through it, check what each commit did, who did it, and even change the graph (as everything in Git is changeable) by removing/rephrasing commits for example. I don't see how using the terminal actually helps anyone understand git, quite the opposite.
Git Tower is actually a great tool, but I believe Gits core concepts cannot be understood well without lots of experiments with the bare Git CLI (what helped me most was making toy repos → testing my hypotheses on how things work → observing/inspecting results).
Of the things mentioned in that comment, only SHA1 is really an implementation detail. The rest, especially how commits and branches relate to each other, is basically the core model of using git. The only thing I'd really adjust in that statement is condensing down the description of trees to "commits are a snapshot of the repository" (as opposed to a diff, which a lot of people seem to think commits are despite it being a niche model in VCS design).
(To give an idea, I'd condense SVN's model down to similar terms: commits are a snapshot of the repository given a revision number, and branches are copies of the folder structure, or parts thereof, conventionally under branch/)
A lot of VCS have used diffs to implement commits. But... they have also used snapshots from time to (every N commits) in order to make checking out old commits more efficient. Apparently this is how Mercurial works and how either Subversion or CVS implements it.
This is based on some article I read yesterday. I could dig it up if this turns out to be wrong.
If you want examples of VCS that treat commits as diffs conceptually, look at darcs and pijul.
I don't think so. Branches are the more accurate analogy because two branches can have many commits on top of each of their common, parent commit - so branches grow as tree branches do, not leaves.
That’s exactly the mistake though - branches (the implementation detail) don’t grow. Branches - under the hood - are “one pointer to one commit”, and they are always the same size.
The suggestion to call the implementation “leaves” is because you get a pointer to one commit at the end of the branch, and you can then follow that branch backwards (via the linked-list of parent-commits) until you reach the trunk. But the “branch” is never actually stored anywhere, it is only implied by its ends.
What about it leaks? I think this term is overrated. https://news.ycombinator.com/item?id=37752291
> The problem is that git doesn't really look much like a datastructure at first glance
All VCS have a data structure. It's inherent.
"having a data structure" is very different than "being a unique data structure that you won't encounter anywhere else". I don't think most programmers have implemented their own version control systems, so the idea that needing to learn a new data structure to learn git isn't obvious even if you consider it "inherent"
Edit: it’s not a tree it’s a directed graph. But I find tree a very simple approximation that most people without a computer science degree can understand.
Believe it or not this actually fits into the model, albeit with a little more complexity.
HEAD is a special label that can point to other labels, specifically to "heads" which are the labels that represent branch tips.
Normally when you make a new commit, the current pointed-to branch head advances forward to point to the new commit, and the HEAD pointer follows that branch head pointer.
"Detached HEAD" is whenever HEAD points directly to a commit rather than a branch head.
When making new commits in this state, there's no branch head to advance, so HEAD itself will advance to the newly-created commit. However this is a little dangerous because once you switch HEAD to something else, commits in the DAG without a branch head at the tip/leaf would be considered "unreachable" and would be invisible in logs, and eventually will be deleted by the garbage collection process.
It's still ultimately all about operations on the commit DAG and labels/pointers therein, which to me is really elegant.
What is not necessarily clear from that mental model is what any given operation will actually do to the files in your repo (called the "work tree"). But Git is very consistent here and usually tries to keep the work tree in sync with whatever is ultimately pointed to by HEAD, unless you specifically ask it not to (as with `reset --soft`).
> not being able to easily start a new branch from an older commit
Unfortunately this is a case of CLI docs obfuscation:
git branch new-branch old-commitBut people also complain about the “branch” term so maybe they would like the more internal “head” term? In which case we're back to square one.
But I'm also curious what you mean by doing work on a previous commit and then merging with main. Maybe it's a way to solve a problem I hadn't considered before.
1. Consider doing a bisect to find when a bug was introduced (which may now be in multiple branches) and being able to fix it immediately then merge and/or rebase over the current versions.
2. Working on a branch, you realize that you'd like to try an alternative approach to a problem. You check out a previous commit, leaving the branch in its non-working state, and try your alternative approach. If it works, you keep going on this version, otherwise you can just switch back. The temporary unnamed branch continues to exist until you choose to delete it (ideally before pushing!) . This unlocks a much lower inertia way to do small experimental work without having to formally label new branches, which carries enough weight to make me hesitate. I would do this all the time in Hg and almost never in git. In git I would probably reach for stash to achieve something similar.
Nah, really, it is terrible.
Git is one of the best examples of "great tool with terrible API".
Funniest thing is that people always argue "learn git internals and it'll click"
Yea, git is the only tool that people argue with straight face that you should know internals of software and act as if it wasnt sign of terrible API design.
Imagine if people acted as if you needed to know Visual Studio's or Firefox's internals in order to use those tools even in "far from very advanced scenarios", rofl.
The other big issue is that people tell you not to use a GUI, and there are a lot of GUIs that are not very good. Using a good GUI (I recommend GitExtensions, GitX on Mac or Git Graph in VSCode) makes Git waaaay easier to understand because the model is literally drawn for you.
Find me a Git tutorial with no diagrams and I will change my mind.
Also the author's "these are core facts about Git" list is not very accurate so I'm not sure how much they really understand it. A commit is not a snapshot and a patch. It is literally only a snapshot.
Edit: I actually searched Git tutorials and there are a surprising number of terrible Git tutorials with no diagrams! Though I guess I shouldn't be surprised about the official Git tutorial and W3Schools.
When you do cherry-pick, you supply a commit id, but actually a patch (diff between the commit and its parent) is applied. So yes, commit (/id) can mean a patch in some cases.
Like, is an ordinary file a patch? No of course not. But you can still turn it into a patch by diffing two files.
In the same way a commit is just a set of files. But you can turn it into a patch by diffing two commits.
> commit (/id) can mean a patch in some cases.
It doesn't. It means "which snapshot would you like me to turn into a patch by diffing with its parent snapshot?"
Also, you absolutely should understand how a debugger works to get the most out of Visual Studio. If you understand how the clutch and torque setting on an impact driver works you will have a less bad time using one and the tool will last longer. <add your own car analogy here>
The higher level commands are not individually leaky abstractions. But you can't connect them at a higher level of abstraction. In order to understand how "git foo" relates to "git bar", you go through the lower layer.
So git has you learn:
1. The fundamentals.
2. A thousand higher-level commands.
3. How each of the higher level commands translate into the fundamentals.
Car analogy: I drive cars and leave the rest to the mechanics.
As a software engineer you have the unique privilege where the people that make your tools have literally the exact same job as you, so you can pretty easily understand their work.
As for understanding Visual Studio's or Firefox's internals, that's basically what you do to write Visual Studio plugins or apps that run in Firefox.
Just because you should understand internals, it still doesn't mean that git's cli isn't terrible.
Also I don't have time nor desire to understand internals of all software that I use
do I have to learn internals of github, jira, email client, teams/discord/slack/irc, power point, etc, etc?
They have similar or higher value to git in work setting (I mean I could switch to other letters management system)
They wrap their internals with an interface that has well documented entry points, but you don't need to delve into them to get a lot of value out of them.
> As a software engineer you have the unique privilege where the people that make your tools have literally the exact same job as you, so you can pretty easily understand their work.
That's not how that works. Writing frontend apps doesn't teach you MUMPS.
_
At the end of the day Git gets "easy" but honestly most people who say Git is easy are like people saying JS is easy: the people who know either well enough will claim anyone who thinks either is easy simply doesn't know them well enough
Linus himself: Generally, the best way to learn git is probably to first only do very basic things and not even look at some of the things you can do until you are familiar and confident about the basics.
Imagine if someone was working on a SQL database and they thought that an `update` would not change anything in-place but instead would create a new immutable row and that they could reset back to the previous row (before the update) right after that since the immutable row (they thought) would still exist until some arbitrary garbage collection time. You would end up having to explain the same thing but kind of reversed.
Other equally-powerful tools (mercurial, sapling, etc) have considerably better UX though
Curiously, the latter also let me setup different accounts that I can switch between with a simple dropdown, which was otherwise annoying when you have Gitea, GitHub, GitLab and others to manage, way easier than https://docs.github.com/en/account-and-profile/setting-up-an...
Either way, suddenly you see the graph of your repo and most of the common actions are a click away, you can just let your brain idle and think about other things you're doing instead, in addition to that working really well with staging chunks of your code, or individual files, cherrypicking and so on. Even Git LFS or ignoring stuff is a context menu away.
Then again, personally I prefer squashing in merge/pull requests instead of rebasing, or even just doing regular merge commits and leaving the history as is (which doesn't really scale, but I haven't gotten to the point where that matters that much), so how I use Git won't work for everyone.
The problem is that the documentation and the developers go between these modes so fluently that you have to know the context in order to distinguish them. Of course to them it's obvious that you are treating the incoming changes as a “changeset” since it's a diff (a patch) from an email (not a snapshot).
This is the “name” section for `man git-rebase`.
> git-rebase - Reapply commits on top of another base tip
How should someone new guess that this means to (effectively) take the diff of each commit and applying those changes? Instead of working on the snapshots?[1] Because I think you could also interpret it as: take the tree (the snapshot) of each of these commits and make an alternative subgraph where the parents are different, propagating from the new base tip. Which means that you can make a new root commit (disregarding old history by “squashing” seems to be popular on SO for some reason) starting from A by rebasing A on `--root`. But it doesn't work like that.
If you want to do that you should instead use git-replace.
In my experience, for every little question about cherry-picking or rebasing, you have to go down to the fundamental concepts. Because the way people work is not like a sponge which only absorbs the information that it is given; people learn 10% about something and then automatically infer the rest, and sometimes 50% of that is wrong. So then you have to unnest that.
I guess this is my long-winded way of agreeing with the author.
[1] Working on the snapshots would also be useful. You could use that to format all of the commits on the branch.
I've been using it daily for years now, but I started out with subversion which was quite nice because it had TortoiseSVN.
Later went to Mercurial which was pleasant to work with but since GitHub arrived to the scene the choice both private and professionally became the obvious. Besides git and hg are IMHO not that different anyway from a daily use pov.
Without using any external tools. You've got 5 minutes. Should be plenty of time to delete 1 file and/or figure out how to do that simple task right?
Good luck.
Delete every mention of a single file in all history of work of a large team over multiple years.
How does that sound trivial?
So I would dare to say that your example is not entirely a good one? In my 10 years of using Git, in both privately and professional this has happened exactly zero times. But that's just me.
To use an somewhat odd analogy, it would be akin to asking the average car driver to replace the crown wheel in the differential. I'm certain everyone can do it with enough youtube tutorials and practice, but.. I digress.
But actually it makes sense, Git is like a car, once they understand what basic problem it tries to solve and learn the commands to push, pull, commit, rebase and merge, it's enough as learning navigating traffic and use the car from point a to point b.
So I believe git is easy enough for normal, but of course when situations like your example occurs it's harder (but understandable considering what problem the tool is trying to solve).
I am not being condescending here, it's my real world experience that tells me that very few people have issues with git, and looking at how many use github/gitlab/gitea/etc everyday shows the same.
Unless you mean some intentionally weird scenario, where you want to just delete the file to prevent it from being accessible period. But that's (afaik) against the whole concept of git, and so verry much outside the intended use.
Do I understand your example?
You could say "well don't commit secrets files" but that doesn't immediately solve the problem.
How about: using the right tool for the job?
In this case, the right tool would be “git filter-repo”.
"Without external tool" Why? Git defines a data structure on disk and not a monopoly on imolementation. The right tool for this job would be BFG.
git-filter-branch is the only mass-rewrite helper tool that I know of in git(1) and it's deprecated.
It’s safer, faster, easier to use.
Oh, and not built-in.
But you're right. filter-repo is a great tool. You can even use pleasant Python "callbacks" instead of a hodge-podge of env variables and shell snippets.
If a point is to be made that confirms OP: removing a file completely isn’t trivial, and thinking one did when one didn’t it is too easy.
Who recommended it?
I’m not saying they recommend it, as the question drives a rhetorical point. But it’s worth mentioning “git filter-repo” as a solution you will not stumble into as easily as the more difficult, brittle and slow “git filter-branch”.
It’s hard to compare when you knew a conceptually simpler VCS before learning git, since you have no clear memory of getting blasted with the full conceptual model of git at once.
I had to teach git to two colleagues recently, and it helped greatly that they could start with GitHub Desktop, since they also weren’t command-line veterans. The two concepts that were hardest for them to stick was
(a) that there is a difference between “main”, “origin/main” and the remote “main”: the idea of a local copy of a remote branch that is different from your local branch seems abstract and unnecessary until you have merge conflicts and you need the conflicting code to live somewhere locally.
(b) that you want to branch off from “main” instead of whatever branch you’re on right now, when you start on a new thing. This is true when you do a pull request and then begin something new.
I talk about branches as parallel dimensions and merging as portals between them.
I don’t remember when it happened, but there comes a point where you go from “it’s hard to use git without screwing up and having to start over” (https://xkcd.com/1597/) to “it’s hard to use git and screw up”. Having “git reflog” makes you quite invulnerable which makes you dare rebasing more.
Reaching consistent undo should happen sooner than reaching complex history rewriting ability, because it greatly affects what you dare do.
> This in turn means that you need to be comfortable and fluent in a branching many-worlds cosmology, so you can distinguish between changes and snapshots that have the same intent and content but which are completely non-interchangeable and imply entirely different flows of historical events.
"Graph of history and causality", context, content, "many-worlds cosmology", changes, snapshots, "flows of historical events"... This makes Git even harder.
Thank you for saying that. I had to pretend I didn't read it because it made me even more confused, and assumed it was because I was an idiot.
> Sometimes this devolves into me talking about wave/particle duality as a crucial metaphor for getting through a rebase intact [...]
Wave particle duality as a crucial metaphor for a functionality of Git?!
This is nonsense.
> While doing so, I have often failed to get the basics to stick, which is incredibly aggravating to someone who prides themselves on explaining things.
[...]
> So, I’ve spent some time thinking about this before this weekend.
And yet, what he wrote after thinking about it only muddied the waters for me, not clarified. Which tells me that git is fundamentally really difficult to explain.
Thank you, that resonates
Speaking from personal experience, if you already learned Mercurial, you might find it annoying that commands with the same name do different things, or worse: almost the same thing, but it's practically the same[0] system. Coming from a centralised system to a decentralised one is a bigger jump but understanding the 'distributed' part of "distributed version control system" seemed to me (speaking from personal experience) pretty straightforward, compared to the V, C, & S.
It's just that a lot of the points made here seem only loosely git-related? Not that that makes them wrong or invalid, by any means -- they do all apply to git. Fully agree that teaching people `git fetch` as a separate thing to `git pull` is probably wise.
[0] the same: git doesn't believe in curtains, mercurial just painted and doesn't need curtains
I am extremely competent with a couple of other version control systems, but that doesn't really help me in terms of understanding git.
I agree that many of the attempts to explain it that I've read (including a number of the comments here) are really explaining things that all version control systems have in common. For me, those aren't the parts that I struggle with. It's how those things map onto actually using git that I find difficult.
It throws up this dialog saying "there are conflicts", and by the time you see this, it has already monched a bunch of files in your local version and now the only way out is through. You need to drop everything and fix all this stuff right now before you can proceed. Even if it's last thing on Friday night and you'd rather just go home and try again on Monday.
If there was just an "Abort" button that stopped trying to push or commit or whatever and just put things back the way they were before, that would take all the worry out of it.
Because usually it would have been easy to avoid the conflict if only you knew it was coming before Git f'd your whole project.
They're hired to write software, not fumble with ancillary tools that they've been using incorrectly for over 10 years.
Either use the cli effectively, without messing up simple tasks, or use a gui that will teach you how.
There's surprisingly a lot of resistance even though we provide Gitkraken licenses for free.
To be clear, we have a few people using only the cli, we don't bring this topic up unless it's a problem.
And yeah I too find a mix of GUI and CLI works great.
This works great.
One exception: if someone does a local rebase on a feature branch, GitHub Desktop will recommend “pull” from their remote instead of “push —force-with-lease”.
git merge --abort
For example, the "branches are more like bookmarks" point. Well, I think it would have been easier if git called them bookmarks, like Mercurial.
About the "fuzzy identity" part, it is mostly a consequence of git fuzzy naming scheme. Like the "git reset" command that does many different things, the result is that no one know what a "reset" is.
I have never struggled with the idea that a commit is a "worldline". I think that calling it a worldline is making it more confusing than it is. For me, it just represents some work done, but the git tendency to mix high level concepts with implementation details gives people weird ideas.
> The forward slash is sometimes a special namespace separator to denote a cached remote branch, and sometimes not.
that's something i have to keep looking up.
Well that clears it up! /s
I think trying to avoid talking about SHA1s of content makes everything blurrier, not clearer. Because them people say things like this, which really don't make sense and instead seem like speculative SF:
> A commit is its entire worldline
Honestly, most Git problems I see stem from long-lived branches. Git flow is a common culprit.
Another variation is that you reify branches according to whatever team that is working on it, like dev and qa and release. As in they have these literal branches and they need to follow a workflow diagram in order to remember what to merge into what and in what order. Then they mess something up and they go to StackOverflow to ask such easily inteligble questions like:
> We are using a standard branching model. But sometimes we find a bug in release/5613248 and need to deploy a hotfix. So naturally we branch hotfix/5613248 from release/5613248, fix the bug, do a PR against release/5613248/joe-the-gatekeeper, then Joe merges that into release/5613248 and also into pre-release/5613248, qa, and finally into develop. But sometimes we get weird merge conflicts when merging into for example qa or develop. How do we avoid that?
Slight hyperbole here.
Basic git isn't too bad. A learning curve, but manageable.
However, next thing you know you're in a sprint and your work depends on somebody else's (process problem, but happens often) and so you are now working on top of their branch. You make some commits, and have to merge in (or rebase - force push) their changes...
... and then you come in to work after the weekend and find that they have merged their code into main. Cool ! You'll just merge/rebase main, except you find that they had also been working on top of someone else's branch, and now you have all sorts of fun trying to unpick how your changes actually relate to what is in main.
Sure, some people are just happy to rebase and force-push, but if you're happy to force-push then the 'control' has gone.
For sure it is recoverable, but not to a beginner.
I do agree that fear is a very important factor. I disagree that it was easier to do things wrong with SVN.
The times I got in a weird state with SVN was usually related to a merge. But it was the clunkiness of the tool and not fear of loosing anything or not having a way out. Last resort was just to do the merge manually. No risk to it, just tedious work.
I find that since most students just `git add .` (a habit I strongly encourage them to break) and then `git commit`, they don't see why the extra staging step exists or is useful. It's just a nonsensical hoop to jump through.
Numpy is hard. But it’s less about accidental complexity of the interface and more about the array programming paradigm being unusual for most people.
Spark is hard. But it’s because distributed data frame processing is hard. Whereas pyspark had made a lot of the actual abstractions easier to work with. Compare this to Hadoop back in the day which has both accidental and real complexity.
Maybe because it is because it was the only one, since it is the first you learned :p
I wouldn't call git "one of the best software ever written", though it certainly was influential. It was a hack job by Linus Torvalds because he was dissatisfied with the existing offering when it comes to source control. What he achieved in so little time was truly impressive, and the foundation of it is good, but it was still a hack job and I guess it gained most of its popularity because of Linux, and later GitHub.
Now, git has much improved, certainly due to its popularity, but its roots as a hack job still show. Maybe, in an alternate universe, Mercurial would have been the one, it is very similar, or maybe Bazaar, or Fossil, which have different philosophies, but are still very capable DVCSs. Interestingly, they all came out at about the same time (2005, except for bazaar, that came out in 2006).
But other version control systems are solving the same non-trivial problem, yet I don't find them hard to understand and use...
Then you come at it with your new understanding of an immutable commit DAG, and run into things that shouldn’t be there at all, yet everybody uses — rebase, cherry-pick, &c.
This article nailed it -- it's the mental model.
The much simpler `git log --graph --color` works great.