Hacker News Reader

Comment by gst | Hacker News Reader

The real problem here is:

... "and I was getting ready to commit a series of important changes" ... Before doing so, I want to merge in the recent changes from the remote master, so I do the familiar git pull. ... "maybe I’m going slightly crazy after 3 days straight hacking" ...

Do I interpret this correctly as that the author has not commited any changes for 3 days?

With SVN there may be an excuse for this, but with Git the right way is to commit as often as possible, and then squash your commits before pushing them. With such a workflow the problem would have been a non-problem - just use git reflog and checkout your previous version.

Of course you wouldn't use a git pull then, but just rebase your local commits on top of master.

Learn how to use your tools, instead of complaining about them!

View on HN

I agree that the author should've been committing far more regularly than he apparently does. In addition, "git pull" is a bad idea (you never know what nastiness someone else might have committed — "git fetch" + "git merge" is a far saner way to stay up-to-date).

That said, hats off to the author for tracking down the problem. Regardless of workflow flaws, the behavior he observed is a bug, and I'm glad it's fixed.

Can you elaborate? A lot of beginner level material explains "pull" as a combination of "fetch" + "merge." What makes the latter saner?

By doing a fetch first, and a merge in a separate operation, you're at least presenting yourself with an opportunity to check the code that you are merging.

With `git pull`, it is all done in one operation.

Of course, if you've already reviewed the code or are pulling from your own remote repository then `git pull` is likely fine.

Agreed. After a fetch, I always review the changes on the remote branch. Depending on the circumstances, I can then make an informed decision about my own code, with the following possible outcomes:

1. I might choose not to merge. The upstream code may be bad, or it may not be ready for a merge. I may need to do some more work to prepare my own code to merge.

2. I might wish to perform a fast-forward merge (if one is possible).

3. I might wish to force a non-fast-forward merge (if a fast-forward is possible). This is often a good idea, as it helps keep groups of commits related to a particular feature isolated.

4. I might wish to rebase.

I treat pulls as a sync and always rebase local changes before pushing them. Merge pulls make for a very messy and verbose history.

When calling fetch and then merge (or using rebase), you have more control over how the other changes impact your commits. Just pulling will result in a merge commit that contains all of the changes between the point you last pulled. This becomes confusing because it will look like you are adding files or making changes, when you are actually not.

The reason this happens is because git uses directed graphs and there is no link between the point at which you diverged from master and the other changes came in. The merge commit creates this. Unfortunately, it looks like you just committed everything that changed on both branches and the history can be difficult to track and/or confusing.

Fetch then merge will make the history more clear, but you still create a merge commit that will include all of the file changes (correct me if I am wrong here). git pull --rebase will take your changes out of the equation, pull in the new history and then replay them against the updated master branch. This is nice because your commit will only include (at this point) any merge conflicts resulting from your changes.

For a better explanation, with diagrams, see http://progit.org/book/ch3-6.html

> Unfortunately, it looks like you just committed everything that changed on both branches and the history can be difficult to track and/or confusing.

I don't think this is really true. You'll just introduce a single merge commit on top of both your commits and the other person's. Yes, if you diff that merge against your own work, you'll see all the work that other people committed, but I think it's fairly well understood that a merge commit is just that - you merging your changes with other changes.

It can be a bit tricky to read the log when merge commits are involved, but try the --graph option or a graphical log tool.

fetch is a fairly straightforward operation with predictable (i.e. get me all the objects and refs in the remote repo) but 'merge' actually moves the branch(es) in your local repo around.

After you do a fetch you can go on to do a merge just as though you did a git pull, but if you break down the steps then you now have the option to do a rebase or other operation as you see fit.

Do the fetch, diff against remote/branch and see what changed, look at the commit logs, and then merge.

Sure, I would agree with that, but in this case I already knew what the one commit I was pulling did. And the fetch,diff, merge wouldn't have helped. The file that was destroyed wasn't changed in the commit that was fetched.

Running

  git pull --rebase

is a good practice. That way you can abort and roll back if there are any serious conflicts.

Be careful with this though: if you have a merge ready to push, and you pull with rebase, your merge will be 'flattened' and all commits in it duplicated on your current branch. I've been bitten by this a couple of times.

Only if you actually want to rebase! While rebasing is great, it rewrites history and should not be done lightly.

It rewinds your work, fast forwards to the fetched branch and applies your work on top of it. It doesn't create a merge commit, which can pollute your tree unnecessarily.

I believe you're describing "git rebase". e.g.

    git fetch
    git rebase origin/master

instead of:

    git pull

which you can also do with:

    git pull --rebase

But really, the preferred way is to use topic branches. So if you're on branch "foo", this is how you integrate into master.

    git checkout master # because it always matches upstream
    git pull            # this will always fast-forward
    git merge foo
    git push

That does create a merge commit but it does it in the right direction (master 1st parent, topic branch 2nd). You get to see the parallel development which is good to preserve in the history. Otherwise, rebasing is nice because it keeps the history linear.

Yeah, you're right. I must have misread the question.

Didn't know about `git pull --rebase`. Thanks for that.

Just to point out that fetch + merge would have caused the same problem. In this case it was a small team, with advance knowledge of what the commit was, so no need to inspect.

But, in any case doing the fetch would have shown that the file was not modified in the fetched tree, and it would have gone ahead and overwritten my changes (without even listing in the merge log that the file was changed).

FWIW, I agree that fetch + (merge | rebase) is in general the best way to go, but I think there is a case when you a pulling in a simple fix from head where doing a pull is legitimate.

After all the documentation says this is a safe operation:

"If any of the remote changes overlap with local uncommitted changes, the merge will be automatically cancelled and the work tree untouched" (from git pull --help).

If this isn't meant to be considered a safe operation git pull should abort if there are any changes to the working directory.

"Safe" or otherwise, it's (IMO) never a good idea to merge uncommitted files.

You're losing history that way: If the merge doesn't actually work, then you've got a screwed up file and no way to roll it back.

It's one of the great strengths of git that you can commit files even if someone else has changed them. It's a bad idea to merge when you have anything significant checked out (I'll leave temporary debugging changes checked out, or very small changes, but that's it). Heck, it's a good idea to check in every few hours, to track changes your making.

Agree 100% I've seen this same type of issue occur in mercurial when people were doing what I call "all or nothing merging". You should not be merging unless you can get to the precise pre-merge state.

the real "mistake" here was not doing 'pull' it was doing 'pull' while having uncommitted changes in the working directory. I'd commit or stash before the pull/

I would have sworn that git would refuse to do git pull with uncommitted local changes.

It won't let you pull if checked-in files have been modified, but it will let you pull if you have uncommitted new files.

Came here to post this. The key thing to know about git is to always, always, always commit your changes before doing anything that affects history. This is not for data loss reasons, it's for being able to document what is happening; when you have a bunch of unsaved work, you don't know what's there and git doesn't know what's there. Therefore, it's very easy to get yourself into a state where you don't "care" about your working copy, and that's when you can do something where you lose work. If you commit before you do any merging or pulling every single operation that you perform afterwards can be reverted cleanly. And, you'll have a human readable note that documents what you were thinking at the time.

If you don't put your work into git, it can't track it for you. You can always uncommit if you don't like what you committed. In fact, you should consider your local history to be a work in progress; just like you're editing your source code to keep it clean, you should be editing your history to keep it clean. Git never deletes history, so even if you edit it, you can always get back to where you were. What you call "master" may have changed, but what was master before your rebase still exists, in its entirety, inside of git. (It's simply called HEAD@{0} instead of master. See "git reflog".)

You are correct, though I'd also point out "git stash" which does a quick no-fuss commit and is meant for situations just like this. "git stash" "git pull" "git stash pop" may not be exactly what you said but it should be enough to prevent what happened to this guy and I'd even guess it is what most new git users expect to happen when you do a plain "git pull".

For the life of me, I could not figure out why he would not save his changes before pulling stuff down on top of them. Some people still go days without saving, which is foolish in my opinion. Change a function (or similar unit), and save. Rule for dummies: save before leaving for the day (but that's not granular enough).

Even without committing, you should at the very least run `git stash` before doing a pull. That's the very reason the command exists!

> For the life of me, I could not figure out why he would not save his changes before pulling stuff down on top of them.

He appears to be using git with the svn workflow (svn update; svn commit).

OK, thanks for the reminder. Even back when I did use SVN regularly at work, we usually worked on feature branches, and seldom ran into collisions requiring that process. I'd forgotten that SVN will force an updage (merge) before commit in some cases. It had not occurred to me that somebody would internalize that as SOP.

That's not my svn workflow; it's svn diff to a patch file, svn revert, svn update, then patch from the patch file (and resolve conflicts if necessary with a decent third-party tool), then finally svn commit.

I run svn diff multiple times a day. I frequently have many different patches related to different functional areas and bug fixes. Effectively I work with local branches implemented on top of svn. I have a whole suit of scripts that automate this. Seemed like the only workable way to use svn in a disconnected fashion to me.

> Effectively I work with local branches implemented on top of svn. I have a whole suit of scripts that automate this. Seemed like the only workable way to use svn in a disconnected fashion to me.

Why don't you just use git-svn or hgsubversion and stop doing that informally?

Or at the very list use something like `quilt`, so that your changes are represented as a stack of patches on your svn "upstream"

I've already discussed it elsewhere on this site earlier, I'm not about to go into it again, sorry.

Interesting that apologising for not replying in more detail, yet hinting at where to find previous discussion of that detail, is treated with such hostility! I'll simply not reply at all in future, in such cases.

They were obviously saved. I was simply pulling before committing, which is a supported work flow of the tool. I don't think I ever said I went days without committing.

How is it relevant how long it was since he last committed? It sounds from the description like this could happen if he'd committed five minutes before.

It's still not cool to clobber work like that.

Of course it's not cool. It's a bug, and it was fixed. But the relevancy is to the title and summary. When you say "git destroyed my data" it sounds like you're saying that git lost commited data in the repo.

What actually happened here is that the working tree got clobbered. That is a vastly less problematic situation. Working trees get clobbered all the time: rogue "make clean" changes, system crashes, someone-stole-my-laptop, errant rm -rf, forgetting which tree your changes are in... I've lost working data to every one of these, and I never felt the need to blame my tools in a blog post.

So yeah. It's a bug (and a pretty embarassing one). It was fixed. Is there anything more to say? Move along.

To add, it's commendable that he tracked down the bug himself, at least I think so. Many people would throw their hands up in the air and walk away with a sour disposition, but props to this guy for working it out to the end.

If "make clean" clobbered a source directory without me making an error in my make file, I would be rather miffed.

"All software has bugs, but bugs that destroy data are pretty devastating." It's hard not to agree with this...

The point was more that if you've ever written a "make clean" rule, you've probably blown away your source tree a few times trying to do it. The software development working directory is the wild west. Bugs that destroy data here, frankly, don't rise anywhere near "devastating" in my book, sorry.

Normally in git you never pull on a dirty working dir, you either stash or commit your work and then pull. That's why this bug is unlikely to trigger normally.

Exactly what I was thinking. I'd have done:

  git fetch
  git stash
  git pull --rebase branch-name
  git stash apply

All this cp business struck me as a bit of not understanding how to properly work with git. I've never had my work clobbered by following the strategy above. But I also don't go very long between commits on my working branch.

Your workflow is a little overcomplicated.. I think the following will produce identical results:

git pull --rebase branch-name

He is using "git stash" to create a commit object for his uncommitted changes, giving him more documentation in the event that Something Goes Wrong. Git will never throw away your data, but you might get into a weird mental state and type a command that throws away your data. By creating a commit object, you can always get back to that state, and you have a little one-line note-to-self that helps you not get into a weird mental state.

I personally do a "real" commit before rebase, but "git stash" is really the exact same thing. I guess "git stash pop" is less scary than "git rebase HEAD^". Either way, you can always undo that operation via the reflog.

Some people write their entire Ph.D. research papers on a computer that's never backed up, not even once in years.

This behavior should be stomped out in first year as a mandatory course, right along CPR.

No, you don't interpret correctly. It was not 3 days of uncommitted changes, it was committing some changes after hacking for three days.

Completely different thing. The 3 days was relevant because I was tired and my first assumption is that I had done something wrong. It turns out there was a nasty bug.

Also, I don't think my workflow is that broken:

"Linus often performs patch applications and merges in a dirty work tree with a clean index."

[1] http://gitster.livejournal.com/29060.html

If I am pulling down some unrelated changes, it is not unreasonable to do that. That is one of the features that git provides.

I know perfectly well how to use git, in a number of different work flows, and I use whichever is most appropriate for the given project at the given time.

Also, I don't think my post was complaining. It wasn't 'OMG git is the worst, I'm never using it again', it was an analysis of a particular nasty corner case in git.

You referenced that article to claim that Linus does what you did. But that article is quite clear that Linus does not do what you do: He either reverts, commits or stashes first. Had you done that you would not have had your problem.

Its one thing to fuck up and get panned on HN. Its another to go looking for justification that you are right. Its yet another to select a single quote from an article to justify your position when the entirety of the article refutes your claim.

You did a great public service by calling attention to this problem (the problem being not using git properly - nobody is going to hold that against you). Don't ruin it now by getting all defensive.

Sorry, that did come across as a bit defensive. However, I think if you read the whole article carefully it explains that after doing a git pull, and git refusing to merge because of outstanding changes you then have a chance to either 'revert, commit, or stash'.

That is exactly the behaviour I expect from git, and exactly what broke down in this case.

Definitely. However, this highlights a problem with git, it should naturally guide you to best practices. Unfortunately, with git it's quite easy to fall into a natural and comfortable seeming pattern of use that has some very negative downsides, such as the author of the article experienced.

Exactly, in fact it normally does. git pull is actually really useful for enforcing this most of the time. If you inadvertently do a 'git pull' before doing a commit, and you have any overlapping changes, then git aborts the merge. This is why 'git pull' is generally a safe operation.

Those were my precise thoughts. This is exactly the kind of problem that is easy to avoid in a distributed VCS and hard to avoid in a centralized VCS.

The kind of problem where merging a commit silently overwrites a completely unrelated file with an older versions?

It was a clear case of git violating a promise of safety made in the documentation, and a nice study of tracking down the bug.

No, you interpret it incorrectly.

From the article:

   git has just gone and destroyed the last 5 or 6 hours worht of work. Not happy!

More to the point, shouldn't it be a git pull --rebase?

Ofcourse it's the victim's fault! The grandma who's been writing a letter on word loses it after an unexpected blue screen of death, ofcourse it's her fault. A guy gets stabbed at 2am at a bus stop; clearly the guy's at fault. Shoulda known better than hanging out in that part of the town at that hour. A car blows up when a woman fills it up with regular gas instead of premium; you had it coming lady, learn to use your car rather than now sitting in the hospital and complaining about it.

We are cool like that, we blame the user when out favorite tool screws the pooch big time.

Git doesn't kill code, people kill code.

No one is denying that, the whole point is git must make it difficult to kill code that easily.

A good tool must promote good habits, and make it difficult to make mistakes.

What version control tool promotes good habits?

Not just that, I believe he didn't commit anything and was using git pull to update his workspace.

Reply on news.ycombinator.com