Moving from SVN to Git in 1,000 easy steps
codeascraft.etsy.com
codeascraft.etsy.com
It's scary. It's fraught with pitfalls. But in the end everyone hopes the result is a better tool.
We need more of these writeups. I personally love git. I just think the learning curve is steep. Think you know git? Next day someone teaches you some fancy new command you didn't even know existed.
I think git is really taught wrong in a lot of tutorials. They never focus on workflow and determining what works for you and your team (if you have one). Once you decide on the workflow all the git commands you need become apparent.
1) there's all kinds of crazy things you can do
2) for a given crazy thing someone's probably already done it, and if it's not too far-out there's probably a command for it.
The reflog makes uncollected garbage available to the user, meaning that if you screw up you can just roll back to an old commit, carrying with it an entire history, even if that commit is no longer referenced by the repo.
Or on a directed acyclic graph of refs; that's the definition that made everything click for me.
http://tom.preston-werner.com/2009/05/19/the-git-parable.htm... is absolutely worth your time if the above statement didn't trigger a lightbulb in your head
Your comment doesn't address the parent's comment at all. Moving VCS is hard in any team with more than a few members, especially when the change is coming from a few and not everyone.
I remember my old company moving from MS VSS to svn, all we hear was 'why, VSS works!', even though it was a nightmare compared to svn. Of course the change had a bunch of teething problems and the conservatives pointed and moaned more.
But who in their right mind would say VSS is better than SVN?
Any organisational change is hard, no matter how 'obviously' better it is.
So the new revisions were kept in Mercurial and the old ones in CVS. A slight inconvenience for the first year, but after that nearly all of the interesting revisions were in the new system.
As for the legacy systems, they already required us to maintain Windows NT4 boxes and a no-longer-supported C compiler, so the additional overhead of a second VCS was tiny by comparison.
Where I work, we've been doing some migrating from ClearCase to Mercurial. Pretty intense work.
1. Commit my code to the central repo.
My new and "improved" Git workflow:
1. Commit my code to my local branch. 2. Push my local branch to the central repo.
As we're using Git as a centralised source control anyway, what is this extra work saving me over using a "traditional" centralised source control? In other words, what's the point in using a decentralised source control system in a centralised manner, and how many teams are truly using Git in a decentralised manner?
Another benefit I've found--as a student with an internet connection that isn't always on--is that I can still commit without it.
It's also much easier to use branches in Git. If I want to try something weird, or temporarily break some code, it's trivial just to create and use a new branch.
Ultimately, the reason you're not seeing much of an advantage to Git right now is because (from a user's standpoint) it's basically a superset of SVN. Once you get used to it and learn more about it, chances are you'll start improving your workflow to take advantage of more of its features. Git can basically support the same workflow as a centralized system, but it also supports other scheme.
This makes it REALLY HARD to keep up with your team members' code, because all you see are huge commit batches at the end of the day containing tons of diffs.
Worse yet, sometimes people use rebase to hide the intermediate commits and you get one huge "Implemented X" commit.
Just commit as you go along. If you're doing something that outright breaks the build or is disruptive, then create a branch -- it's not hard (see below).
> It's also much easier to use branches in Git. If I want to try something weird, or temporarily break some code, it's trivial just to create and use a new branch.
I don't really understand why git users keep repeating this trope.
Here's my SVN branching process:
svn cp ^/trunk ^/branches/some-branch
svn switch ^/branches/some-branch
svn merge ^/trunk
> Git can basically support the same workflow as a centralized system, but it also supports other scheme.
It can, but with more work. I guess I could just alias 'git commit' to 'git commit && git push', but then why am I not just using SVN?
You can create a branch in Git without having to touch the main repository. It's a lot cheaper mentally to create a local branch than a globally viewable one. In particular, it's a lot cheaper to create a branch when you don't have to think about whether the work is important enough to deserve one, or if your crazy idea might end up being an embarrassing failure. When a decision only affects yourself, it's a lot easier to be bold.
> SVN has merge tracking, so it Just Works:
SVN can branch easily, and it remembers how a merge went down, but it doesn't know how to merge a branch back into the truck when the truck has since progressed. When SVN merges, it doesn't know if a difference between the two branches is because it was changed on the trunk or on the branch. It just sees a difference.
Git can replay the changes made in the branch on the updated trunk, or vice versa.
Git is decentralised. If you think you might get conflicts with some other developer, just ask them to push their branch somewhere public where you can fetch their work and look at what they're doing. Or they can export their commits as a tarball and bring it to you on a USB drive if you have no network connectivity for some reason. :P
> Worse yet, sometimes people use rebase to hide the intermediate commits and you get one huge "Implemented X" commit.
To be honest, this still sounds like a workflow problem. if "Implemented X" is too huge to be a single commit, then the person who pushed it was lazy and didn't bother to spend enough time to make good commits. If it really only "implements X", then even if it is big, it's at least self-contained.
You should spend time thinking about what you commit. It's not just a matter of getting your code somewhere safe; the code in your working directory is the raw material from which you create a new snapshot of the project that represents an improvement. This is really difficult to do with SVN, but it should be your default workflow in git. It's more work beforehand, but it improves code maintainability and makes bisecting more effective.
It's likely that most developers are not disciplined enough to do this for every commit while they're coding. Fortunately, rebasing allows you to defer all that work to just before you push. Not rebasing and just pushing loads of small commits makes it more likely that you will not have a clear history of progression for the project.
How the development actually happened is completely uninteresting in git. Ideally the log is a history of patches applied to the project, each of which changes only one thing, and is complete by itself (later commits can depend on earlier ones, but the reverse should not be true)
Disagree, because the commits are merged in and I can clearly walk through them. You should think very carefully about what you're committing and when--but that doesn't make rebase a license to do stupid things to make life harder for other developers, and personally I am completely and entire convinced that that's really all rebase does; I consider it one of the biggest brain damages of git and in pushing the SVN->git migration crusade (because even with rebase stupidity it's vastly better than SVN--I'd prefer Hg, but that ship has sailed) at my current employer I'm pushing to have "don't mash your commits" made into policy.
> How the development actually happened is completely uninteresting in git.
"In git" has no bearing on what is interesting in the development cycle, and it's a little arrogant to claim that that's the case. My version control system does not decide what I find interesting.
Some of the first things you'll want to look at: doing many small commits, using many lightweight branches, and using commit --amend or rebase -i on local changes to fiddle with them before pushing them. Think of each branch as the result of your work, despite it containing many commits; you can edit the branch as a whole until you get it right, and then push it, much like you'd edit the SVN working copy until ready to run "svn commit". You get to have more structure in your working copy, and keep that structure when you push to the public repository.
One big recommendation, though: learning Git works a lot like learning the UNIX command line. You can learn a pile of individual tools, but it also helps to know the philosophy of how they fit together, so you can more easily string together impressive ad-hoc solutions when you need to. I'd suggest reading more about Git's repository structure, and thinking about it more as a DAG, and how the commands fit in that model.
You say you don't get it yet and I think it's the issue. Coming from years of SVN I was in more or less the same situation (but I choosed to switch so it was obviously less painful). Then I started playing with local branches and funny stuff like that. I can assure you that once you get yet you'll never wan't to go back :)
Let's say you have 3 bugs to fix. In SVN you might work on them 1 at a time (start to finish -- hopefully no blockers!), or descend into madness and try to work on all 3 simultaneously. In git, you'd make 3 branches and work on each separately.
Oh, a hotfix came along while you're halfway through a bugfix? In SVN, you cringe: do I manually copy my changes somewhere else and revert? Ugh.
In git, you just make another branch from your clean master, do the hotfix, and pull that change into your bug branches at your leisure. In svn, this branching is theoretically possible, but not used practically -- they are too cumbersome and manual (you need to track what changes went where, and could accidentally apply them twice).
I've written more about it here, if you're interested:
http://betterexplained.com/articles/intro-to-distributed-ver...
With svn you really do not want to do this since in my experience you can only work at one thing at a time per checked out directory. Instead you have to write it down or try to remember.
svn diff >patch-saved-changes
svn revert -R .
(work work work)
svn patch <patch-saved-changesBut what I was specifically referring to is the -p flags of various git commands (stash, reset, add, checkout, ...) so you can interactively only stash away what you wish instead of the entire working directory (or an entire subdirectory of it).
The tools are more powerful, but they're also correspondingly more complex, and the net value is in my experience negative.
People simply feel more productive as the number of knobs they're turning increases. However, SCM is the least important knob in my day-to-day development. What matters to me is turning around quick changes with associated unit tests and ultimately committing code that works the first time. The less I think about SCM knobs, the sooner I can move on to the next piece of actual work.
Git stash is a killer feature that every developer can relate to, and highlights git's power.
For about 4 years now, I've been doing branch based iterative development. This is a great way for a team of engineers to work together and have a continuous release process.
Meaning, you deploy trunk and do development on 'iterative branches' (ie: you have branches called iteration-0001, iteration-0002, iteration-0003, etc...). All works happens on the iteration branches which are then tested and merged to trunk on a regular basis as part of the release process. If you have work that spans more than one iteration, you make a branch off the current iteration.
Unfortunately, subversion has a very hard time keeping track of development that way. The reason is that if you (and your co-workers) want to work on another branch off of the iteration branches (so you don't hold up a release), then merging changes from the iteration branches into your branch become a nightmare because merge tracking on your branch has no way to know what is happening on the other branches. If you don't do it right, you can end up with a branch that can never be merged back.
The way merge tracking is handled is a central design failure of Subversion. I made a long blog posting about this a while back: http://lookfirst.com/2011/04/subversion-mistake.html
Git handles all of this beautifully.
So, while today, you might not need the extra functionality of git, it is good to have a tool which can support your needs in the future.
Our QA team would then pull the developer's assigned branches into a QA repository, do a merge and test. We had no probems [2] with a team of half a dozen programmers across as many time zones (one in Russia) using that model.
[1] My current employer uses SVN and I was told, in no uncertain terms, "The use of Git is a fireable offense." The lead developer really hates Git.
[2] The problems we did have came from upper management that kept changing their minds about what to work on.
Reminds a bit of this quote in LWN from David Woodhouse (a Linux kernel developer at Intel): 'If my corporate overlords told me I had to use my Exchange "messaging" account for external email communication, they would get a quite clear 'no' in response. My response may also contain suggestions that they use certain other objects for purposes for which they were not designed.'
More seriously, it sounds like someone who had a bad experience with Git at one point got put in an undeserved position of power in which they got to enforce the results of that bad experience on everyone else.
I set technical direction and while using git isn't a firable offense, we will never support it as the official SCM system.
The reason is quite simple: in a centralized environment, it increases costs and complexity and provides no value to offset that.
We don't want:
- People hiding their work in local branches where none of the other engineers can follow along.
- Getting massive rebased commits once every day (or worse yet, every few days).
- An open-source Linux-inspired development model consisting of a massively distributed ecosystem of competing/disparate/intertwined branches.
But in any case, some of the issues you've listed represent bugs in development processes, not in Git.
> - People hiding their work in local branches where none of the other engineers can follow along.
Good developers using Git don't "hide their work" in local branches any more than they "hide their work" in an editor before they save it; sometimes you just have work in progress that shouldn't get pushed yet. (I certainly hope you have policies about when to commit, which include things like "the code compiles" or "passes tests" or "smells right"; if so, why aren't you concerned about people "hiding their work" in their uncommitted working copies?) If that work in progress has a sufficiently large scope that it needs more developers involved, then the branch may want to live in a repository accessible to others.
Before I had git-svn, when I had to work with SVN repositories, I ended up having several local copies of the repository, and doing the "small self-contained commits" thing manually by copying bits into and out of working copies. Git provided much more structure to that process, as well as making it faster and more efficient.
> - Getting massive rebased commits once every day (or worse yet, every few days).
Don't do that. You can and should push arbitrarily often.
> - An open-source Linux-inspired development model consisting of a massively distributed ecosystem of competing/disparate/intertwined branches.
If you have a project anywhere near as large as the Linux kernel, you'd be lucky to end up that well organized and optimized. But while you still have a team that can regularly communicate with each other on a list low-volume enough for people to read all the mails, you don't really need that ecosystem of hierarchical branches and repositories; nonetheless, you can get a lot of value out of Git even when using it with a single central repository.
That right there, tells me that you have never actually used git in a team environment with a good workflow enough to actually learn about its advantages and disadvantages. Sure, git is not perfect, but to boldly say that it offers no value in a centralized environment is, ah, how shall I say it... self-assured to the point of trollishness.
I, and many many other people have used git for years, both in centralized environments and not, and I can tell you that git (and more broadly, any decentralized VCS) has tons of useful features in a centralized environment compared to SVN/CVS and their ilk. Just off the top of my head, features I could not live without now:
- much more powerful branch merge tracking
- much better conflict resolution compared to CVS/SVN.
- partial commits
- history rewriting (massively useful in cleaning up before pushing) - ability to work without network connection
- ability to cherry pick commits between branches
The problems you described are failures of workflow due to bad process and/or bad developer education, _not_ failures inherent in the design of any given distributed VCS. I have never had to deal with any of those issues, because we have always ensured that we have a good workflow in place.
Most of the features you're discussing encourage that workflow/bad process/bad developer behavior, because they were built entirely to support Linux's development methodology in which that behavior isn't not considered a hindrance.
However, you clearly have your mind made up, so, let us leave it at that.
* Old workflow: "commit code" and "share code with team" are jumbled together.
* New workflow: "commit code" and "share code with team" are separate actions.
This looks so obvious, but coming from years of using SVN, it took me a while to understand the implications. With git I could fix (with commit --amend) those embarrassing "whoops, forgot to add an untracked file" kinds of mistakes. Over time, I found that my local commits became smaller and more frequent because I can reorder and squash them before pushing (or git-svn dcommitting, as most of my clients run SVN).
Consider a larger change where you might implement it one way, decide you were on the wrong track, back off a bit and implement it some other. With SVN these false paths are either in the common repository confusing the history, or I'm in a disconnected state for an uncomfortably long time. With git, all these changes are local commits that are ultimately kept out of the main history because I've reordered the change history into something comprehensible.
It took me a few months to grok git. Reading an article on the filesystem structure of the repository helped me a great deal.
Deal with the merges and conflicts when you want to. Also, it's lot faster to commit. You shouldn't be pushing out every commit -- just commit a lot and when you want others to see it, push it.
More generally, I never liked how to cvs and subversion a branch is basically just a copy at the file level. That mixes version control metadata among the file paths that are themselves being version controlled.
Pretty freely is not freely. That's like having 'decent' freedom of speech, y'see. Git allows you to commit any time, so fast you won't even notice the pause.
> As for the faster Git commits, is that just because you are only commiting locally? You still have the overhead of pushing to your central repo.
Yes, it's faster because it only commits locally. You should not push every commit to the central repo, and you should be committing more or less every time you change a file. Git allows you to be far more granular than SVN or perforce did.
It also allows you to throw things away and try things out without worrying about hurting someone else. Branching in SVN/Perforce is simply not worth the effort, but on Git, it's trivial.
Having your own private repo other than what is on the server allows you to experiment and make a lot more mistake without affecting others.
My old Perforce workflow: code, code, code, code, code, code, code, code, commit
My new Git workflow: code, commit, code, commit, code, commit, code, commit, code, commit, code, commit, push
It's the difference between single-piece flow and large-batch production in Lean Manufacturing.
If you try to use git like svn, it's pretty frustrating. Seems like a lot of work for no gain. But if you embrace the git philosophy of commit early and often and local feature branches, you'll understand how git got so popular.
In svn, I often run into the situation where I'm working on a new feature, want to save what I've done (because it works), but it's not ready to check into the central repo. Then I usually do something that breaks the whole thing locally, but I have no way of undoing that change since nothing has been committed yet. I've seen svn users copy and paste entire directories just to "snapshot" their work. With git and local commits, you don't have this problem. You "save" your progress as you go along by committing it locally and when something breaks, you revert locally. Then when your feature is ready, you push the whole thing to the central repo.
Likewise, but mine frequently looks like this: branch, code, commit, code, commit, code, commit, rebase -i, rebase branch on master, code, commit, rebase branch on master, merge, push.
Git is a far more advanced tool compared to SVN. It will enable you to do things that SVN is not capable of, and it will save you from trouble that would have resulted in a multi-day firedrill in SVN.
For example, with Git you can maintain local history on your desktop quite easily. Since there won't ever be conflicts, committing is effortless. But now you have the ability to rollback to a previous state quite easily, far beyond the ctrl-z capabilities of your favorite IDE.
Also, in general with Git merging your changes into the central repo ought to be easier than it was previously. If there are conflicts they'll be easier to sort out, for example.
I tried to do mostly branched development in subversion but gave up on that due to all conflicts.
I think the real pain comes when there are problems. How do i unfuck what i just did?. You're getting stressed out, there are people waiting on you to get your commits in. I had my flow down with svn, but now what do i do? Compound that with 100 engineers experiencing that on some level as we transitioned, it was trying.
It's true of changing any tool or workflow, though. You rely on muscle memory or run on auto-pilot and suddenly you don't know what to expect from your day.
My only complaint about learning git is the lack of distinction between what is a keyword and what isn't. The command line syntax is chock full of words. In particular, "upstream" was the most confusing thing for me starting out. Only much later did I realise that there was nothing special about "upstream". It's just an arbitrary name for a remote repo, but you wouldn't know that as a newbie reading these tutorials. It is only by convention that upstream has a special meaning.
# commits that are on the current branch but not upstream
git log @{upstream}..
# commits that are upstream but not on the current branch
git log ..@{upstream}
# the commit path between the current branch and upstream
# (all commits are exclusive to either branch)
git log ...@{upstream}Obviously that wasn't scalable. (My inbox of commit log emails was insane.) So we transitioned to git. Plus, most of the engineers were already using git-svn. It just took someone who had a free weekend to "git svn clone" and move everyone over to that.
It was PAINFUL at first. I was one of the grumpy people, but after I got the hang of git, I am so glad we just ripped the bandaid off.
"If you can deal with your current source code system, do not go through this pain. Seriously. This was a long, painful process for us. Over the years, many tools, systems, and processes had become deeply intertwined with our subversion installation. That said, if your team is small, or your source control system isn’t tied into anything, go for it! Just do it as soon as possible – the only time better than today was yesterday."
The first half of that paragraph basically says that it very well might be more painful to switch to git than it is to keep using your existing system.