The 10 commandments of good source control management
troyhunt.com
troyhunt.com
I agree with this statement, but that home is definitely not with your code. There are tools better suited for that. For instance nexus & maven.
Nexus store artifacts (which are really any piece of data: a jar, a tar.gz, a zip file). That's where you put your dependencies. Maven manage a project's build, that's where you tell what are your dependencies.
But really, there are many other tools out there to handle that (ant, nant, pypi, etc.).
The lowest-budget solution is simply to store the third party and open source projects in a reliable network file share (NFS, samba, etc). Then you can either sync locally or just build/link directly against the mounted file share.
It's also important to keep each project in a directory structure that includes the version number in the path (e.g. /nih_libs/boost/1.44.1/...) so that you can easily drop in a new version and start using as needed on a project by project basis. I'm always amazed how many places neglect this step and then have nothing but pain when they want to upgrade to a new version of a lib.
Can you elaborate a bit more on this?
I've also seen really simple projects get by with just an "install_deps" task in the Makefile, which you run first thing on a new clone. ("The simplicity of Maven meets the dependency management of Make," the wags will say.)
However, yes, checking in dependencies with your code is the way to go.
Deps that can be reliably included with just a meta-descriptor (i.e. Gemfile, pom.xml, etc.) are still exceptionally rare. Thus the natural choice is to put the rest right next to the code that it belongs to. Well, natural choice unless you're stuck in a 90s java mindset...
That said the Maven dependency model isn't too bad; fortunately you don't have to adopt Maven to get it, you can just use Ivy.
Perfect? No. But standard practice? Yes.
Since the histories of the dependencies don't get tracked, merging them can be tricky. Especially when dependencies get upgraded separately by multiple people.
I disagree. Even back in 1995 it had fatal flaws. More than once it got so confused that we had to restore the source tree from backup. There was also the continual problem of files being locked by others when they should not have been, hampering progress and complexities. I was glad when we could kick it out of the door.
I slammed all the files in the day before and handed them in all at rev 1. Cranky prof.
11. Never commit revisions which can't be built. (In the exceptional cases, always state in the commit message that the code doesn't build.)
If your commit is broken, it's not meaningful, it's just broken.
> If you wait until everything works you'll end with huge meaningless conflicting commits.
Reading comprehension, please. b0sk talked about commits which can not be built. It's a very clear and simple requirement and it definitely does not mean is not feature-complete.
> Of course it is quite annoying if you use a centralized SCM since everything you commit becomes public. Well this is actually why you should not use centralized SCM.
There really is no relation, and a DVCS will not save you when an axe-murderer with a short temper tries to bisect a bug, and you break his bisection because your commits made the whole project un-buildable.
> If your commit is broken, it's not meaningful, it's just broken.
Sometimes I make broken commits just to have something for the written record. Sometimes I rebase them away. Sometimes I ask friends to pull from a broken commit (gasp!) when I need their help to fix my bug. Often times I 'git commit --amend' to fix broken commits before I push. In any case, there's no reason to artificially hide these mistakes as long as the result works.
> There really is no relation, and a DVCS will not save you when an axe-murderer with a short temper tries to bisect a bug, and you break his bisection because your commits made the whole project un-buildable.
Git allows you to automatically skip untestable commits. See http://www.kernel.org/pub/software/scm/git/docs/git-bisect.h...
In my workflow, I usually use a 'master' branch and 'topic' branches. The master must always build, as you say. Topic branches don't -- they're experimental by definition. When a topic is ready to go, we rebase and clean up the commits. This way, we get a traceable, ever-buildable source tree from master and experimental branches when we want them.
Problem?
With DVCS you can create your own branch and push it to other systems super easily, so there's little excuse for checking in broken code to a shared branch. Also, with a lot of newer VCSes there's often a feature to "shelve" a changeset to a central DB without committing it.
Not to mention that breaking the build for more than a few minutes raises some serious red flags about the way you're working.
But there are a few cases where it's a necessary evil. For example, when upgrading a large Rails application from Rails 2.3 to 3.0, you're likely to make hundreds of small changes before everything works again.
In this case, I create a new branch, prepend "BROKEN:" to each commit message, and record the number of unit tests that are currently passing. Once all the tests are fixed, I hand-test, add new tests for any regressions that weren't caught by the automatic tests, and merge back to the main development branch.
Only if no one else ever saw that part of history, right?
:: shaky eyes ::
There's always repercussions to checking in unbuildable code. Suddenly, nobody else can contribute or pull from that branch without first fixing your bad code, for one. What's worse is when they think it's a mistake and do tweak your code. You'll have to backtrack then, adding noise to the file's history. Another is when you need to roll back. If you allow broken commits, there's always a change of hitting a broken version. In times of emergency, you really don’t want to be bogged down by unbuildable code.
That's what branches are for. That's the whole point of committing often.
A disciplined approach to branching will deal with the problem.
He's talking about versioning schema, not data -- he should have been clearer about this.
There is never a reason to use source control to version your data This will be painfully obvious to most people, but I’ve seen it done before, and more than once too. Source control management exists to version, um, source code...
[1] http://www.troyhunt.com/2010/07/rocking-your-sql-source-cont...
I've seen this before, but never found a suitable alternative. Where does it belong? Suppose multiple developers are compiling a C++ .dll which testers are grabbing through websvn and testing. To track down crashes they get, we need the associated .pdb for the right revision. Where should these files be kept? In a plain folder where each revision gets its own subfolder named as the revision number? That means updating two separate locations with each build, using two different interfaces...
The DLL is a compilation output, it's not to be in source control.
> To track down crashes they get, we need the associated .pdb for the right revision. Where should these files be kept?
To track down crashes they need the DLL to start with.
Testers should either have the ability to build the project on their machine, or they should be able to grab the output from the CI server (just go to the CI server, open the latest correct (compiled, tested, green) revision, grab files from there and test that.
Mmm... I've been working in a medium-sized group project, and we've been having pretty good luck with source controlling our driver .lib file. The intermediate object files are discarded, but it's really nice not needing to recompile the library every time somebody updates the library code.
(This is a 'project' that produces a library rather than an executable, for inclusion into other 'projects')
Yes. Although revision number naming would be problematic if youre using git as it uses SHA1s to identify revisions, naming them with date.branch.author.sha1 might be better.
That means updating two separate locations with each build, using two different interfaces...
It just means having your VCS do the build and create the directory in a post-commit script.
Or you could just actually get a real CI system.
In my company we setup the following workflow :
-> commit code
-> code is built and tested using Jenkins
-> generated artifacts are stored on Nexus
On top of that, daily we deploy everything on an internal pypi (we do mostly Python). Deploying the code in production is then not much more than running a script that easy_install everything from this internal pypi...
[edit] typo
That, yes.
What's the point of versioning a build? After it's built, it will never change. Back it up, of course. But don't version it.
Automate your build and let it do the checking-out, compiling, testing, tagging, copying, emailing and what not, for all eternity.
My company does this, and I'm not entirely sure how I feel about this. The motivation for delegating changes is that different groups in our company have different check-in priveleges. This results in half of my changes being committed by me and half the changes being committed by someone else. This does, of course, present some coordination problems though.
I violate this one sometimes. I'm not a VCS magician, so sometimes I get commit errors, and wind up having to make a second commit. I figure identical commit messages makes it pretty obvious the two commits were intended to be one.
I spent what I thought to be a generous amount of time trying to wrap my head around Ivy and I just couldn't make it work the way I thought it should work. I use Maven, and while I'm not a Maven...er...maven, I can work with it well enough and I'm certainly adept with its dependency management mechanisms. Ivy just seems to have way too many moving pieces and their documentation always seems to be missing key pieces of information that link theory to example. I'm left with an incomplete understanding of how Ivy works and, as a result, I can't use it effectively.