Empirical Evidence for the Value of Version Control?

Text is rendering blurry in the thing I'm building. It used to look nice and crisp, but now its as though it's all nudged half a pixel off the grid. It looks terrible. And I don't remember having touched the text rendering stuff or the vector code recently. wtf?

So I pull up the change log for the project and it gives me a giant list of everything that has ever been done on it back to version one where it was nothing but "test.html" moving a yellow box around the screen in time to an audio track. I grab a version from a few weeks back, 50 checkins ago, and click "update to this version". Reload the project in a new tab and sure enough: beautiful crisp text. Tab over to the current version: blurry blurriness.

So it's on. Jump forward to yesterday: still bad. 2 weeks ago: good. 1 week ago: bad, and so on. Until I'm looking at one checkin that definitely contains change that broke this. As luck would have it, it was a big one, so I get to try individually updating each file in turn. And since the culprit is nowhere near touching either text rendering or vector stuff, I get to go line by line on it until in disbelief I find the single "if" statement that's been flipped to a configuration that allows opacity animations to occasionally get left at 0.9997 instead of locking them at 1.0.

And we're fixed. In ten minutes.

Now I can honestly say that I don't think I would have found the cause of this bug without version control. All the places I would have thought to look were innocent, and the most innocuous thing in the world turned out to be the culprit. With nothing but tarballs and hand-renamed folders I would have been completely screwed. Not just more time to fix this, but several orders of magnitude more time. And most likely we would have simply resigned ourselves to ship with blurry text and kept an issue in the tracker about it for years.

Now, in case you missed it, this happened to me three days ago. It's not some isolated instance I heard about once. Version Control saves you like this on a daily basis.

If you see a software company that's currently in business and shipping software, I'd say you can use that as empirical evidence of the value of version control.

BitKeeper pressrelease on the subject: http://www.bitkeeper.com/press/2004-03-17.html

Also the discussion and article at LWN as the kernel moves _away_ from BitKeeper may be interesting: https://lwn.net/Articles/130746/

If you look at fex. ChangeLogs for the Linux kernel before and after, and also, after the git change, you can see the rate of development increasing quite a lot with the assistance of proper tooling.

"Empirical" and "experimental" are not synonyms. Experimental evidence constitutes a small range within the set of all things which might count as empirical evidence - e.g. anecdotes are empirical evidence but not experimental evidence.

It is rare for something as vague as version control to undergo formal investigation via experiment. As this thread shows, there is a broad range of often incompatible activities which might be called "version control."

Typically, investigations would be via trial (and error), not experiment. The experience gained from such trials is empirical evidence - evidence from experience is all that "empirical" means.

Although the strength of experimental evidence is typically based upon statistical correlation, the strength of empirical evidence is based on various other types of judgements.

It would not be unreasonable to give the reference to Moore substantial empirical weight, if one attributed significant authority to Moore (perhaps based upon previously finding that Moore's judgement about similar matters corresponded with one's own experience). On the other hand, such an appeal to authority would never be acceptable within an experiment.

In a sense, the idea of experimenting with version control, in a formal sense is a bit absurd - at the point where the benefits or disadvantages appear obvious, the experiment would be abandoned, e.g. at the point where productivity rises sharply or the shipping date is missed.

In a sense, the question seems to miscategorize version control as something other than a tool. Tools are particular. So is version control. There are some programming tasks where it is hammer to nails. Other tasks however, use screws.

Version control, as others in this thread have noted, is not a monolithic thing. It's a cluster concept. Implementations may be hammers, screwdrivers, rivet guns, or glue.

No, I rather suspect not. What's more curious is the implied suggestion that VC has no value if it has not been empirically proven through formal scientific study.

I also suspect that you will find no papers on the effectiveness of building a house using a hammer and nails and making it up as you go versus using a nailgun and blueprints, but as man has known since he first discovered fire, better tools pretty much always mean faster and higher-quality results.

* when you make a change that introduced some bug, you can see exactly what was changed, and roll it back

* when you delete a feature because it's no longer useful, then some time passes, and you realize that feature is useful (or you can use that code for something else), you can just pull up your old code and re-add it.

* you can delete old stuff, knowing that it's not in fact gone.

* you can do more radical experiments by branching your code. While you could do this without version control by just copying your code to a new location.. with VC you could keep all of the file/code history.

I work alone writing software projects. No one has seen my code in 10 years (since I started). I use SVN. Every bit of code gets put in SVN.

If you aren't sure about version control, take my word for it: you want to use version control. It will make your life easier. Just do it, and in a short time, you'll see exactly what I mean.

When working alone on a fairly straightforward, linear project, you might actually get away using Dropbox. Try to work in a team of any substantial size and there will be trouble.

For a practicing developer, that is like finding a professional carpentry blog with the question "Can you think of any practical situation where a hammer is useful", so I fear that will taint the opinion of those whose first contact with that blog was through this link.

At least they should have answered themselves - or since that's what they are suppose to do - talk about some theory that could render source control obsolete ?

Seriously, does this need research? Just try it, and you'll see some pain go away, not the least the mess of multiple copies of project directories and zip files.

Not that there isn't anything research worthy about version control, but it is entirely possible to recognize it as a good thing without it.

Until the evidence somehow materialises I think the best approach is to accept that software engineering is mostly a craft, not a science. That way we can heed the advice of respected software artisans with a clear conscience - all of which would recommend the use of source control.

In the first style you ensure that your commit messages properly describe what bugs your checkin fixes, that it follows the formatting convention you have for writing checkin messages, that there are no extraneous whitespace changes producing unnecessary hunks and that you don't accidentally insert a big block of uncommented out code.

Loose committing is when you periodically commit the stuff you are working on. Several times per day, when you feel you've reached something good, you checkin what is in your repository and, at best, you write a comment like "fixed the bug with bla".

Of course, there are several possible shades of gray between totally loose and totally anal committing. What is the right strategy may very well vary depending on how many developers are involved in the project.

For me, I've found that loose works best for my own projects. Prettifying commits and thinking up good descriptions is annoying and can disrupt my flow. I can't be bothered to force myself to do it just "because it's the right thing" and I usually don't revisit old commits anyway. I'd definitely like to hear other developers thoughts on the issue though.

Aside from the obvious software collaboration benefits, it's been particularly valuable this week when we have had to manage multiple different releases for the gifting season (promos, merchandising etc) across all six of our sites. All of our post-Christmas sales are all waiting in branches to be deployed.

It stores the code. It stores all versions of the code. On a rudimentary level, when you break something, you still have the good version.

On a more sophisticated level, with a proper version control system and branching strategy you can -

Have multiple developers working on the same codebase and most changes automatically merged in (though obviously you need some human oversight)

Support multiple different releases of a product from the same tree.

Roll forward patches and fixes from one version to the next, again largely automatically.

...

Maybe some of these are less important for things you run on your own servers rather than stuff that gets deployed at customer sites, but to me it's difficult to imagine working without it.

I expect if it was properly studied we'd be able to identify a rough level of complexity below which using version control takes extra time and delivers not much benefit.

e.g. Linux Kernel definitely couldn't be done without version control, whereas some of the 24 hour things I've done solo at hackathons would probably have moved faster without git.

edit: but don't get me wrong, for any serious, multi-developer project I'm definitely using version control

There are a lot of common accepted practices that actually have weak scientific foundations. The waterfall method comes to mind. Asking for the basis of a practice is still useful, even if the answer is just "look around."

Ideally your software has a fitness function that it must pass or which it is compared with other iterations of your software. With each change made to the software it ideally becomes more fit, however we know that some changes cause bugs, reduce fitness, etc.

Source control allows regular human beings to revert to an earlier more fit stage and progress from there.

If it were possible to write a correct version of a program on the first try and never would the fitness function change then there would be no use in source control.

Frequently small programs will be created with out source control. Like a script to print the numbers 1 to 100, Fizzbuzz, etc. These kinds of software generally don't benefit from source control and thus it is not used. Software of simple complexity usually can be written correctly in a few iterations.

When working with multiple programmers the primary added benefit is file syncing and visibility into who made what change so it can be inquired as to the purpose/impact of the change.

In short if someone isn't using source control tell them that the prototype they showed you last week was perfect for a new client, you need what was built last week in the next 15 minutes to demo for a client, but the menu color should be blue instead of red.

OT: I've been trying to figure out a way to auto-create commits everytime a file is saved, then quickly quash those commits into a single new commit when substantial changes have been made. Anyone know of a readymade solution?

Because it's in plain sight

No, this is not a matter of "trying to find it"

If someone questions this they're already wrong.