I wonder if they have some kind of automated test suite just to make sure everything works fine. Having worked on large codebases, it's almost impossible to make sure everything is still alright manually, after a new deploy.
You can do continuous deployment without an automated test suite. Heck, I've hacked source code on production servers more times than I like to admit, skipping the deployment part altogether :p
Tests are the reason why continuous deployment works. The path going from what the programmer has typed to running on the production server becomes so short, running the test suite non-stop increases confidence on the changes.
And then once code is live we have many system and business-level monitors in place, so we know almost immediately if anything's wrong. More info about that here:
Currently, our test suite is not that large and it takes around 6-8 minutes to complete, but with testing every minute counts (testing is generally not that fun and a slow test suite just makes it more painful).
We have a two types of tests:
- Twisted tests - this tests runs asynchronously and finish pretty fast so they are generally not that problematic - Django tests - Django tests don't run in parallel so they are pretty slow. Recently, I was playing around with the Django test runner and I have made some modification to it to run the tests in parallel. Now the Django tests finish around 50% faster.
The only problem with this solution is that it is a "hack" and it requires some modifications to the Django core (I guess I should play more with the nose parallel test runner).
We also use some other "tricks" which make tests run faster - for example, MySQL data directory on a test server is stored on a ram disk.