(I mostly do unit testing, but also do some testing against APIs to ensure that I'm passing them data in the format they expect. One of my external service providers had downtime yesterday, and autotest caught it about six hours before they did. I wasn't successful in waking them up, but at least I was able to push a hotfix to AR to minimize inconvenience while I waited for their API to come back.)
[Edit: I might consider, if tests were taking that long, spinning up a fleet of VPSes to munch through them for me, maybe with a continuous integration server. Unit tests should be disgustingly parallel, since they're small units of work and shouldn't depend on each other. That means if one Macbook can execute the suite in 15 minutes then you should be able to have 30 Macbook-equivalents running either in the server room or the cloud munch through the same test suite in 30 seconds, give or take a little startup time.)
I'm running a Mac Pro with 8 cores, so there is a fair bit of parallelization I can do locally too. Unfortunately, the tests all depend on the database, and while I can certainly use tools like deep-test to spin up separate DB's for each worker, I've found that doing so adds a full 60 seconds to the test run. I fear that until we eliminate the database from (most of) our tests, super-fast runs will continue to elude us.
CI and distributed tests are good things, no question, but I'm still looking for ways to make it possible to run my tests locally in TDD-fashion. I'm far from out of ideas, it's just a matter of making time to experiment.
Most of my coding is done in minute loops (add test / watch tests fail / add couple of lines / watch tests succeed). YMMV.
Your CI server should be able to accept a patchset and run it without committing it. TeamCity does this (or roll your own CI server can do this too!).
Then there's hydra: https://github.com/ngauthier/hydra
And the excellent presentation on speeding up your tests with hydra that I can't find. It was one of those showoff presentations on GitHub somewhere...
It is a step by step guide with code showing how the guy reduced his suite's running time from half an hour to 15 seconds.
A lot of it is preventing so many db objects like in the OP, but it also discusses parallelizing the tests using hydra to make use of multicore. This ties into the db access thing as well though as to do parallell tests you have to make all the cases set up and tear down their own fixtures transactionally. Not doing this is a leading cause of slow tests anyway.
He even has metrics for stuff like running the whole suite off a ramdisk, the effects of SSDs, etc etc. He worked on this at his company for weeks/months apparently.
It's too hard to explain here, just watch that video if you do ANY ruby testing!
(Also, on our moderately large codebase it noticeably impaired the performance of my whole machine because of the way it constantly scanned the entire directory tree. I know there was some talk of adding edge triggering via kqueue or fsevents, but no one had done it at that time.)
Find autotest/discovery.rb for your project to see something like:
Autotest.add_discovery { "Rspec2" }
Autotest.add_discovery { "Rails" }
# both of the frameworks have their predefined rulesets
No one prevents you from adding your own discovery rules.Also you're looking for autotest-fsevent if you're using a mac. Probably something similar exists for linux.