Parallel tests. Use all your processor cores to speed up your development

iafonov.github.com

35 points·igor_a··6 comments

It used to be that parallel tests bundled up the tasks in "groups" and went one group at a time. e.g. 12 tests on 4 cores ==> 4 groups with 3 tests each.

The problem was that some of the groups might have ended up with tests that take 5 minutes each (finished after 15 minutes) and other groups end up with the 20 second ones (finished after 1 minute). Did that balancing every change?

Another question: How does this work for different RSpec formatters? (specifically: junit xml output or html output). Will it still result in a single file?

And just like that, we're on the cusp of reinventing batch scheduling! :D

In all seriousness though, I think that a look at OS scheduling would provide useful inspiration for test scheduling.

Just a queue that forks processes would work fine, no need to get into OS details :D

I guess I will have to come up with a new excuse when I get caught browsing HN now.

Are there any similar tools for Python?

If you're using nosetests, there's a --processes=NUM option that comes from the multiprocessing nose test plugin, but you should make sure all your tests are safe to parallelize, which, might be very tricky if you didn't think about parallelization before hand.

Reply on news.ycombinator.com