While I've never implemented something like this, I imagine it would be straightforward for compiled languages if you have a good linker:
1) compile each unit test as a statically linked executable
2) make sure that each unit test outputs to an individual file; e.g. ./test_a >> test_results_a.xml
3) jigger your build process such that the linker removes dead and unused code from each test executable. Now each test executable contains code that is directly used by the test.
4) ensure that your build process only touches a statically linked test if a dependency has actually changed. This comes for free most of the time, but you may still have the linker touching files that it doesn't have to. You can remove these cases by staging the test files using a binary comparitor tool like rsync. If you have a crappy linker, or there's something in your test executables which is always changing (like a build stamp) you might need to compare using more advanced tools (like binutils) or something like Google's Courgette.
5) Run your tests only when the test is newer than the results--that is, when test_a is newer than test_a_result.xml. This is straightforward to implement using most build systems, like make, or using a test runner script.
Bam--now a test only runs if some dependent code has changed. If your test also uses data, you should list it as a dependency in your build system--either manually or by detecting open() calls at runtime via a shim over libc, or via strace. Your test run takes much less time, and, best of all, developers get a lot less spam to read through. This is perhaps a much greater benefit than increased speed.
Dynamic languages are a much harder nut to crack, as are integration tests that are loosely bound to the code.