Basically the only sane answer is Bazel (or similar). We currently use path based test running because I couldn't convince people it was wrong, and it regularly results in `master` being broken.

I don't really have a great answer for when that stops scaling - especially for low level things e.g. when Android changes bionic do they run every test because everything depends on it? Technically they should.

The only other cool technique I know of is coverage-based test ranking, but as far as I know it's only used in silicon verification. Basically if you have more tests than you can run, only run the top N based on coverage metrics.

There's always the escape hatch of marking a derivation as impure.

However, most tests should really have no reason to be impure by design. And tests that do depend on side effects might as well be re-evaluated every time.