I'm a big fan of fast, self-contained hermetic tests, but at the end of the day you should be testing the actual behavior of your software. This means all your dependencies should be versioned so you can bisect actual behavior, not mocked test behavior.

I've also rarely ever touched software that is actually free of nondeterminism, so I am deeply skeptical of caching anything but the simplest test case. And those are fast.