There are two problems with automated testing. 1) tests take too long to run 2) difficult to root cause breakages.
Most devs solve this with making unit tests ever more granular with heavy use of mocks/fakes. This “solves” both problems in a narrow sense: the tests run faster and are obvious to root cause breakages.
But you didn’t actually solve the problem. Since the entire point of writing tests in the first place was to answer the question: “does my system work”? Granular and mocked unit tests don’t help much.
However, going back to the original question, we can actually reframe the problems as: 1) a work scheduling problem and 2) a signal processing problem.
Those are pretty well understood problems with good solutions. It’s just that this is a somewhat novel way of thinking of tests, so it hasn’t really been integrated into the open source tool chain.
You could imagine integration tests automatically be correlated to a micro service release. Some CI automation constantly running expensive tests over a range of commits and automatically bisecting on failure. Etc.
Put another way, automated tests don’t go far enough. We need yet another higher layer of abstraction. Computers are better at deciding what tests to run and when, and are also better at interpreting the results.