A tool for finding the causes of unstable Python tests

This is what happened during the work yesterday.

Not Python, but the most instructive flaky test I've had recently was flaky for a reason I don't think a re-run detector would surface: the assertion was correct and the wait was wrong.

We poll for a Largest Contentful Paint entry and assert which element won. The helper returned as soon as entryCount >= 1. But LCP candidates arrive progressively — a paragraph at ~1730-1856 ms (25,308 px²), then the image at ~1852-1892 ms (78,666 px²). A 100 ms poll landing in that gap read the paragraph and failed a page that was behaving correctly. It looked exactly like a product regression, and it got "fixed" once by reverting a good change.

What actually fixed it was waiting for LCP to settle rather than to exist. 15/15 stable since.

Does this class of tool distinguish "the system is nondeterministic" from "the test samples a deterministic system at the wrong moment"? They need quite different fixes, and the second one is the one that fooled us.