Let's write unit tests for unit tests!
The build asserted that the number of failing unit tests was 0.
It turns out that "breaking the unit test library completely such that 0 tests run instead of all thousands of them" did not break the build.
When someone noticed a week later, that was not fun to clean up.
Also, tests that once made sense might have become useless, so it seems relevant to have a way of automatically detecting obsolete "this doesn't test anything" tests even though they could have been useful once. Without relying too much on human intervention, that is.
Tests for tests exist. One such technique is called "mutation testing" [1].
I don't know of any job that actually uses them, but they make sense to me: the tests need checks too, otherwise how do you know they are adding value? It's too easy to write worthless tests, see them pass, and get a false sense of security. Techniques such as mutation testing check how sensitive your tests are to actual bugs (or program changes): do they detect them? If not, they aren't useful.
- most test frameworks use unittests to test themselves.
- tests need to have value. the dev/team/organization needs to come up with the sensible measure of value that guides poeople what to test and how to test it.