Every time mistakes in cancer tests come up, I have to bring up this textbook counter-intuitive probabilistic result that happens when one tries to detect an a priori very rare condition.

It follows from a straightforward application of Bayes formula:

- Suppose a person has 1% chance of getting cancer, so p(cancer) = 0.01

- Suppose that the test has 99% sensitivity and specificity, i.e., p(positive | cancer) = p(negative | no cancer) = 0.99 (no test is perfect in the real world, in this case the test is wrong only one time every 100).

- To use Bayes we first need the probability of a positive result regardless of disease status, i.e. p(positive) = p(positive | cancer) * p(cancer) + p(positive | no cancer) * p(no cancer) = 0.99 * 0.01 + 0.01 * 0.99 = 0.0198

- Then by Bayes we have p(cancer | positive) = p(positive | cancer) * p(cancer) / p(positive) = 0.99 * 0.01 / 0.0198 = 0.5

And things get worse the rarer the condition is. For example, when p(cancer) = 0.001 then p(cancer | positive) = 0.09, while for p(cancer) = 0.5 the computations above give p(cancer | positive) = 0.99.

In other words, the rarer the condition, the more precise tests have to be to have confidence in its results.