Hacker News Reader

Comment by darawk | Hacker News Reader

> If you get tested again, you can reduce your uncertainty

I've always been bothered by statements like this about medical tests. This assumes that false positives are statistically independent. But isn't it more likely in general that false positives would be highly correlated in individuals, test administrators, or labs? E.g. If the same person takes the same test from the same doctor and sends it to the same lab, it seems extremely unlikely that the results will be independent. And to me at least, it seems highly likely that a false positive will correlate with some aspect of the individual's biological (e.g. some similar substance in the blood to what's being tested), and as such even using a different doctor/lab would not be all that likely to ameliorate this issue.

View on HN

That's a nitpick on a correct statement. Unless two tests are always perfectly correlated, you will reduce your uncertainty. They don't need to be independent.

Sorry, I should have provided a more complete quote:

> If you get tested again, you can reduce your uncertainty enormously, because your probability of having cancer, P(B), is now 50 percent rather than one percent. If your second test also comes up positive, Bayes’ theorem tells you that your probability of having cancer is now 99 percent, or .99.

This statement is incorrect if the results are correlated at all.

Exactly, unless the false positive is causal some how, testing again will reduce the uncertainty some amount. Though that amount may be less than if the false positive isn't completely independent.

I think this is a good question - I hope someone who actually knows about this stuff chimes in with some answers.

In the mean time as a thought experiment, going with a blood test example:

* levels of the compound being tested may rise or fall naturally, and the test checks for a certain concentration or above. * the patient may have the condition, but not have had it long enough for markers to have risen to "trigger" levels. * The patient may not exactly follow pre-draw instructions on eating etc, skewing results * in the case of false negative: the person's immune response may be temporarily suppressing the marker * in the case of something like cell counts - this sample could just be a random local variation * The tech or doctor or whoever could randomly make a mistake on one sample, but not on each sample.

And so on. My devil's advocate point here is that the patient, doctor, lab, and so on are not deterministic code - there are a lot of random inputs in the entire process chain.

I think the idea is that, if you retake the test, your system will be in a different state (our organism's biochemistry fluctuates naturally). So if something in your system caused a false positive, there's a chance it won't be there when you retake the test and you will get a true negative.

Basically, you're thinking of your blood composition as stateless, when it may also be counted as an "external factor".

Ya that is definitely the idea. The question is is that idea correct. It seems almost certain to be the case that sometimes it's correct and sometimes it's not, but i'm not aware of any research into which cases are which, and I think that is a huge problem.

These are questions that themselves would need to be answered with scientific research. In the absence of real empirical data, I'd be uncomfortable with saying it is "more likely in general that false positives would be highly correlated." Without data, we don't know how likely it is.

Which is why the author of the article needs to mention that the second test is independent of the first.

You could also look at it the other way: Using the same doctor and lab and procedure is the best way to eliminate a false positive, because if the cause was external, then the cause may not be repeated. But if you went to a new lab/doctor/whatever, you've now introduced new variables that could cause a false positive on top of whatever already caused it.

Given that it could go either way, it makes sense to think of each one as independent.

Now if you really wanted to take advantage of Bayes, if you got a positive test then you should get two more tests, one with the same lab and one with a totally independent lab (or even if you got a negative test, assuming your first Bayes run gives a 50% confidence)

Down mods for this?

More information allows for better reasoning. Repeating the same test and also doing a "more independent" test are the minimum next two things you should do given a positive result on a single test.

Reply on news.ycombinator.com