Hacker News Reader

Comment by mlyle | Hacker News Reader

You have to be careful with screening tests.

Say that this test has a false positive 1 in 1000 times. If you test 100,000 people, you'll get 100 positives that need invasive further testing and followup, and 5 real pancreatic cancer cases.

Society will pay for 100,000 tests, and 105 cases of followup. You may cause lasting harm to some of those 105 people. And then it's not clear if you can improve the survival of the 5 pancreatic cancer cases much. They'll live longer after diagnosis (because you diagnosed earlier) but not necessarily longer overall.

(One other screening effect: You'll find more "real cancer" that is so slow growing that it may have always remained subclinical before the more sensitive testing; And the most serious cancers, you won't find so much sooner, because they grow so much in the interval between tests.)

View on HN

You would need to take into account how aggressive a given cancer is and our ability to treat it.

For instance, prostate cancer blood screening often led to radical treatments that are no longer thought to be worth it for most people.

> most prostate cancer grows so slowly, if it grows at all, that other illnesses are likely to prove lethal first

https://www.nytimes.com/2023/05/08/health/prostate-cancer-sc...

In the case of pancreatic cancer, it is much more aggressive and you need to catch it early.

> In the case of pancreatic cancer, it is much more aggressive and you need to catch it early.

It's not clear that the cancers that you would find early with a more sensitive test are those more aggressive cancers.

The pancreatic cancers we find with our current detection (generally after becoming symptomatic) are typically quite aggressive. But are they all the cancers? Likewise, if the cancer is aggressive, it can grow quite a bit between screening intervals and not be found all that early.

(Part of why we think that "finding cancer early" is such a benefit is that because the smaller/earlier cancers we find are less aggressive than the cancers that we first find when they're huge and spread. There is definitely an effect from earlier detection but our estimate of it has been confused by this effect.)

As we've increased cancer screening, we've found that survival rates have gone up, as have survival times after detection... but unfortunately we've often also found that the screening doesn't always reduce the number of people dying of that cancer at a certain age. Instead, you find more cancers, and you find them earlier so more people live to 5 years, even if you've changed nothing. Cancer treatment has gotten better, but most of the benefits we have expected from better cancer screening have not materialized.

Finding pancreatic cancer early sounds good. And it may be able to reduce mortality from pancreatic cancer, but it's not a sure thing.

My grandfather (a doctor) always used to say this. There’s also an aggressive fast growing kind of prostate cancer, but treatment basically does nothing for survival rates (or at least that was the case decades ago when he was practicing.)

So his advice was, don’t look, don’t treat. Either you have the slow one and treatment is harmful, or you have the fast one and you’re going to die soon anyway.

Your grandfather's take has become increasingly accepted for prostate cancer. There is more of a watch and see attitude to make sure that the patient doesn't have a rare case of aggressive growth.

As you mentioned, the outcomes aren't significantly different, regardless of how you treat it.

From the article linked above:

> Researchers followed more than 1,600 men with localized prostate cancer who, from 1999 to 2009, received what they called active monitoring, a prostatectomy or radiation with hormone therapy.

Over an exceptionally long follow-up averaging 15 years, fewer than 3 percent of the men, whose average age at diagnosis was 62, had died of prostate cancer. The differences between the three treatment groups were not statistically significant.

The irony is he died of prostate cancer. He ignored his own advice and treated it. It did not change the outcome or buy him much time, if any.

You can just run test multiple times to eradicate this possibility or you can confirm it with another method.

> You can just run test multiple times to eradicate this possibility

The measurements are not independent and the quality of the measurement is not improved by this.

> you can confirm it with another method.

Yes. And usually the other method is invasive and expensive and bears some risk.

And then you get results like the blood test saying "very likely cancer" and the biopsy saying "uh, probably not?" that you need to decide what to do with.

I'm confused. If this blood test gave a false positive that wasn't due to an anomaly in the blood itself, then why can't we assume that the likelihood of getting a false positive twice is lower than getting it once?

> I'm confused. If this blood test gave a false positive that wasn't due to an anomaly in the blood itself,

That's the errant part. A small amount of the false positives will be because of lab issues. The rest will be because this patient is different in some way, but not all of them are cancer. Medicine doesn't have very many perfectly specific tests.

So you have a patient who has some weird enzymes around because they're genetically different, for example, and they always pop positive on this particular test. Or has an unusual diet that causes some other non-tested-for-enzyme level to be high enough to set off this sensor. Or whatever.

In this case, the specificity is 98%, so this false positive rate is about 2%.

There was a seminar given to the breast cancer society by an epidemiologist years who who presented them with a scenario:

Prevalence of breast cancer: 1%

Sensitivity (percent of people who have the disease test positive): 90%

Specificity (percent of people without the disease test negative): 91%

And asked, "How many people who test positive have the disease?" (i.e., positive predictive value)

It's only 1/11. I think only 20% got it right (in a 4 answer multiple choice question)

Yah, I'm well familiar with the base rate fallacy and I still catch myself screwing it up. It's so unintuitive. I use mental math a lot to counter my intuition.

If sensitivity is high and the base rate is low, you can approximate it with .01 / (1-.91).

Or, mental math assuming 10,000 people is not unreasonable for your case (100 with disease, 90 true positives; 9900 * 9/100 false positives.. divide everything by 9 to make it easier, 10 true positives and 99 false positives, or 10/109 or say "9%".

For people surprised by this argument, the phenomenon he's invoking here has a name: the Bayesian Base Rate Fallacy.

Can't you just run the test again instead of doing a full follow up? 1/1000 * 1/1000 = 1/1,000,000

You only get that probability if the test results are completely uncorrelated, chances are, they're not.

I'd assume the chances of getting a second false positive if you already got one are much higher.

You are assuming that those false positive rates are fixed, but they aren't. The "positive" criteria are done by an analysis exactly as sophisticated as a human scanning a list of numbers. The process is a joke and it needs to be improved by more data and better analysis, not this nonsensical "don't test people because they might be positive" argument.

No, I'm assuming there's a tradeoff between sensitivity (spotting cancers) and specificity (having your positive results actually be cancer).

ANOVA to pick variables and then reasonably selecting thresholds is a fine process that avoids overfit.

The big problem is, biology is messy and measuring lots of people to find correct thresholds is really expensive and time consuming. It's not really a technological problem, though technology has helped a little.

> You may cause lasting harm to some of those 105 people.

Could you elaborate on this?

For example you might do surgery on people who wouldn’t benefit.

you will do surgery on some of the 105 people. Some of them might die from complications, infections, etc or at least have lasting damage. Since several of the 5 people will not be any better off with treatment it's entirely possible that the screening produces palpably worse outcome.

The earlier you screen, the worse this is, since the ratio of false positives vs true positives gets higher and higher, for example 1000 vs 5 or 10000 vs 5.

It's also psychologically harmful to have the positive test hanging over your head. I'm nearing the age where doctors start harassing about colonoscopies. You can do an at-home test instead of the full procedure, and it has a very good chance of ruling out the need for a colonoscopy. But it also has a high false positive rate; there's a decent chance that you'll end up in a state of "need a colonoscopy, also a colon cancer screener flagged you". I'm dreading the colonoscopy prep, but I'm not doing the at-home thing.

I've had colonoscopies twice. Was just half a day of inconvenience if you schedule it for the morning.

The scope, right? Not the at-home test? The only thing about the scope that bugs me is the prep.

I've been scoped twice and the prep sucks. It's shitting your ass out for half a day. On the positive side, getting sedated isn't half bad and you get the day off work.

I'm another one who's had two colonoscopies. Try to schedule it for the morning so that most of the fasting time will be while you're asleep. The prep is not a lot of fun, but it's only about half a day and it's very much not as bad as gastroenteritis or anything else that'll give you a good bout of diarrhoea.

And if you aren’t a fan of being sedated (I’ve never been, and I’d prefer to keep it that way), you can opt to do it without sedation. Had a colonoscopy and an upper endoscopy the same day, no sedation. A little uncomfy, but not painful. If you can push yourself through hard weight training sets, or run a 50k, or anything else that entails a bit of discomfort, a colonoscopy is no biggy.

How come nobody seems uses this kind of math when it came to COVID prophylactics? Or did they?

Then let's take those things into account when calculating what tests to do. Surely, though, we can do better as a society than solving this with "no preemptive testing except for extreme risks".

> Surely, though, we can do better as a society

We haven't even solved the most basic shit like shelter, food, education, &c for millions of people in the west, as a society we're faaaaaaaaar from universal yearly full health checkups. As an individual feel free to get private checks, they'll gladly take your money

The fact that there are huge costs in the USA to even periodic medical checkups has severely impacted longevity in the USA to the point it ranks close to Cuba in longevity. Those with a health plan are close to the highest ranked nations. The poor without a plan at all are around ~4-5 less long lived. There is a nice rabbit hole in this data. https://www.google.com/search?q=longevity+charts&rlz=1C1CHZN...

This has a huge GDP cost in the USA, that needs to be addressed. The causes are big pharma/hospo/AMA/insuro/lobbyo.... One wonders why the AMA is there? - they limit the numbers of doctors trained in Universities/training hospitals to forestall price competition among doctors by various means. Dentists do the same.

Cost of a yearly checkup should be "taken care of", because Obamacare mandated free annual checkups, as long as you don't accidentally trigger any other billing codes while you're there. But, regardless of cost, there's a shortage of providers, so it's hard to schedule the checkup. And there's still a lot of uninsured people out there.

There is no proven health benefit to periodic medical checkups for healthy adults. At the population level it's a waste of resources. But certain preventive screening services are covered at no cost to the patient because they've been shown to be effective through high quality research studies.

https://www.healthcare.gov/coverage/preventive-care-benefits...

fixed annual, I agree, but symptom/test based assessments are useful

The US Preventive Services Task Force (USPSTF) is the body doing that meta-analysis and writing recommendations. The recommendations are for general patients (high-risk patients should be identified and guided by their doctors), and are based on how much the screening/prevention will extend or improve patients' lives. The USPSTF explicitly does not consider monetary cost.

https://www.uspreventiveservicestaskforce.org/uspstf/recomme...

There's a ton of research and regulatory oversight in this area, and the choices made generally make sense. You can safely assume that the testing recommendations are 3-5 years behind the research, though.

We do, it's not as if we aren't doing any testing. I've been getting a yearly prostatic antigen test for several years now.

The recommendations tend to take these into account, and then you and your doctor adjust.

Sometimes politics gets into it, like with the recent changes to breast cancer recommendations, but, overall, it works well for many people.

> Society will pay for 100,000 tests

For better or worse, under the American healthcare system, the patient pays for those tests, sometimes covered by insurance. If the tests are paid for out of pocket by the patient, is there still such issue?

I think the issue is exactly the same no matter who pays.

To reframe it from the individual patient's perspective, when you take a test simply for the sake of screening, there is the chance you'll learn something true that helps you, and the chance you will learn something false that hurts you

The economic argument doesn't change whether it's a private cost or purely a social cost (private costs are included in social costs, since private expenditures are part of society's expenditures).

Reply on news.ycombinator.com