The false positive rate reported is ~5% in a sample with an equal number of people that did and did not develop cancer, while the ratio in reality is 1/122 (i.e., 1000 people developed cancer, 122000 did not).
Could still be a useful development, but it does make me wonder about the cost/benefits at this stage.
Since this test is designed to identify cancer very early, a likely solution to the low PPV problem would be to re-test the individual, perhaps even multiple times. The likelihood of multiple consecutive FP test results, barring a biological reason for the FP, would at some point warrant a more accurate test, be it a screen or diagnostic test depending on tumor type. Also keep in mind that the PPV of the test would increase with a person's age - an 80 year old is at a greater risk of developing cancer than is a 30 year old. This test would also be useful for people with hereditary cancer syndromes.
The cost benefit would be huge - imagine if everyone could identify their cancer 6 months or even years earlier. Mortality would greatly decrease, as would spending on treatments, etc.
It’s tricky to find that balance.
Screening tests like these will represent a paradigm shift in cancer surveillance and as such will require those administering the test, most likely a pcp, to properly educate each patient about what the test results mean and what steps should be taken next.
"You've been flagged for further testing."
"Why?"
"Um..."
"Didn't I just take a cancer screen? What was the result?"
"Ah..."
"Level with me, doc. Do I have cancer?"
"Well, [brings out probability textbook] you see ..."
As a society, we're just getting too used to accommodating people's ignorance about statistics and science. Where does it end?
Do we have to really have to give up promising therapeutics and diagnostic tests because we're afraid people's feelings will be hurt?
It shouldn't even be that hard to state. "X% of people your age [+ whatever other factors they're using for a reference class] have [type of] cancer. Based on that and your test results, there's a Y% chance you have that type of cancer." Forgive me if I'm too disconnected from the average person, but is that too difficult to understand?
The problem is that "tested again" has its own complications.
This is not theoretical. Part of the reason for updating the breast cancer screening guidelines is the fact that biopsies to confirm were causing more problems than the "cancers" they were finding.
Medical probabilities are tricky things.
In one case the patient’s attitude about the test is not the determining consideration. A perfectly stoic and rational patient would be just as vulnerable to the side effects as anyone else.
A biopsy is the next step to confirm "cancerous/non-cancerous" after a mammogram screen gives a "positive" result.
The problems with the screening is that once a "positive" comes up on the screen most patients cannot stop themselves from going to the next stage even if the probabilities don't warrant it.
The only medical "solution" was to decrease the frequency of screenings and raise the age at which first screening should occur.
Does it not work to say "If we take a biopsy, assuming normal treatment plans and outcomes for someone in your reference class, there's a 0.1% chance the biopsy kills you, and a 0.05% chance it saves you from cancer compared to the plan I'm recommending", or whatever the truth is?
(If the next possible step is a biopsy, under what circumstances does the calculus imply one should do the biopsy? When the screen results look sufficiently bad? Then maybe they shouldn't be presented as a binary "positive"/"negative".)
No. It does not.
A diagnosis of "you might have cancer but the probability is low and we should just watch it" is not something that most people can psychologically take.
For more than half of the population, that kind of understanding is simply untenable.
And, even for people I knew who could understand it, several of them had so much anxiety that it drove their blood pressure up and it was better to just go get the biopsy so they didn't wind up with a stroke or heart attack.
In addition, "mortality probabilities" are a lot harder to psychologically absorb when you are 50 than when you are 20.
By the time you are 50, you've probably had more than a few of your friends and family in a coffin from a "low probability event".
Emotions are difficult. At some point, I will get diagnosed with a positive prostate screen (practically all men who live long enough do) that will almost certainly be "We should keep an eye on it." I hope I can weather it with the aplomb of my engineering background. We shall see.
Doing some reading about prostate cancer specifically, it seems like it actually is of the form "there is no appropriate action to take, regardless of the outcome of the screen"—or at least it's close enough that some experts think so. "In 2012 the United States Preventive Services Task Force (USPSTF) recommended against prostate cancer screening using PSA.[39] As of 2018 a draft for new recommendations suggests that screening be individualized for those between the ages of 55 to 69. It notes a small potential decrease in the risk of dying from prostate cancer, but harm from overtreatment." And: "A study in Europe resulted in only a small decline in death rates and concluded that 48 men would need to be treated to save one life. But of the 47 men who were treated, most would be unable to ever again function sexually and would require more frequent trips to the bathroom." https://en.wikipedia.org/wiki/Prostate_cancer_screening#Guid...
“The result was ‘probably not,’ but the other option is ‘almost certainly not’ so we’d like to do more tests to make sure.”
(If the patient presses for more information, you don’t need to pull out the textbook yet; a simple analogy to explain why a cancer screen needs more tests might be by analogy to a rock screen, which sometimes catches clumps of dirt that need to be prodded a bit more to determine that they’re not actually stone.)
The important bit is to present it as routine and not yet a cause for concern, to avoid causing excessive alarm.
I think that's an unwarranted assumption. Sure, another source of false positives is contamination or user error even, but other than that most FP are indeed due to a (usually unknown) biological reason. I'm reminded of a friend that was just positive she was not pregnant. She tested herself three times with three different pee stick brands. She was wrong. They all use the same chemistry, and for some reason something in her body interferes with those tests.
https://en.m.wikipedia.org/wiki/Positive_and_negative_predic...
P(Having cancer given positive) = P(Positive given having cancer) * P(Having Cancer) / P(Having a Positive in general)
Lets say 1% true positive rate is a "useless test".
0.01 = 0.88 (Detection rate from paper) * X / 0.05 (inverse of specificity)
X = 0.0005
Incident rate of Stomach Cancer (most common) is 27,600 per year in US. The denominator for this fraction is iffy because there are a lot of biases on current detection rates. If we go ahead with "adults in the US" then the rate of cancer is 0.00014 [cite](https://www.cancer.org/cancer/stomach-cancer/about/key-stati...)
Basically, if we applied this test to all adults without other diagnostic factors. This would be worse than 1% true positive detection rate. I don't know about how effective intersecting with other diagnostic criteria would be on increasing the odds a patient actually have the disease and it might or might not still be useful after that.
This sort of study is mostly bullshit anyway. They took a relatively small pile of data, made a small classifier with high accuracy (maybe overtrained, maybe not) and really can't validate it.
...and the test does not determine location, so it's impossible to treat even with a positive test.
So from a clinical standpoint it's nearly useless, except to schedule patients for more frequent traditional testing. ...the counter point is that the stress caused to a patient is probably not trivial, even from a physical health perspective.
...but from a scientific standpoint, it's awesome and well worth further study.
That sounds very useful to me!
Being able to catch a cancer early on because of increased surveillance is hugely valuable. Part of why cancer is so dangerous is because it often grows/spreads for years before you get any symptoms. If you're in the unlucky cohort of <50 years old and have an asymptomatic cancer, it might go decades before anyone thinks to check.
And mind, increased surveillance typically means once a year tests or scans, so it's not so overly onerous. It's possible even that generally being in frequent contact with doctors and more seriously minding your health could have net benefits in preventing other maladies as well.
But, the even more important metric to measure is mortality. Detecting a cancer that is 100% fatal 1 day earlier than traditional tests obviously wouldn't be that impressive. Even if the PPV is very high.
It's also very possible that a cancer that is detected accurately 6 months earlier, but that detection still doesn't improve our ability to treat the cancer. In these cases, it might even look like mortality is improved if you measure time from detection to death ("early detection extended survival by 6 months"). But, actually, it didn't change much.
So, that's all to say that the "Experimental" in the title should temper expectations considerably, because that has real meaning.
Unfortunately I can't edit my comment anymore.
If we had a test that detected everyone and they believed it, we'd improve cancer rates.
The key however here is the advanced prognosis that may allow totally different therapeutic approaches, extend the understanding risk factors etc. and the fact that you do not need to have a specialist on side, taking a blood sample can in principle be done at home and the rest is automatic.
Edit: Oh, and 5% false positives is not that bad actually, I do not have numbers is my head but would expect other essays like breast cancer screening to be also in that range.
Even if the first option is "get more regular checkups over the next year+, to characterize the growth rate."
You don't have to nuke an organ from orbit, if you have historical data that indicates a tumor isn't aggressive. Or you would, if you've been watching it and it suddenly takes off.
But that doesn't make GitHub a bad idea.
Using an SCM tool isn’t going to cause additional problems except for reminding yourself of all the dumb mistakes you made last week.
You haven't worked at a shop that mandated Team Foundation Server, have you? ;)
Reminds me a bit of the PSA test for prostate cancer. More of an early warning, but with the weak predictability, doctors are moving away from routine PSA tests.
If you only get endoscopies after positive results, you get an endoscopy every other year. If you skip the test and go straight to the endoscopy, you get one every year. Thus you get half as many tubes shoved down your throat if you get the blood test.
For a more reasonable false positive rate like 5%, you would have a 50/50 chance of getting a false positive any given decade. I'd much rather get 4 endoscopies than 40.
Indeed, for any false positive rate less than 100% (which isn't really a test to begin with) still on average decreases the number of endoscopies you need.
Really the metric that's important is the false negatives: if that's too high then I can't safely forego the endoscopy just because this year's blood test was negative. Even then though, if the blood test has a long lead time then maybe it's safe to wait and see how next year's blood test goes.
All that said, I'd rather risk dying while propofol'ed, than from say colon cancer. I mean all this is about how we die, not if we have to die.
By the way, they were both invented around the same time (CT is only older by like ~5 years).
Correct me if I'm wrong, though?
Cells can repair (maybe faulty) a certain amount of damage, but may suicide when too much is broken (double-strand breaks). The amount and type of ROS generated by ionizing radiation also depends on your antioxidant state and how well tissue is saturated with oxygen (more ROS if you exercised before exposure). Generated ROS are a significant factor in cell damage, it's not just direct DNA hits. Some ROS can last for weeks and travel across cells to fuck things up.
I think we can confidently say, lowish radiation exposure is: not great, not terrible ;)
Also (and this is according to the XKCD chart) you'd need 50 CT head scans before you'd hit the clear statistical cancer risk.
But it's not like the whole population gets a CT scan every year, or even during their entire life. If you need one anyways, there's probably a more pressing issue that 1/50th the clearly statistical risk level.
They also differ in diagnostic value for different types of use cases. But I am lacking any expertise there.
Either way, CT plus constrast agent is the standard in cancer diagnostics/staging.
Because MRIs take longer, they are a more scarce resource. If you are in the hospital and need a scan, presumably its because the doctors are trying to diagnose something somewhat urgently. If not, they should discharge you and let you get the scan scheduled as an outpatient. So, CT it is for almost all common conditions in the hospital as first line. (even stroke usually gets CT head first to rule out a bleed, where the sensitivity is still pretty good)
Even if we had 100% precise AI assisted image analysis, once a year fullbody MRI... the MRI resolution would still not catch the "before things escalate quickly" pre stage of cancer. With imagery catching _most cancers early is inherently a thing of mostly luck.
The point is that treating a cancer patient easily costs tens or of hundreds of thousands plus dire consequences for the patient themselves. So there is a lot of room for improvement if the diagnosis is made that much earlier.
That could help direct research into the development of cancers at its earliest stages, and potentially pave the way for more effective prevention.
My understanding is that there is some dependency on the type of cancer, but mostly it depends on your immune system response. Doctors are pretty clear that just having a cold can affect the numbers.
I would take positive steps waiting for a retest, such as cutting out refined carbs, saunas or other heat therapies, and eliminating stress and other harmful situations.
false positive? probably still a better lifestyle will have it's own benefits.
(IMHO)
If it is just some variance in the test, then multiple testings could help (And the test could probably be improved to reduce false positives pretty easily)
But if there is something intrinsic to those tested (say, some hormone level or other) Repeat tests will give repeat false positives. AND it may be harder to eliminate false positives by refining the test