Hacker News Reader

Comment by wccrawford | Hacker News Reader

Without information about false positives, this is just basically saying they wrote an algorithm that sometimes points out cancer early. But if it is only correct 1% of the time, nobody is going to listen to it. It'd do even less than the current "You really need to check for cancer!" statements that we already have.

Edit: From the paper:

> A deep learning (DL) mammography-based model identified women at high risk for breast cancer and placed 31% of all patients with future breast cancer in the top risk decile compared with only 18% by the Tyrer-Cuzick model (version 8).

So better than before, but still only detects 31%. If I'm reading correctly, it's 95% correct? I guess that means 5% false positives? That wouldn't be bad.

View on HN

> I guess that means 5% false positives? That wouldn't be bad.

Due to Bayes' Theorem, this can result in the majority of patients with a positive result being patients without cancer. Depending on what treatment is taken, and the psychological burden of erroneously thinking you have cancer, the negatives of the additional screening could outweigh the positives.

FWIW, it's not like the doctor is going to send you home with a diagnosis based on this screening. The psychological burden would persist only until further testing and analysis. And in any case, human radiologists have FPs and FNs, too. The question is how close and in what circumstances are we approaching current human radiologist performance.

No, the problem is work up rate. If you significantly increase the rate of biopsy (typically one of the possible next steps) eventually you are going to kill people, as biopsy is not a zero risk procedure.

You have to be really careful with FP rates in screening. Of course the answer to this could just be "don't biopsy due to only this result" but at some point that renders the approach pointless, if you can't find a cost effective way of mitigating the FPR.

95% on a 1 in 1000 event means that over 1,000,000 trials

950 true positives

~50,000 false positives

False positive breast cancer is pretty bad.

Cost of false positive is retest and observe, this seems like a very low cost. Also, stop consuming sugar.

What about the anxiety and distress caused to the patient? How many might undergo an unnecessary mastectomy to be on the safe side?

I have a problem with this vision.

You can be told the risk factors for everybody, but when you are told the risk factors tailored to you it becomes dangerous? Should we tell everyone they're going to live forever so they live in ignorant bliss while healthy?

I can see being unprepared for having that information, but I don't think the solution is not having the information.

It's not a risk factor; it's a test outcome. You would be told that you've tested positive with the follow on tests and anxiety that causes.

There have been studies into exactly this: https://www.ncbi.nlm.nih.gov/pubmed/22859786

Specifically on breast cancer screening, the National Institute for Clinical Evidence in the UK has published management guidance that is quite interesting: https://cks.nice.org.uk/breast-screening

Can't read that last link outside the UK.

They wouldn't be wold they were positive for cancer. They'd be told that the computer predicts that they are at very high risk for breast cancer. That's knowledge that I think people should have, personally.

Sorry, please explain to me if my interpretation is correct:

1. The cost of a false positive is low. 1a. The cost of a false positive is retesting (what is retesting? Re-mammogramming? Isn't healthcare in USA expensive?) 1b. Observe (so future re-mammogramming and hypervigilance).

2. Additional to the cost of 1a and 1b, cease consuming sugar. Which is a non-insignificant lifestyle change.

What I'm saying is... the statements "Rerun expensive tests and maintain hypervigilance seems like a low cost, in addition to major lifestyle changes that affect every level of one's socioeconomic place (eating out with friends/family, cost of food, food availability, habit breaking, possibly culture clashes, etc)." seems pretty much equivalent to absolute nonsense.

Given a positive result. P of having the disease are 2%. Given a false result, it’s about 0. Given 2 positive results, assuming independent results (I think that’s unlikely but let’s assume) then the probability of having it is 0.04.

I’m not disagreeing that there usefulness to tests, but there is danger too.

If by whatever mechanism your FPR significantly increases biopsy rate, it's not a low cost.

It seems they are simply assigning patients to the top risk decile (more accurately than before). Telling someone they are at a high risk is different than a flat-out diagnosis. Better efficacy of the predictive model (fewer false positives and fewer false negatives than before) is a good thing.

The false positive information is in the ROC curves such as Figure 2.

TLDR but 95% correct does not necessarily equate to 5% false positives: you can also have false negatives. The bottom line is the new algorithm increases the efficacy of the prediction.

But it said it only detects future cancer in 31% of patients, which means way more than 5% false negatives. I'm assuming that means that the 95% is only on the positives.

Reply on news.ycombinator.com