Comment by jlg23 | Hacker News Reader

Comment by jlg23 | Hacker News Reader

Parent Full thread

> 1) Mammograms are not interpreted in a vacuum. In fact mammograms are usually the first in a long line of tests before a breast cancer or other diagnosis is ultimately made.

The paper specifically talks about mammography, it does not claim to replace a complete diagnosis.

> 2) Speaking frankly as a radiologist myself, tests like mammograms aren't even that good in terms of overall diagnosis. Thats why ultrasound, tomosynthesis and MRI are often used as supporting evidence and/or alternative exams.

From the abstract: "2) successfully extends to digital breast tomosynthesis"

> 3) There is controversy over the overall utility of mammograms, particularly in the screening context.

> It strikes me that the people that push these "radiology is ripe for disruption" [...]

The paper, which I just skimmed over, does not read hyperbolic, for that we'll have to wait for popsci journalists.

OTOH, if one leaves the 1st world context, any type of successful diagnosis automation in medicine is a blessing for areas where you simply don't have enough trained medical staff.

I wasn't just commenting on the abstract presented. I was commenting on the comments I see here, as well as comments I see related to similar papers all the time.

I also interact with AI/ML researchers all the time. Most of them are typically some combination of: 1. Poorly informed about the appropriate context and utility of medical imaging. 2. Trying as hard as they can to push AI/ML as the most important technology in medicine today. 3. Pursuing a very task-specific project which they claim is massively generalizable in some (incorrect) way.

There's one point that often comes up when I chat with my MD friends: All of them agree that more information is not strictly better while diagnosing. In fact, most support that unneeded information is actually worse because it confounds the issue.

My engineering mind just can't come up to terms with this. Why wouldn't you collect all information you possibly can? You can always ignore irrelevant data you have, but you cannot consider data that you don't have!

The closest I've been to rationalizing this: diagnosing is a stochastic process so complex (and with the search space so large) that the random noise in extra data is likely to point you towards wrong directions. Plus you can always collect more data afterwards if your initial diagnosis turns out to be wrong. This is of course very simplified, but it makes sense.

However, I just can't turn off my inner voice from screaming "more data is always better". I guess that's why I'm not an MD :)

It's pretty hard to ignore that extra information though.

You run a screening company. You take people at high risk of lung cancer -- people who smoke a lot and have smoked a lot for many years -- and you provide low dose CT scans of their lungs.

Bob comes in. You scan his lungs and you find spots.

What do you do now?

You're probably going to start providing treatment to Bob. Will this help Bob live longer? Will it improve his quality of life? It might not.

https://blogs.bmj.com/bmjebmspotlight/2019/02/15/understandi...

> What do you do now?

Hopefully get other tests done, to confirm diagnosis.

I'm with GP here. I can't understand this attitude either. Having more information should never make you more wrong. This holds for uncertain information, because uncertainty can be quantified and tracked (if you're not doing this, then you're doing voodoo, not science).

I can see two reasons why you wouldn't want to gather more information in medical context. One, many tests carry risk to patient's health and well-being, so there's no point of doing them if that risk outweighs the expected value of evidence gathered. Two, I suspect that gathering information also gathers legal obligations and risks to doctors.

> Hopefully get other tests done, to confirm diagnosis

Those other tests involve things like "needle biopsy" -- they shove a needle through your chest into your lung into the suspect tissue to get a sample. This carries risk. We can justify that risk if it saves life. But this is the problem with screening -- often it doesn't save life (of course, it depends on the type of screening).

https://www.radiologyinfo.org/en/info.cfm?pg=nlungbiop

> Having more information should never make you more wrong

But you can see how having lots of low-quality information could make someone more wrong -- these are not clear signals, because if they were it wouldn't be a problem. These are almost noise. We're taking data from a large population ("4 in 100 people with this result have this disease") and trying to apply it to the individual, and when we try to get more information we subject this person to more radiation in scans or invasive procedures or both. We increasing the risk, but not necessarily saving life.

> there's no point of doing them if that risk outweighs the expected value of evidence gathered

Yes, this is exactly the balance that doctors are making. They're looking at all cause mortality and seeing if life is saved.

> You can always ignore irrelevant data

Everything we know about human psychology says you can't.

> Why wouldn't you collect all information you possibly can?

From a decision theoretical viewpoint, you would certainly want all the information you could get. For humans running a business in today's medicolegal environment, it's a very different set of issues:

1) Collecting information costs time and money.

2) Making good decisions requires the most precious resource of all, which is doctor brain-time. There isn't enough of it to spend on information with little probability of benefit.

3) If you get sued for malpractice, the unneeded data you collected probably would not have helped the patient, but it could help the attorneys arguing that you missed something. Juries struggle to understand the cost of false positives.

Even though there are valid issues here, doctors don't always make the right tradeoff in this regard. Oftentimes, I think it is more an issue of lack of training or experience that leads a doctor to consider a test to be unneeded. In the case of mammography, if doctors spend too much time doing screening and not enough time doing diagnosis, their screening performance degrades, which I think is due to a lack of feedback on their decision making[1].

[1] https://www.ncbi.nlm.nih.gov/pubmed/21343539

MD here.

In the short term, more information might cause harm because doctors are risk averse & scared of lawsuits and err on overbiopsy/overtreat, and many of our treatments aren't as good as we think they are, and all of this makes patient anxious.

In the long term, turning the information firehose on full blast means we can work out which incidental findings are best ignored or pursued and overall more data will help us.

The problem is that it is unethical to do #2 in the short term even if it is the long term ethical thing to do.

I also interact with medical doctors all the time: Most of them are typically some combination of: 1. Poorly informed about applicability or claims thereof of CS methods. 2. Trying as hard as they can to push the image of "the human doctor always knows best". 3. Pursuing university degree and then work in a very narrowly defined field without much relevant further education/updates believing their now 50 year old knowledge is set in stone.

I completely get your attitude, I think I agree with you overall and if I was not this lazy I could comb through my bookmarks and find the studies supporting what you said.

But I was just responding to your comment in the context of the paper linked. Which, at least when skimming over it, does not read like what you (IMHO, rightfully) criticize in the broader debate.

And yes, read the first paragraph as a tongue-in-cheek response, we both know that overgeneralizations don't help any debate ;)

Reply on news.ycombinator.com