How could it be harmful? Is it just because of errors, i.e. false negative means person won't talk to doctor and won't be caught early; false positive means needless worry, maybe tarnish more general medical industry? Or something else?
Does a patient take a negative exam as "I'm good!" and forego actually seeing a doctor, or having the exam read by a radiologist?
A positive result is also a challenging situation. I've talked to patients before after diagnosing them, and as you could expect most people are shocked, scared or they want to know what they should do next.
In breast radiology specifically (at least in the US) there is a well defined reporting lexicon and classification system. From what I can tell, the system does not use the lexicon or BI-RADS classification. https://www.acr.org/-/media/ACR/Files/RADS/BI-RADS/BIRADS-Re...
Breast cancer can be very subtle, and can be hard to figure out, especially if a patient has had surgery or other benign abnormalities. https://radiopaedia.org/cases/development-of-dcis?case_id=de...
Diagnostic and screening images also undergo rigorous QA for the entire system. Does the model have some kind of QA built in?
There is a just a small disclaimer at the top that says "we will not store your data on the server. please do not worry about privacy issues". Not exactly a formal or legal binding agreement.
In fact, you can't even connect over https. http://mammo.neuralrad.com:5300/upload
These are just a few things that come to mind off the top of my head. Like I said before, I do think that what coolwulf works on and is trying to do is good, and in the long run can help doctors better characterize findings and help patients. But like anything in health care, there are a lot of edge cases and side effects that you have to think about. The stakes are also very high.
So we don’t have to worry as much about the “Does a patient take a negative exam as ‘I'm good!’”, because in the normal flow of medicine that gets this scan the doctor will check the scan for them either way. The website probably could have better explanations of this, but most likely negatives will already be double checked independently by doctors, and positives will most likely be handled correctly by bringing it to their current current attention to double check the scan or run more tests.
I wonder how the false positive/false negative rates for this tool compare to that of a trained human. At some point we may reach parity, in which case what's the harm of trusting the automated result?
I don't think that a human reader in the loop will be going away for a long time. Sure, they can be at parity, but does a combined read have better results? This is an active research question.
I think what will happen is that over time, human interpreters and AI systems will "co-evolve" in a sense, where people will pick up on where models are wrong and also learn how to use models to understand their own blind spots. These are also active research topics that are in their infancy.
EDIT: To be clear, I believe that this is a different group of authors from coolwulf. Pubmed link:
It's our job (as radiologists and engineers) to shape that in a way that benefits patients.