Deep learning outperformed dermatologists in melanoma image classification task
ejcancer.com
ejcancer.com
* They give the network is huge advantage: they teach it that it should say "no" 80% of the time. The training data is unbalanced (80% no vs 20% yes) as is the test data. Of course it does well! I don't care what they do at training time, but the test data should be balanced or they should correct for this in the analysis.
* They measure the wrong things that reward the network. Because the dataset is imbalanced you can't use an ROC curve, sensitivity, or specificity. You need to use precision and recall and make a PR curve. This is machine learning and stats 101.
* They measure the wrong thing about humans. What a doctor does is they decide how confident they are and then they refer you to a biopsy. They don't eyeball it and go "looks fine" or "it's bad". They should measure how often this leads to a referral, and they'll see totally different results. There's a long history in papers like this of defining a bad task and then saying that humans can't do it.
* They have a biased sample of doctors that is highly skewed toward people with no experience. Look at figure 1. A lot of those doctors have about as much experience to detect melanoma as you do. They just don't do this task.
* "Electronic questionnaire"s are a junk way of gathering data for this task. Doctors are busy. What tells the authors that they're going to be as careful for this task as with a real patient? Real patients also have histories, etc.
I could go on. The number of problems with this paper is just interminable (54% of their images were non-cancer because a bunch of people looked at them. If people are so wrong, why are they trusting these images? I would only trust biopsies).
This isn't coming to a doctor's office anywhere near you. It's just a publicity stunt by clueless people. Please collaborate with some ML folks before publishing work like this! There are so many of us!
Not balancing and using ROC is a terrible combo, but the metric is the problem, not the lack of artificial balance.
The imbalance is totally artificial and objectionable though. Where's the evidence that doctors see a 80/20 split in real life? If there is going to be an imbalance they should make it reflect the actual statistics of the task that the doctors perform not some artificial number. It doesn't even reflect the statistics of the dataset they started with (which is 90/10 unblanaced).
Admittedly, the correct analysis for when the data is unbalanced is more annoying and ROC curves are easier to interpret. That's why in something like ImageNet even though the training set is imbalanced, the test set is is balanced.
Comparisons against humans are also harder when the data is imbalanced in a way that reflects the training set, not the task. Humans don't know they are supposed to say "no" 80% of the time. That rewards the machine and that isn't easy to correct (you can correct what you think about the machine results with respect to a baseline, but not what biases the humans had).
Cause they definitely don’t. Even in a select subpopulation - say, people going to a derm for screening - you’d expect one melanoma per 620 persons screened (as per the SCREEN trial). Since most people have more than one mole for evaluation, and even those with melanoma will have multiple innocent moles... a mole count >50 triggers a referral for screening, though in more cautious docs, possibly as few as 25...
If you wanna be really generous and consider our hypothetical high risk group to have an average of 10 moles per person, that’s 6209:1, not 80:20.
If possible, you should write a critical response to this paper, focusing on its methodological flaws, and send it to the editors. It doesn't have to be long; critical response are usually a couple pages at most. This is likely the most effective way of removing (or at the very least, heavily qualifying) bad science from research journals.
http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comm...
I agree that a formal comment is best although not necessarily easy. A comment on PubPeer is easier but it will probably only be seen by those with the PubPeer extension.
I do machine learning in computational biology and cancer. The issues described in the parent comment are known among experts. It’s too bad so many others don’t know or care.
If the authors actually wanted to do good ML research, they could always have reached out to a decent ML researcher who could have told them all of this. There's no shortage of us. The journal could have reached out to an ML reviewer. Why wouldn't they? But no one did, because the results look good and so they send it off to press and it's good for both the authors and the journal to have something that is hype-worthy. It's just the sad reality of modern science.
Do you mean this literally or is this a metaphor to illustrate the point? If you actually mean most papers are false it'd be nice to see a link on that!
https://www.annualreviews.org/doi/abs/10.1146/annurev-statis...
Any chance we could connect over email or something?
Why should it be balanced? It should be the expected natural clinical class distribution, no? The humans have priors about this too. If anything, it should be more imbalanced, as I would guess (I would hope!) that less than 20% of scans are malignant.
A̶F̶A̶I̶K̶,̶ ̶a̶ ̶R̶O̶C̶ ̶c̶u̶r̶v̶e̶ ̶c̶a̶n̶ ̶b̶e̶ ̶m̶i̶s̶l̶e̶a̶d̶i̶n̶g̶ ̶f̶o̶r̶ ̶a̶n̶ ̶i̶m̶b̶a̶l̶a̶n̶c̶e̶d̶ ̶d̶a̶t̶a̶s̶e̶t̶,̶ ̶b̶u̶t̶ ̶t̶h̶e̶ ̶A̶U̶C̶ ̶i̶s̶ ̶s̶t̶i̶l̶l̶ ̶o̶k̶a̶y̶ ̶f̶o̶r̶ ̶s̶e̶l̶e̶c̶t̶i̶n̶g̶ ̶m̶o̶d̶e̶l̶s̶.̶ Edit: This is incorrect, a PR curve + PR AUC should be used for model selection if imbalanced. I agree it would be really misleading if they (say) just reported accuracy (since the null classifier of always guess negative would give 80% overall accuracy). I̶ ̶t̶h̶o̶u̶g̶h̶t̶ ̶t̶h̶a̶t̶ ̶t̶h̶e̶ ̶A̶U̶C̶ ̶f̶o̶r̶ ̶R̶O̶C̶ ̶c̶u̶r̶v̶e̶ ̶s̶h̶o̶u̶l̶d̶ ̶s̶t̶i̶l̶l̶ ̶b̶e̶ ̶a̶ ̶v̶a̶l̶i̶d̶ ̶m̶e̶a̶s̶u̶r̶e̶ ̶s̶i̶n̶c̶e̶ ̶i̶t̶'̶s̶ ̶s̶h̶o̶w̶i̶n̶g̶ ̶h̶o̶w̶ ̶m̶u̶c̶h̶ ̶b̶e̶t̶t̶e̶r̶ ̶t̶h̶e̶ ̶m̶o̶d̶e̶l̶ ̶p̶e̶r̶f̶o̶r̶m̶s̶ ̶t̶h̶a̶n̶ ̶r̶a̶n̶d̶o̶m̶ ̶g̶u̶e̶s̶s̶i̶n̶g̶.̶
How do you usually handle imbalanced data? I've had some success with SMOTE or weighted loss for imbalanced datasets, but I'm embarrassed to say I've been using AUC with ROC curves as the default - if this gives inferior model selection than AUC with PR curve I'll have to start doing that instead.
My understanding is the PR curve is preferable to ROC since the ROC can make it difficult to discern differences between models on imbalanced data; but the ROC is still a valid way to compare/measure models.
I wish they added this context in the limitations section. The paper only says:
"There are some limitations to this system. It remains an open question whether the design of the questionnaire had any influence on the performance of the dermatologists compared with clinical settings. Furthermore, clinical encounters with actual patients provide more information than that can be provided by images alone. Hänßle et al. showed that additional clinical data improve the sensitivity and specificity of dermatologists slightly [5]. Machine learning techniques can also include this information in their decisions. However, even with this slight improvement, the CNN would still outperform the dermatologists."
Your points hit on validity issues. Where would it fit on the errors of omission/commission scale?
EDIT: I'll add that the ratio of positives to negatives in the training set is irrelevant and in no way invalidates the study. As far as testing goes, there is always a balance you must strike in a reader study involving doctors. Ideally, you would have the exact ratio a doctor would encounter in practice, but for a screening study, that is typically impractical as you would need a huge number of cases and doctor time is expensive. A ratio of 1 positive to 4 negatives is entirely reasonable, although the doctors (particularly the less experienced ones) will almost certainly have an elevated sensitivity and reduced specificity since they will know it is an enriched set, but this is reasonable for ROC comparison purposes as it mostly just selects a different point on the doctor's personal ROC curve. Note that some studies even tell the doctors beforehand what percentage of cases are positive.
Also, please explain the problem with using ROC here. The probabilistic interpretation of ROC's AUC is the probability of correctly ranking a random mixed pair (i.e. ranking the positive example higher than a negative one). How is that metric affected by the 80/20 split of the test data? Genuinely curios here...
The train/test data being imbalanced in the same way does give the model an advantage, but I don't think that making the test set 50% would solve the issue completely either. Doctors have been "trained" on the true distribution, while which is not 50% (I'd guess that the true distribution is actually extremely unbalanced).
The model isn't simply learning to predict no 80% of the time, it is learning the distribution of the data with respect to the input features. For example, let's say that we have a simple model with only 3 binary features. It may learn that when features X_0, X_1 and X_2 are 1, the probability of cancer is 70%. This isn't a simple multiplication of the true probability by the upscaling factor though--it depends on the percent of negative samples with this feature vector and the percent of positive samples with this feature vector.
If we are to change the test set to be 50% positive and keep the same train distribution, the model no longer has the correct information about cancer rates with respect to feature distributions, but neither does the dermatologist. The specificity and sensitivity continue to not be interpretable as predicted specificity and sensitivity in the real world.
There is no issue with reporting specificity/sensitivity if they had used the true distribution of cases. Yes, the curves/AUCs will look better than the precision/recall rates, but they do not mis-represent what the doctors are interested in (what percent of people will be missed, and what percent of healthy people will be subjected to unnecessary procedures).
Anyways, the classifier doesn't actually seem to be that good, there's actually doctors that were better than the classifiers if you check the paper.
Your first two suggestions combined together are very wrong. If you made the test dataset balanced and then measured PR curve the precision would be way too optimistic as it is directly affected by the class imbalance. ROC curve on the other hand is invariant to the test set imbalance.
You can find interesting this short article I have written about this problem: https://arxiv.org/abs/1812.01388
But sensitivity and recall are the same thing...
Posting high scores on your validation set is only as informative as your val set is representative of the real world. 70% specificity, 84% sensitivity looks OK on paper (maybe -- as another poster noted, it's equally fair to say it's good evidence that image-only diagnosis is bad no matter what does it), but it doesn't always feel that way in practice. As a cheap example, your word error rate for a speech recognition system has to be extremely low in order for that system to be nice to use -- way lower than most otherwise acceptable looking scores.
This analogy only gets you so far, and i don't mean to impugn this study's test set, but another example is just because you can post 99.9% on MNIST doesn't mean that your system will approach that level of accuracy on digit recognition in the wild.
If the authors have done a diligent job here, that should be good evidence of it's accuracy. It's also encouraging to see they do multiple training runs, getting similar accuracy, and that their ROC is generally better than not just the average physician, but almost all.
"This challenge leveraged a database of dermoscopic skin images from the ISIC Data Archive1 , which at the time of this publication contains over 10,000 images collected from leading clinical centers internationally, acquired from a variety of devices used at each center. The images are screened for both privacy and quality assurance. The associated clinical metadata has been vetted by recognized melanoma experts. Broad and international participation in image contribution ensures that the dataset contains a representative clinically relevant sam- ple". [Gutman et al.]
This paper selects a subset relevant to a certain condition (beyond my expertise).
But that sounds pretty good to me. If the test set in the paper is carefully randomly selected, as the authors described (and I've no reason to disbelieve them) then performance on test set should be a good proxy to actual performance on unseen data, as the underlying dataset is designed to be representative of clinical practice.
Of course that's not the same as having a clinically useful product.
But commenters here are knocking these cancer researchers as if they are idiots, and saying very harsh things about their methods, when it seems like 3 minutes reading their Dataset section gives reasons to think their setup is actually pretty ok?
Lets say the data was collected from 5 different clinical centers. One risk is basically that when you deploy the model, it only works at those clinical centers due to idiosyncrasies specific to those centers. Or suppose certain doctors were more likely to take melanoma images, and certain doctors were more likely to take non-melanoma images, and both sets of doctors used different techniques to take images. These are just some ideas, there could be any number of confounding factors.
Basically, one should be default suspicious of most research papers working with small datasets (the test set is only 100 images - this is very small) that have not been deployed in the real world or otherwise validated independently. The root comment of this thread is basically saying this (a different comment is the one calling the researchers idiots).
There _could_ be.
But when the source dataset was carefully gathered for a competition and the academics are saying "Broad and international participation in image contribution ensures that the dataset contains a representative clinically relevant sample", talking about multiple different equipment and labs, things look pretty promising.
A lot of ML systems are built with much less rigourous datasets and do a good job when you put them in production.
10k such images were gathered from this process.
The authors then randomly select 100 images from it to use as a test dataset. 100 is a small number. But that smallness is not relevant to the _selection_ issues here. Its only relevant to whether the measurements of performance are statistically significant (i.e. that we didn't end up with a sample that by chance is particularly favorable to the ML approach.) 100 data points is enough that that's unlikely (though one should check.)
Additionally the authors talk about using a much larger validation set, and performing multiple runs and checking the validation accuracy is similar. Unless they deliberately left out the damning fact that their accuracy was a lot _higher_ on their small test set than on their validation set, then its even more unlikely that the test set is a sample that is particularly favorable to the ML approach.
You could argue it happens to be particularly _unfavorable_ to the humans, but that seems a stretch. Perhaps they should have created multiple test sets, and a couple of different batches of human raters etc. But honestly, their setup seems pretty good to me.
> (a different comment is the one calling the researchers idiots). That's fair.
These are very small evaluation sets for deep learning. My point is the work is promising but should be viewed with healthy skepticism (by default).
I would really not read anything in particular into "Broad and international participation... ...sample." That's just a claim in a paper, it's not "the truth".
Evaluation is a statistics question, and it doesn't matter that the deep learning model used is high capacity and needs a lot of training data.
There's nothing inherently wrong with validating a complex model on a small amount of data.
The paper has a section 4.2 that gives a statistical analysis. Granted, it'd be nicer if they had enough data to show statistically significant differences.
In other words, while dermatologists may have been outperformed by deep learning in image classification, it is not evident if deep learning could do the same against dermatologist diagnosing in person.
Also, not clear what the overall ratio of false negative/positives was in each case.
Also, unless I missed it in the paper, I'd be curious to learn if the cases, in Fig. 4, where majority of humans and CNN disagreed, were the same where majority of humans disagreed too or not.
That said, the results here are still impressive.
Source: I worked in clinical melanoma research for 2 years.
That is, will medical images of diseases we diagnose in the next 20 years look a lot like the ones from the past 20 years, or is there a danger of over-fitting on an evolving data set? Could either the technology or the biology of the disease evolve?
In a prior life I was a quant trader, and financial market data is notorious for having the non-stationary problem. On top of market rules and structures changing all the time, once someone discovers a profitable trading idea, their own actions change what the data looks like for everyone else from that point forward.
https://twitter.com/IAmSamFin/status/1122271463170564100
Another example of change over time:
> One difficulty in such a comparison is that Gleason grading standards have shifted over time, so that scores below six are now rarely assigned, and assigning a higher grade has become more common
Example #1: Let's say that cancer rates are increasing over time and cameras are improving over time. You might end up with a weird artifact in your model that higher resolution images are more likely to indicate cancer.
Example #2: Let's say that cancer-detecting algorithms are widely successful and so someone makes an app that lets you upload images of skin and the app tells you the probability of you having cancer. Suddenly a model that was trained on suspicious lesions is being used on normal freckles that people uploaded for fun. You end up with a lot of false positives. Maybe you try to combat that by including images uploaded to the app (that you somehow obtain labels for). But now you have a model that predicts that photos taken in brightly lit medical offices are likely to be cancer and blurry images taken in bathroom mirrors are not cancer.
You could argue that Example #2 is more about the difference between training data and data to be scored, but the fact remains that outside of tightly controlled scenarios, the way data is collected nearly always changes in time and ends up affecting model performance in unexpected ways.
However - you have hit on a very real problem. Imaging systems have got better over time, imaging quality even on the nominally same system can be different from different sites. Image coverage can change by both policy and system capabilities, etc.
It's worse the more sophisticated the imaging systems are. Consider MRI, which is perhaps better thought of as equipment to perform physics experiments than as an imaging device. In that case, nominally equivalent scans from different vendors (even different generation from the same vendor) can have significantly different characteristics. And there is a ton of processing going on, there is no such thing as "raw" data here - even the vendors themselves may no longer be able to really (or at least easily) characterize what is being done.
So yes, in any machine learning applied to these data sets, you have a very real risk of learning odd characteristics of the sample data and hurting your generalization.
Biology isn't as likely to be a problem I think, but biological response to changing treatment protocols, sure.
The fact that they fiddled with the balance of classes in the test set makes the above even worse.
> For example, Roberto Novoa, a clinical dermatologist at Stanford University in the US, has described a time when he and his colleagues designed an algorithm to recognize skin cancer – only to discover that they’d accidentally designed a ruler detector instead, because the largest tumours had been photographed with rulers next to them for scale.
Source: https://physicsworld.com/a/neural-networks-explained/
At Gigantum (https://github.com/gigantum/gigantum-client) this is literally our raison d'être to make this process as simple as possible.
Certainly it's a component, but (for example) patient history is, too.
Of course the key here is that the training set is crucial to building a high quality model - which of course needs a set of specialists to give their consensus on the diagnosis of the patient based on the images.
Presuming those folks can agree - the technology becomes a force multiplier for good. If they disagree or label things problematically - they become a force multiplier for bad.
As long as you have procedures, you will have a need for an extremely competent clinician that can synthesize all information and coordinate with use of hands or devices.
Other than that yes, robots are not capable of replacing manual work in médicine yet.
In the future your doctor will have an image scanner in their office and you'll get 'really cheap' diagnosis of this to back up the doctors opinion.
Then you'll go for biopsy etc..
It was justified (cost wise) to replace many human labourers on auto assembly since machines don't get tired, need breaks, have off days. It could certainly be argued it is even more important in the field of health care (reduce costs and improve outcomes) for all forms of image scanning.
The authors also seem to note this in their conclusion: “Our findings suggest that artificial intelligence algorithms may successfully assist dermatologists with melanoma detection in clinical practice which needs to be carefully evaluated in prospective trials.”
They use the verb assist. I think this will be the case, tools like this will be made available to board certified dermatologists to enhance their work. It won’t replace them anytime soon.
This field continues to be among the most competitive for physicians in the US. The barriers to entry are not only high, the number of residency spots get smaller each year when compared to total applicants. If interested, here’s some stats on med school rankings and derm match results: https://escholarship.org/uc/item/59p3z80r.
But over many years I wonder what will happen. Will there be paradigm shifts that make us rethink about all these specialities and subspecialties? Should we combine them or do something else?
Or even better, allow them to spend more time on the hardest cases. And allow people with no access to a dermatologist now, access to a machine almost as good?
Imaging specialties might end up becoming more "efficient" (such as radiology), but medical patient data which is used to diagnose and triage is some of the worst there is (source: I worked with it). The only positive outcome I see from ML for the medical field for the foreseeable future might be to reduce the number of misdiagnoses if all things go very well.
No, it's not.
The system is an image classifier with a HUGE false positive rate (and false negative rate). When false positive rates exceed actual incident rates in the population (or far exceed by orders of magnitude, in this case), then it's practically worthless. This is something that the medical community just does not get about statistics (edit: I'm not making a wild generalization here, there are articles out there about this issue).
What this study actually shows is that an image-only diagnosis of melanoma sucks and should never be used. It doesn't matter if it out-performs doctors, because in either case the diagnosis is garbage.
When booking an appointment with your doctor, you could take a photo and submit it. If the NN detects something abnormal, it could be useful in prioritizing your appointment or at least flagging it for reviewal by the MD.