AI used for skin cancer checks at London hospital
bbc.com
bbc.com
(we've been also allowed to use a university vm with 3tb of ram and that's nice)
The best use of AI in medicine would be to automate away administrative bloat to let people get proper medical care.
These limitations are often acceptable but I think as long as it works how it does, denying someone a person looking at them in favor of a statistical stereotype should be the last thing we do.
I can see if this was in a third world country and the alternative was nothing, but in the developed world the alternative is less profit or fewer administrators. We should strongly reject outsourcing the actual medical care part of healthcare to AI as an efficiency measure.
Let me put it differently. Suppose I don't tell you it's ML. It's a machine that you don't know how it works, but I let you do all the tests you want, and turns out they're great. Would you then still be against it?
When I came back to Australia, it was checked and immediately removed as an obvious melanoma.
Perhaps the idea of Comparative Advantage also applies to healthcare between countries with natural variances to types of disease?
https://www.youtube.com/watch?v=tGgn5nwYtj0
https://www.abc.net.au/btn/classroom/sun-safety-campaign/103...
Goes to the biggest university hospital nearby he can find.
Was initially dismissed but waited it out for the infectious diseases specialist and they quickly agreed with his self-diagnosis.
They kept them in hospital for a few days so a parade of clinical students/residents could come by for the specialist to say: “this man has malaria”
Then again, my one experience with a university hospital was pretty shitty, so maybe that's it?
The Jordanian doctors just had poor priors.
OT but what an odd way to address a place. I know it's called down under but not everything there is back to front.
My neighbour who is a doctor moved to another city because that's where he managed to get a spot to train for his field.
It appears that he's learning much more there than he would back home because in this country some procedures are rare outside of his current location.
Dude got plenty of experience dealing with gunshot wounds, which probably helped him when that one North Korean defector came over the border, riddled with bullets (you may have heard this story, it was big news at the time). The especially weird coincidence was that the surgeon's mentor from the states was in Korea at the time this happened, what are the odds.
https://abcnews.go.com/International/doctor-north-korean-def...
https://www.nytimes.com/1987/08/18/world/ulster-doctors-lear...
At most you had to deal with a stitch or two but often only a bandaid. Nowadays the hydrocolloid bandages seem magic.
99% accuracy in diagnosing benign case
I assume whoever is working on this knows that, so this is mostly a criticism of the article. That said, this is a horrible use of AI.
https://skin-analytics.com/ai-pathways/derm-performance/
A few peer reviewed pubs down the bottom of the page
Even if the success rate of the human eye was in the 99.5%+ range, why not have an extra sanity check from an AI model?
You're confusing False Negatives with True Negatives. For Non-Benign (Positive) vs. Benign (Negative) classification:
* True Positive Rate (TPR): non-benign classified as non-benign.
* False Positive Rate (FPR): benign misclassified as non-benign.
* True Negative Rate (TNR): benign classified as benign.
* False Negative Rate (FNR): non-benign misclassified as benign.
> It's quite easy to correctly classify 100% of benign cases as benign.
You can engineer a 100% TNR if you just classify everything as the "benign" negative class. The FNR is going to be 100% too, but that doesn't matter -- you correctly classified 100% of benign cases as benign.
> why do people die from having lesions misdiagnosed as benign?
Because the FNR is not 0%. FNR is important. You probably want a decent TPR in there as well. And FPR can be very important too, depending on how life-changing/painful/invasive the treatment for a positive case is!
def is_benign(mole: Image) -> bool:
return True
I lost my wife to melanoma. She noticed a lesion within days of it appearing, and a doctor saw it within 48 hours and felt it was benign. My wife didn’t accept it and had a plastic surgeon remove it and biopsy, then had a margin removed by surgical oncologist, the standard of care at the time. It came back as a brain tumor 4 years later and she was gone in 6 months, even with the incredible advancements today.
So I’d hold the position strongly that anything that improves overall detection rates and access to care is incredibly important and will save lives. Weeks matter with melanoma. Today with immunotherapy Molly would be fine. But if she hadn’t advocated and gotten the original thing removed, it would have cost her 4 important years.
She went back once a year for checks for 4-5 years. It was only when she was called into see an oncologist and told an unrelated x-ray lead them to discover she had stage 4 metastatic melanoma (brain, liver, spine, femur, lungs and i’m sure i’m forgetting something) that we found out that they’d only been giving her visual checkups each year, no PET scans or anything else. The oncologist was shocked that the checks were so basic, mom didn’t know she was supposed to have anything else and she was dead in about 8 weeks.
We were told that the form of melanoma only came back like that in 1% of patients and usually simple visual checkups were enough. I have no idea how true that is.
Another lesson learned is that if at all possible, go to a national cancer center. Even if for a second opinion analysis. The level of care is different and better than what you find in community oncology or hospital practices.
- this can prioritize urgent patients for the severely overworked doctors
- medical error is a leading cause of death, this serves as a second-opinion (97% true-positive rate and 79% true-negative rate)
- it can be used as evidence by a nurse or physician advocating for a patient's treatment
It sounds like they are inverting the scenario here. The question is not "do you have skin cancer?", it's "can you safely go home without seeing a doctor?".
For this new question, we set the acceptable false positive rate to zero (we never want to send a real cancer case home), and determine the false negative rate (we accept that some benign cases will be seen by a doctor).
The reason for the interest in identifying benign cases, rather than trying to identify the positive cases, is that it improves the situation for everybody: benign cases identified by AI are sent home almost immediately, everyone else has a shorter waiting time, so benign false negatives can be assessed more quickly by the doctor and given the all clear, and more time is now available for spending with the real cancer cases.
The numbers they're citing are 7000 cases with 5% real, so 350 real cancer, 6650 benign. If we can accurately say that 6500 of those benign cases are benign without wasting the doctors' time, then we're down to only 500 people needing to see a doctor, which is a huge improvement for everyone.
It's a tool that can be used in amongst the current methods to help detect skin cancer, it shouldn't be used at the only method.
But healthcare is a rather conservative industry (for good reason) so it has taken a while to build confidence in the technology and get regulatory approval.
It seems to be that you could be doing a _much_ better job of filtering this pipeline before it gets to this point. How can so many _urgent_ cases end up being negative?
They're using AI to solve a problem that probably shouldn't exist.
Someone I know recently had a referral - it's pretty light touch, you just get a prompt appointment, and they do a minor op to remove the mole, and send it to the lab for testing. Luckily in their case, it wasn't cancer. But nothing in the process seemed weird, it was just the way of the GP escalating it because they couldn't be sure. Hypothetically, if the AI had been able to diagnose with higher certainty than the GP, all of this could have been avoided, so definitely room for improvement.
In the US, we'd just go straight to a dermatologist, who would either remove it on the spot, or for a location that's liable to scar badly, refer to a specialist surgeon. For somebody fair skinned with lots of sun damage like myself, it's an annual "ritual".
Even for private health care, you usually see a GP first (could be a private GP or NHS, in-person or video) and they then refer you to the next thing whatever that may be. ( N.B. that a NHS GP can give you a referral that you use for private treatment)
There are some things you can just straight up book an appointment for yourself without a referral from a GP, but 95% of the time you start with a GP.
No idea specifically about dermatologists, but my expectation would be that would be the sort of thing that would need a referral for. Perhaps for some "non-medical" procedures and 100% for cosmetic procedures you don't need a referral, but anything even tangentially close to The C Word would almost certainly be sending you down the normal channels.
Sample of one, but every spot (5+ over the past decade) I’ve asked my dermatologist about has resulted in a biopsy and of those several were cancer.
Not to be too cynical but... is there a financial incentive for them to do a biopsy?
Of course it might be best practice as well.
In the US, you would go straight to a dermatologist, because your health insurance plan (or high paying job) allows you to do that.
People with an HMO would begin with the GP, because that's what the plan requires.
yes, and in the entire GP practice they do not have a single device invented in the 21st century. All the tools they have are a stethoscope, oxymeter, otoscope, blood pressure monitor, basically stuff you might have at home
They can't, for example, do a rapid antigen test or a lateral flow test, ultrasound or anything else you may consider a sign of modern medicine and diagnostics. Stuff countries in Eastern Europe and East Asia have already adopted.
Their diagnostic conclusion is heavily influenced by a spreadsheet that lists statistic probability of illness for different demographics, as advised Mckinsey.
If you happen to have a serious condition but fall into a group that is 'suppose to be' healthy (young man with Pneumonia) you will be bounced and told to go home until your condition deteriorates.
They will gatekeep you from getting an X-ray. When your condition deteriorates and is incontrovertible you will be taken seriously, but now you need serious treatment instead of a quick round of antibiotics.
Popular opinion in Britain is that the NHS is great, but it's overburdened. But it's fallen behind - management practices are archaic, diagnostics is poor, and there is lack of accountability and first reaction to medical errors is to brush off the patients / victims.
Over the past few decades, the first reaction of NHS management is to cover up their mistakes.
Just read the Wiki page about Great Ormond Street Hospitalk, the hospital that treats the country's most severely ill children:
> Great Ormond Street Hospital was involved in a scandal regarding the removal of live tissue and organs from children during surgery and onward sale to pharmaceutical companies without the knowledge of parents in 2001
Or the case with bone surgeon Yaser Jabbar, at the same hospital, who caused severe harm to 22 children and the hospital fought the parents and brushed issues under the carpet. Issues like removing wrong organ and getting length of a foot wrong by 20 centimeters. He operated on 700 children before someone put a stop to it
Personal experience in the US... treatment can be done on the spot at the primary dermatologist's office (curettage usually) or referred either to a Mohs specialist or a plastic surgeon (both of whom usually treat at their office). Short of a melanoma that's progressed/metastasized, I don't think I've ever heard of a skin cancer being treated at a hospital (not saying it doesn't happen, only that that would be an exception, based on personal/family/friend experiences).
Just doing a quick google search on dermatologists in the UK mostly pointed at hospitals that offered the service, so may have something to do with that.
I'm working on deciphering the Google Mock API right now. It's just one of those fucking things you do, you know? Figuring out how to set expectations on parameters, etc. Hundreds of thousands of SDEs and students went through it before it became second nature.
I dream of playing around with some future testing language where I can build mock blocks in a Scratch-like manner. Awesome, eh?
Fuck yeah, Alan Kay.
Maybe I could work on the team that builds it.
But who am I kidding? I get to be pushed into other work while I waste time to filling out unnecessary tax paperwork.
Because to get more medical lab work done, I need to have authentic IRS transcripts.
A written 1040 isn't enough.
Otherwise I have to go to the hospital to get imaging.
It wasn't necessary. Signed fully electronic non-forgible IRS forms for health purposes could have been done with minimal citizen pushback.
With, and this is key, an educated public who understood the economics.
So I'll work on wasting my time educating loved ones on the economics of paper-less forms and all forms of monopolies while I say no to future shows in Chicago or Canada or wherever.
I suspect it has a lot more to do with these lines:
> medical photographers taking photos of suspicious moles and lesions
I.e. it might not be ready enough or validated for an average person snapping their own photo and:
> The images are then transferred to a desktop computer for greater analysis before the tool determines the result
There is more to it than the phone app and it may not be packaged in a way that is currently worthwhile to distribute to home users.
Both of these (and other things) may change with time of course.
From my anecdotal experience it tends to flag the same moles as dermatologists do, and they have actual dermatologists review images where the model has low confidence, so overall pretty happy with it.
Note that I am not affiliated with them in any way.
About SkinVision SkinVision was founded in 2012 and provides a mobile phone application, which supports individuals with the early detection of the most common forms of skin cancer (melanoma, squamous cell carcinoma, basal cell carcinoma, and precancerous actinic keratosis). SkinVision is the first CE marked skin cancer application based on extensive clinical trials, conducted in partnership with Erasmus Medical Center (EMC) and the university clinic of Ludwig Maximilian University (LMU). Research shows the app has a sensitivity of 95% and a specificity of 78%. The SkinVision app is commercially available worldwide on iOS and Android except for a few countries, such as the United States and Canada. SkinVision is based in Amsterdam, the Netherlands.
The SkinVision Service is a Medical Device and is registered with the Australian Therapeutic Goods Administration (TGA).
I recently tried to show some images to a vet. Something in my phone fucked up the amount of red in the image, making them useless (guess what, figuring out how much blood is present is pretty important for medical applications)
Probably at some point we will all have a separate medical camera with specified response and with a specified led illuminator. Apple will probably get their phone certified in some medical camera mode. Right now I don't think phone cameras can be trusted
Not all applications need accurate colour, (no idea about cancer checks) but some really do.
So yes, taking the photo at home is perfectly doable. I still went to hospital though.
https://skin-analytics.com/wp-content/uploads/2024/06/Artifi...
I still have my textbook from almost 20 years ago: "Artificial Intelligence: A modern" by Peter Norvig. Lots of topics covered as AI there that surely do not meet your definition.
Just clasify the method used as what it is to a supposedly educated crowd. There is no need to classify a marketing term. And yes, for that book it's also just marketing.
Modern day usage of AI is heavily conflated with generative AI such as LLMs and Text-to-* generation systems.
My preference, as someone who works in a very closely relate field to the article, is that science should stay away from non-specific terms like AI. I've had my director approach me seeking grant ideas on using AI. It didn't go down very well when I suggested we should try to use image recognition to automate data entry. That seemed 'old', they wanted a chat bot to do... something.