Claude thinks it's a tumor, radiologists disagree

1) We already know LLMs when given the opportunity, will often make up information.

2) I am not sure its surprising that a general purpose model is not great at reading MRI scans.

"We should stop training radiologists now. It's just completely obvious that within five years, deep learning is going to do better than radiologists"

G.Hinton, 2016

My girlfriend has had some medical issues for a few years and none of the 3-4 doctors she has seen have been able to diagnose her. That is not to say that no doctor can. Just not the ones she has seen.

I do not believe in the infallibility of medical professionals. And I have seen AI totally get it wrong every day. But, I am of the opinion that we are not very far from the point where AI will be better at diagnosing issues than an average doctor. That isn't a high bar. An average doctor isn't very good.

I think AI is almost there already for an average programmer. An average programmer isn't very good either.

On some days, Claude (and other LLMs) will hallucinate on one input and produce garbage and in the next update it will then believe something else in the next unpredictably.

Perhaps using general purpose LLMs for safety critical use cases without humans isn't a good idea. This is why we have a 'human in the loop' giving the feedback to resolve the trust issue that LLMs lack.

Maybe we should trust LLMs like Claude to pilot a plane without any human pilots on board and raise VC money on the idea on replacing all pilots since we keep having these conversations on LLMs being compared with humans and countless VCs screaming over robots replacing humans in all occupations and 'accelerating everything' which could not be more wrong.

Yeah, there was a paper about AI excellence in diagnosing radiology images a few (5-10 now?) years ago, that turned out to be largely driven by accuracy skewed bias in the input data (IIRC it was using the existence of lab-style lighting, rulers, etc as its diagnosis).

Chat-GPT and all the LLMs are doing essentially the same "predict the next work in a sentence" to sound as plausibly real as possible, which is essentially the same gimmick, only we call it hallucination due to VC funded AI firms need to pretend LLMs understand anything.

Really contextualizes those stories that read as "we don't know how the AI found the medical issue, but it did!".

Bloody hell, this sort of thing is going to get people killed. The magic box is not a person, let alone a qualified doctor. Do not ask the magic box questions about anything important.

There are much better AI approaches to the problem like: https://mediaire.ai/en/mdbrain/

To be honest Claude vision is not that great, it's probably Claude’s biggest weakness compare to GPT-4.

Claude isn't an approved medical device so why is willing to provide a diagnosis?

For some reason I read this and then think of Arnold saying, “it’s not a toomah!”