Radiology is probably an edge case, due to needing the right inductive bias and training data that general purpose multimodal language models may lack. At least until someone creates RadiologyBench and provides an incentive to the labs to make that part of their data pipeline.

I have witnessed a non-radiology case (a STD on the face) where the LLM got it right and the doctor didn't (he said it was diabetes related). It may have saved that person's life by causing them to go to a different doctor with the LLM's diagnosis and getting the proper treatment.

What I don't understand is why don't doctors use LLMs as a tool for a quick second opinion before arriving at a determination? They have my data. They can put it into GPT 5.6 Pro or Fable, read the output, and choose to disregard it if they really want to, but if there's a 20% chance that it spots something they missed, why not?