Google's healthcare AI made up a body part – what if doctors don't notice?

This reminds me of a story where an OCR error[1] likely contaminated training data (and the English language) with the term "vegetative electron microscopy". The article I linked also shows that some journals defended the legitimacy of the terminology.

I'm not sure if this class of error really counts as a hallucination, but it nonetheless has similar consequences when people fail to validate model outputs.

[1] https://news.ycombinator.com/item?id=43858655

You're right!

The correct diagnosis for your stated symptoms is that you have a Cloomie in your left Glompus.

A daily megadose of Ivermectin, over a 7 day period, should resolve your condition.

> Now imagine your doctor is using an AI model to do the reading. The model says you have a problem with your “basilar ganglia,” [basal meaning at the base, ganglia meaning clusters of neuron cells: neuron clusters at the base of the brain] conflating the two names into an area of the brain that [D]oes [N]ot [E]xist[!] [Dramatic, serious stare into the camera.] You’d hope your doctor would catch the mistake and double-check the scan. But there’s a chance they don’t. [And that brings us to the emergency room, where you are now, a forty-nine software developer presenting with a psychotic obsession for fact-checking everything you read on the Internet.]

Sounds like just a typo, not "making up a body part"

The arrogance of calling it a "simple misspelling". We get it; you have commands from above to deploy AI and you're too pathetic to morally question the directive, but at least let's not pretend that LLMs make typos now. "Oh, oopsie, it was just a typo."