Google’s medical AI chatbot is being tested in hospitals | Hacker News Reader

Google’s medical AI chatbot is being tested in hospitals

theverge.com

40 pointsel_hacker43 comments

LLMs hallucinate way too much for use in anything serious. And the privacy implications are harrowing

I barely even trust LLMs anymore to help me with orientating research, I keep finding out that half of what it says is a lie

Are you saying it's better they don't try? But you tried... Not quite sure what to think.

I totally agree on the hallucination front, that is a big problem.

But the privacy thing is less of a concern if we are only talking about "using" (not training) these medical models. You could upload your entire medical history to such a model within the context window, and then provide your latest symptoms to get a diagnostic from the LLM. The LLM has not remembered any information about you at this time (it isn't "within" the model itself), it is only in the context window.

Anyway, just pointing out that there is a world of difference between training these models and using these models. If you are only using, there is really very little fear of privacy assuming the whole interaction is discarded.

Have you tried GPT-4? Haven't seen it hallucinate once out of the 100+ questions I've asked it.

The one mistake I've seen it make was giving me the wrong pin number when asking it to read a datasheet via a plugin, but even then it was a real pin and I wasn't using core GPT-4.

Sure, but have you seen the workload of most doctors? A huge portion of it is simply writing notes on patients.

Note writing is ripe for an LLm, even with hallucinations. Write out your quick bullet points, let the LLM expand that content, re-read and correct. Since a human remains in the loop, it doesn’t have to be perfect.

Hallucinations will be squelched out pretty soon. Check perplexity ai app, it works pretty well with little/no hallucination imo.

Reading with the mindset of 'spot the hallucination' actually helps me focus more on the output of gpt4 and learn more quickly from it.

I don't read Wikipedia with as critical an eye.

We had superhuman diagnosis systems 50 years ago!

> Mycin was never actually used in practice. This wasn't because of any weakness in its performance - in tests it outperformed members of the Stanford medical school. It was as much because of ethical and legal issues related to the use of computers in medicine - if it gives the wrong diagnosis, who do you sue?

https://cinuresearch.tripod.com/ai/www-cee-hw-ac-uk/_alison/...

A lot of medicine is about pattern recognition and in that sense this could help just as it does for software development by boosting newer doctors who are early in their careers and building their intuition. I've seen this gap first hand even with doctors from the best schools and residency programs.

Though, it's probably going to take a few more iterations before its ready for prime time.

I say this as a non medical person but one with three doctors in my immediate family

I think it could also help experienced doctors who may be a bit stuck in their way of thinking and any doctor trying to diagnose rarer diseases and disorders. so many medical issues seem to have overlapping or identical symptoms so it would certainly help to remind doctors of the more rare options they may need to test for.

It seems inevitable in the future I won't have a choice whether my info (however personal) gets sucked up into some corporate database via AI chatbots. Even if I were to refuse to use one personally, I have no control whether the nurse I talk to is just typing what I tell them into a chatbot on their laptop.

Or even the doctor. They could consult with a chatbot rather than other doctors in refining their own opinion. I think that is more likely in developed countries, where this is more likely going to be a tool to be used by doctors directly, a more efficient way to sift through 10s if thousands if maybe relevant published studies and literature.

FWIW, healthcare has strong regulations around data. If your provider uses an approved chatbot (vs using public ChatGPT), there is more or less a guarantee that your data won't be sucked up by a corp.

I'd rather my doctor use a negotiated and approved model/tool than an unapproved one.

> there is more or less a guarantee that your data won't be sucked up by a corp

Not true. "The HIPAA Privacy Rule establishes the conditions under which protected health information may be used or disclosed by covered entities for research purposes" [1]. Your de-identified data can be sucked up by a corp for research purposes. You are protected because your identity will be removed before data is used for research.

https://www.hhs.gov/hipaa/for-professionals/special-topics/r...

Sure, I was emphasizing on the "however personal" part in the original comment. I am ok if my de-identified data is used for research.

Don't be distracted by today's excitement over chatbots. Companies are out there consolidating this data right now through traditional methods. Oracle, for example, has been saying the quiet part out loud.

https://www.forbes.com/sites/saibala/2022/06/26/larry-elliso...

It would be interesting to see if AI became antivaxer which we do see in some doctors of today.

this is a great comment for a clear demonstration of how hard it is to make real boundaries for humans and their healthcare services. "antivax" is a policy statement, not a medical statement, yet the comment is likely in good faith. The blurry gray lines involve humans being a social animal, not just a single body; administration, rules of law, economics are all part of "policy"

Oh no. And what patient data did they train it on?

There’s a push to privatize healthcare in Canada in the last few years. I worry the public system will crumble into an AI driven, grifter run trash heap and we’ll get the worst of everything where taxes stay high, because we have “free” healthcare, and real healthcare is private and only accessible to the rich.

>According to Google senior research director Greg Corrado, WSJ says, Med-PaLM 2 is still in its early stages. Corrado said that while he wouldn’t want it to be a part of his own family’s “healthcare journey,” he believes Med-PaLM 2 “takes the places in healthcare where AI can be beneficial and expands them by 10-fold.”

lol

Got it, so it's only for poor people :^)

What could possibly go wrong?

Google is suggesting this should be used in poor countries without enough doctors because they know it is (or essentially will be, after the tidal wave of malpractice suits happens) illegal in the US.

Perfect is the enemy of good.

It's more like barely tolerable is the enemy of complete crap.

Tell that to someone when their kid dies because some chatbot hallucinated the wrong diagnosis or treatment.

Let's not do that (but still keep in mind, that it is true). An individuals understandable inability to deal with statistical truth in times of personal hardship should not stand in the way of the best medical care, statistically.

And…if they didn’t have access to a doctor anyways? This sounds like a huge improvement over traditional non-medicine that would be used instead.

Lots of people simply don't get care due to the cost.

> Tell that to someone when their kid dies because some chatbot hallucinated the wrong diagnosis or treatment.

MedPalm seems to be accessible only to healthcare facilities. If the chatbot hallucinated the wrong diagnosis or treatment, the doctor will still be liable.

Real solutions are the enemy of random bullshit. A chatbot that we know will misdiagnose is of no use to anyone.

And there it is. You aren’t worried it will make an error, you believe it will always be wrong. Since doctors misdiagnose all the time anyways, just claiming that it might misdiagnose wouldn’t be strong enough to disavow it (since it almost certainly will misdiagnose at least as much as a human doctor).

This idea that because a human is sometimes wrong, it's okay for the chatbot to be wrong is one I find to be dangerous and unhelpful. The fact that people are fallible shouldn't encourage us to choose solutions with a high or unpredictable error rate. Some failures have a greater cost (i.e. launching a rocket) than others (delivering a package).

Does the chatbot misdiagnose less than a doctor? When the chatbot does misdiagnose, is it in manner similar to a doctor? In the same way that AI generated images always had weird and creepy fingers, does this chatbot consistently misdiagnose the common cold as brain cancer?

IMHO, without some real data, we're simply experimenting on people. That is wholly immoral.

After some contradictory information on error rates and failure modes, a Google employee is quoted in the article:

'Corrado said that while he wouldn’t want it to be a part of his own family’s “healthcare journey,” he believes Med-PaLM 2 “takes the places in healthcare where AI can be beneficial and expands them by 10-fold.”'

Yikes! 10-fold expansion, excluding any interaction with people this person really cares about. Let us remember that this is a 10x times increase over the current "places where chatbot are beneficial in healthcare", that is to say zero.

How is it illegal in the US? For one, it seems that it does conform to HIPPA. As per lawsuits, if the doctor is using it, they can't claim the chat bot gave them incorrect information - they should know better.

Double penalties. You are poor so you don't get good care and you also get to be a cobaye for a tech that can randomly (wrongly) guess your health problems.

Note: In English, "cobaye" (French) is "guinea pig". (Similarly used in English as an example of an animal used in lab testing.)

Devil's advocate: most health problems in developing nations are the basic stuff we solved 100 years ago in the west. An AI that can detect when to give someone a dewormer or some vitamines or something else without occupying a doctor could be very helpful

Not if the ai is hallucinating. Imagine if your doctor was extremely knowledgeable, but extremely high on acid every day and prone to confidently lying to you

Do you know how it works? If not then what's the point of this reply?

What are the odds that 5 doctors will do that?

Furthermore, what are the odds that an ensemble of 100 models whose output is ranked / voted / etc will be wrong?

What if we used LLMs to guide the search and ask questions / order and rank exams and analyses?

This brings us to the bitter lesson, we can scale LLMs and compute far far far better than we can scale humans.

As long as we can use compute to scale search and learning from data, we will keep making progress.

Chess was unsolvable, Go was unsolvable, Starcraft was unsolvable. We will get there.

We’re not discussing a long term thing. We’re discussing a business today using todays tech

Diagnosis is usually the easiest part of treating such conditions. The problems are more in lack of medical supplies, facilities, and personnel to administer treatment.

Reply on news.ycombinator.com