Hacker News Reader

Comment by thw_9a83c | Hacker News Reader

It sounds like you should never trust any medical advice you receive from ChatGPT and should seek proper medical help instead. That makes sense. The OpenAI company doesn't want to be held responsible for any medical advice that goes wrong.

Obviously, there is one piece of advice: Even if LLMs were the best health professionals, they would only have the information that users voluntarily provide through text/speech input. This is not how real health services work. Medical science now relies on blood/(whatever) tests that LLMs do not (yet) have access to. Therefore, the output from LLM advice can be incorrect due to a lack of information from tests. For this reason, it makes sense to never trust LLM with a specific health advice.

View on HN

>It sounds like you should never trust any medical advice you receive from ChatGPT and should seek proper medical help instead. That makes sense. The OpenAI company doesn't want to be held responsible for any medical advice that goes wrong.

While what you're saying is good advice, that's not what they are saying. They want people to be able to ask ChatGPT for medical advice, give answers that sound authoritative and well grounded medical science, but then disavow any liability if someone follows their advice because "Hey, we told you not to act on our medical advice!"

If ChatGPT is so smart, why can't it stop itself from giving out advice that should not be trusted?

At times the advice is genuinely helpful. However, it's practically impossible to measure under what exact situations the advice would be accurate.

I think ChatGPT is capable of giving reasonable medical advice, but given that we know it will hallucinate the most outlandish things, and its propensity to agree with whatever the user is saying, I think it's simply too dangerous to follow its advice.

And it’s not just lab tests and bloodwork. Physicians use all their senses. They poke, they prod, they manipulate, they look, listen, and smell.

They’re also good at extracting information in a way that (at least currently) sycophantic LLMs don’t replicate.

> They poke, they prod, they manipulate, they look, listen, and smell.

Rarely. Most visits are done in 5 minutes. The physician that takes their time to check everything like you claim almost does not exist anymore.

Here in Canada ever since COVID most "visits" are a telephone call now. So the doctor just listens your words (same as a text input to an LLM) and orders tests (which can be uploaded to an LLM) if they need.

For a good 90% of typical visits to doctors this is probably fine.

The difference is a telehealth is much better at recognizing "I can't given an accurate answer for this over the phone, you'll need to have some tests done" or cast doubt on the patient's accuracy of claims.

Before someone points out telehealth doctors aren't perfect at this: correct, but that should make you more scared of how bad sycophantic LLMs are at the same - not willing to call it even.

> telehealth is much better at recognizing "I can't given an accurate answer for this over the phone, you'll need to have some tests done"

I'm not sure this is true.

Again, it's not that all telehealth doctors are great at this, it's that LLMs are awful at caving in to saying something with warnings the reader will opt to ignore instead of being adamant things are just too uncertain to say anything of value when continually prompted.

This is largely because an LLM guessing an answer is rewarded more often than just not answering, which is not true in the healthcare profession.

I follow the logic, I'm just not sure the claim is right.

LLMs almost never reply with I don’t know. There’s been mountains of research as to why this is, but it’s very well documented behavior.

Even in the rare case where an LLM does reply with I don’t know go see your doctor, all you have to do is ask it again until you get a response you want.

That depends entirely on what the problem is. You might not get a long examination on your first visit for common complaint with no red flags.

But even then just because you don’t think they are using most of their senses, doesn’t mean they aren’t.

It depends entirely on the local health care system and your health insurance. In germany for example it comes in 2 tiers. Premium or standard. Standard comes with no time for the patient. (Or not even being able to get a appointment)

I don’t know anything about German healthcare.

In the US people on Medicaid frequently use emergency rooms as primary care because they are open 24/7 and they don’t have any copays like people with private insurance do. These patients then get far more tests than they’d get at a PCP.

Agreed, but I'm sure you can see why people prefer the infinite patience and availability of ChatGPT vs having to wait weeks to see your doctor, see them for 15 minutes only to be referred to another specialist that's available weeks away and has an arduous hour long intake process all so you can get 15 minutes of their time.

ChatGPT is effectively an unlimited resource. Whether doctor’s appointments take weeks or hours to secure, ChatGPT is always going to be more convenient.

That says nothing about whether it is an appropriate substitute. People prefer doctors who prescribe antibiotics for viral infections, so I have no doubt that many people would love to use a service that they can manipulate to give them whatever diagnosis they desire.

> Physicians use all their senses. They poke, they prod, they manipulate, they look, listen, and smell.

Sometimes. Sometimes they practice by text or phone.

> They’re also good at extracting information in a way that (at least currently) sycophantic LLMs don’t replicate.

If I had to guess, I think I'd guess that mainstream LLM chatbots are better at getting honest and applicable medical histories than most doctors. People are less likely to lie/hide/prevaricate and get more time with the person.

> Sometimes. Sometimes they practice by text or phone.

For very simple issues. For anything even remotely complicated, they’re going to have you come in.

> If I had to guess, I think I'd guess that mainstream LLM chatbots are better at getting honest and applicable medical histories than most doctors. People are less likely to lie/hide/prevaricate and get more time with the person.

It’s not just about being intentionally deceptive. It’s very easy to get chat bots to tell you what you want to hear.

So ask it what blood tests you should get, pay for them out of pocket, and upload the PDF of your labwork?

Like it or not there are people out there that really want to use webMD 2.0. they're not going to let something silly like blood work get in their way.

Exactly. One of my children lives in a country where you can just walk in to a lab and get any test. Recently they were diagnosed by a professional of a disease which chatgpt had already diagnosed before they visited the doctor. So, we were kind of prepared to ask more questions when the visit happened. So I would say chatgpt did really help us.

That makes sense. ChatGPT helped by providing orientation advice and guidance regarding your children's medical condition. After that, however, you visited a doctor who is taking responsibility for the next steps. This is the ideal scenario.

AI can give you whatever information, be it good or wrong. But it takes zero responsibility.

Reply on news.ycombinator.com