Comment by throwaway153371 | Hacker News Reader

Comment by throwaway153371 | Hacker News Reader

throwaway153371

Unpopular take: ChatGPT is better at diagnosing medical problems than a doctor who does a cursory evaluation.

By cursory, I mean an evaluation that does not take into account a very thorough assessment of all the evidence. P(condition | evidence) is not very accurate when the evidence is vague and sounds like many other conditions. It is a rare doctor who spends more than a few minutes asking about all possible symptoms, even those that may not occur to the patient that they have. Such doctors certainly exist, and I’ve had a few, but for the most part I have to do my own research and present it to a doctor for it to even be considered as a possibility.

As anecdata of one, I’ve been having all sorts of weird rashes that come and go. Could be eczema, lupus, psoriasis, who knows. Googling makes you think it could be anything. So I took my entire medical history, all medications, all symptoms, their frequency, etc. and threw everything into ChatGPT with the prompt “Give me a list of possible diagnoses ranked by likelihood based on the evidence”. At the top of the list “Mononucleosis (EBV induced)”.

Ah, but I forgot to mention that I had mono 12 years ago. I can’t catch that twice. So I told ChatGPT that I had it previously. “Still mono. Possibly recurrent due to some problem of the immune system.” So I went and got an EBV antibody panel. Positive for all three, including the EBNA test which is positive if you ever had mono previously.

It blew my mind that a LLM sussed this out, because mono wasn’t even on my radar.

It’s not just unpopular, it’s dangerous.

Please people, for the love of life and health, don’t trust your medical issues to LLMs.

You’re lucky it was just mono.

Maybe find a better doctor if you feel you’re not getting good enough care…

I agree with this. Any coder can tell you that AI will write some decent code, but it takes (at least at this point in time) an experienced coder to tell "this is crap code" or "this is decent code" – find a better doctor, because it's going to be hard to tell whether the advice it's giving you is the equivalent of crap code or good quality code.

I guess I don’t understand how it’s dangerous. I’m not giving myself any kind of independent treatment based on the output of an LLM. But it provides possibilities that doctors may have overlooked. Those doctors can then empirically assess the possibility in detail to determine whether it is correct.

I think parent comment is suggesting it's dangerous for you to publicly tell others that ChatGPT is better than their doctor at diagnosing them (however little of your doctor's attention you get) regardless of whether there is or isn't truth to it in certain cases.

People can't know when they should/can trust an llm over a doctor. People will also misuse LLMs in the context of medical diagnosis where asking the right questions and examining the right places (and properly) can be critical.

People have biases, people can have psychosomatic symptoms, people can be easily influenced, etc.

> I think parent comment is suggesting it's dangerous for you to publicly tell others that ChatGPT is better than their doctor at diagnosing them (however little of your doctor's attention you get) regardless of whether there is or isn't truth to it in certain cases.

Yes, exactly.

Doubly so when one has no medical expertise.

The danger is on the other side, you give the AI all your symptoms, it tells you you're fine, nothing is wrong, don't do anything. And then you ignore it and don't go to the doctor and fall over dead.

Unlikely, probably. ChatGPT aims to please.

There's also the risk that it recommends something that is in appropriate, but within your ability (like over-the-counter meds that shouldn't be used in your condition).

But - all this stuff happens all the time anyway. Most people don't involve a doctor until they encounter something really wrong that they don't recognize. They're more likely to mention it to someone else offhand and may take whatever recommendation they get.

> it tells you you're fine, nothing is wrong, don't do anything

Fair point. I could see a lot of people doing that.

> The danger is on the other side, you give the AI all your symptoms, it tells you you're fine, nothing is wrong, don't do anything.

It goes both ways. Replace "you give the AI" with "you give the doctor" and the issue "nothing is wrong" does not magically disappear. As a matter of fact, many doctors I have spoken with couldn't diagnose a thing and I wouldn't trust them.

One of them googled about the potential symptoms of the disease in front of me because he had no familiarity with it. And that's fine I guess since humans cannot know everything? But this only reinforces the point that LLMs can definitely have their place in assisting people, and their doctors, with diagnosing their health conditions.

Imagine you have a doctor who has read through literally all the medical books, documentation, research topics, experiments ... whatnot. Human of that kind does not and simply cannot exist. I think not many people understand the value and potential of LLMs, no matter how much this particular topic would be seen as sensitive or controversial.

> I’m not giving myself any kind of independent treatment based on the output of an LLM. But it provides possibilities that doctors may have overlooked.

You might not, but others would.

I mean people take random advice from the internet all the time. IIRC There was a whole craze about ingesting or injecting bleach to “cure autism” or something.

Random advice from an LLM is no better.

Even worse, maybe, since there’s not going to be a bunch of negative replies to whatever the LLM outputs.

I think there are two ways using LLMs for medical advice are dangerous: 1) Treating oneself after getting bad advice or 2) Not getting treatment at all because an LLM thought a critical condition was normal.

But the article and the comment both describe situations where they talked to a physician first, went to ChatGPT to refine their understanding, and then went back to a physician. Seems like a good use of LLMs.

OC started their comment with

> Unpopular take: ChatGPT is better at diagnosing medical problems than a doctor who does a cursory evaluation.

Which is dangerous and harmful advice.

No reason to muddy the waters.

It must be wonderful to live in a place where that's an option.

Due to my partner's work, I live in a town of about 100,000. There is a doctor shortage - New patient intake can be months. The system hangs on with physicians' assistants. Going to a specialist often means months of waiting or hours of travel.

We are living in a degraded health system between the lack of doctors, burnout of doctors and nurses, and a private equity system buying up everything in sight and sucking out every last dollar it can.

Maybe it's better in large cities.

I don’t live in a large city.

I’d bet that, if you live in the US, there are options for doctors within a 1-2 hour drive from you.

I drive an hour and a half to see an independent dentist I trust.

I’m lucky enough to have a decent doctor only 20 mins away.

Either way, I’d never trust my health to an automated system. Especially one which can’t be held accountable in any way.

Depends on how you use it. Just using LLMs is probably not good but LLM + seeing the doctor is quite likely better than just seeing a doctor occasionally.

At least LLM advice is normally quite sensible compared with a lot of the junk advice out on the internet.

This doesn't seem relevant to the post, rather an anecdote where you confirmed that you had EBV in the past with the help(?) of ChatGPT (which you already knew). I don't see where you diagnosed the cause of your rashes.

I didn’t. I followed up with a doctor and mentioned the possibility of mono who then performed the EBV tests and provided the official diagnosis after previously seeing a different doctor who incorrectly diagnosed the problem as eczema.

What research there is about SOTA LLMs and diagnoses does suggest they are really good.

Reply on news.ycombinator.com