Comment by basil-rash | Hacker News Reader

Comment by basil-rash | Hacker News Reader

Some red flags:

> It hasn’t been tested on people with real health problems — only on actors trained to portray people with medical conditions.

Obviously the FDA won’t allow them to operate in real patients, but designing a system that knows how to identify which script you had your actor play is very different from designing a doctor.

> an LLM has the unfair advantage of being able to quickly compose long and beautifully structured answers, Karthikesalingam says, allowing it to be consistently considerate without getting tired.

The last thing I want is to read through pages of AI blogspam at the doctors office. Where’s the AI that takes its pages of fluff and boils it down to the point?

That said, I am cautiously optimistic, as there’s definitely room for improvement. I’m lucky to be basically the same demographic as doctors, so they tend to actually listen to me and believe me. I know from second hand experience that for folks who don’t look like their doctors, getting one to take them seriously can be herculean. One individual I know had the UCLA (male) medical staff call security to escort her out of the building because they thought she was being too hysterical and should just tough out whatever it is she was dealing with (they refused to look). She then went to a (female) private practice gyno who took one look and immediately saw there was indeed a massively pain-causing complication present.

Let’s hope the AI doesn’t train on the wrong transcripts though…

Aside, the best part is UCLA didn’t want to let her graduate until she paid the medical bills from that “visit”. She ignored every one and they seemed to forget about it. Medical incompetence has its upsides.

Aside aside, this is also why “Shouldn’t the best test scoring individuals be admitted to (medical) schools, regardless of demographics? To hell with diversity, you want the smartest doctors possible, don’t you??” is flatly invalid.

> The last thing I want is to read through pages of AI blogspam at the doctors office

I think you underestimate how much of an effect the doctor's style of response can have on patients. A curt response and an embellished response may both provide the same facts, but the latter is more likely to resonate with people. At the end of the day, everyone thinks they're special and not just "another patient".

I am not advocating for "pages of AI blogspam" -- LLMs can be instructed to craft better responses than that -- in the real world, doctors suffer from empathy "drain" very often, LLMs don't.

That's all well and good if the patient doesn't know they're interacting with an AI though!

To clarify, my stance on this whole thing is mixed.

That’s true when the doctor is a human, I want humans to acknowledge my humanity and (ideally) be able to put themselves in my shoes and empathize with how I’m feeling. But a robot cannot do that. A robot has no feeling, and will never be able to understand my feelings. The best it can do is parrot its authorized “authentic compassion” markov chains. Perhaps to some that’s enough, but I’d prefer to just get the data I need and leave.

>The last thing I want is to read through pages of AI blogspam at the doctors office. Where’s the AI that takes its pages of fluff and boils it down to the point?

Relative to the average patient, you are an extreme outlier. You are almost certainly well above the population average in terms of your level of education and cognitive ability. The average patient is well below that population average, because healthcare is very disproportionately needed by people who are older, less educated, have English as a second language, are suffering from cognitive impairments etc etc.

The number of people who complain that their doctor spent too long talking to them and gave too thorough an explanation rounds to zero, which makes an infinitely patient and free-at-the-margin AI doctor an obvious improvement. A well-trained LLM will make the same sort of assumption about you that a human doctor does - this guy presents his history concisely and in clinical vocabulary, so I can probably skip the pleasantries. The LLM should (if trained towards the right outcomes) not make the sort of prejudiced judgements that caused your friend's unfortunate experience. If it does, we'll see it in the data.

Also cautiously optimistic about this. If it can get people quicker diagnoses, or even handle very simple stuff and reduce doctors' workloads without compromising on quality, it's great. You'd need to verify there is no data collection anywhere, and that the business model stays straight-forward and doesn't become advertising-prone.

I'd much rather see something like this being owned by a government of some sort, building it in the open and without corporate incentives.

For the top-shelf LLMs at least, you can just ask them to be brief. In fact they are very skilled at consolidating lots of data into a few points.

But that won’t happen when the team making it is focused on pseudocompassion as an OKR. Engineers in general need to understand that what people want from a human and what they want from a bot can be different. Compassion from a robot is almost never desirable.

Reply on news.ycombinator.com