Hacker News Reader

Comment by doctoboggan | Hacker News Reader

My wife is a doctor and she already has an AI scribe that records visits, transcribes them, summarizes, and prepares her orders.

It seems like having that scribe also review lab results wouldn’t be far off, but we are still not ready to let the AI system interpret and inform the patient without human intervention.

The problem here wasn’t that a human missed the low platelet count, it’s just that it wasn’t reviewed yet. Both a doctor and an AI system would haven spotted the issue immediately.

I am not saying this isn’t a great result, but the AI intervention must come from the patient side, not the care providers side. The providers have too much at risk, while the patient is able to pick and choose their own AI and decide what levels of risk they are able to accept.

View on HN

It doesn't need "AI" or a "Scribe" to spot a low platelet count in lab results either, computer systems have been able to do this for decades.

There's a systemic failure that a computer wasn't already flagging up the low count for immediate follow-up.

What I find interesting is that AI (or more specifically LLMs / chat-GPT ) has had such an impact it's broken down barriers from people who previously would be against any computer involvement or automation. People have gone from, "You must not automate anything" to "We should feed everything into chat GPT"..

And from someone who has in the past been interested in statistical modelling, it's dizzying. It's as if statistics were seen as boring and fallible, but "AI" is exciting and wonderful.

Most modern EMRs will flag results like this. If it happened to arrive in the middle of the night like the author said, the PCP would have just been asleep. I don’t think there are systems for waking up PCPs for concerning lab results.

I think the suggestion is that the EMR should flag dangerous conditions like this, and notify the doctor on duty to at least take a look at it.

This would be enormously expensive. The number of things an EMR thinks are an imminent threat will vastly outnumber the things that actually are thanks to liability concerns.

So now you need an extra on-call doctor just to filter out the false positives, or the on-call doctor gets notification fatigue and ignores them.

Or in the best case you send a lot of people to ER in the middle of the night for no reason. Again thanks to liability concerns, but this time on the part of the primary care practice.

> I think the suggestion is that the EMR should flag dangerous conditions like this, and notify the doctor on duty to at least take a look at it.

Or a PA or CRN. They are cheaper. As the last resort you can have an AI doing it :-)

And nor should there be. PCPs are basically by definition for non-emergent situations. If the PCP was at all concerned that the patient had something that was an immediate threat, they would have sent them to the ER in the first place.

Having someone on-call to deal with routine lab results that just happen to catch a possible imminent threat would be an enormous waste of resources.

The main issue is that some developer has to be told to make it so that if plateletCount < 25.000 plateletCountColour = "bright red". That is, a big issue in software development is managing requirements.

AI is more flexible and can make up requirements on the go, that is, based on information it has it "knows" that a low platelet count is bad and what the recommended course of action is.

Of course, search engines and public websites do the same thing - "what should I do if I have a low platelet count" or "low platelet count and red skin dots" are valid search queries.

I believe it's because statistics can be interpreted in different ways and requires analysis (work). "AI" just gives you an answer, even if it's wrong.

Yes, but this doctor in particular doesn't seem to have any such automated system. They seem to reviewing it all manually. From the post,

> I was told I’d receive a response in 2-3 business days

2-3 "business days"? Yea, I can see how LLMs are going to be super helpful in patients taking their health more in their own hands.

Interpreting blood results it not something objective. There are bounds sure but the interpretation is always based on all the available results.

Source: resident in a few months

I think there needs to be a third option. Today I can plug a device into just about any vehicle and diagnose / debug tens of thousands of issues. This needs to be a thing for humans.

I should be a able to put a few drops of each of my bodily fluids into a set of disposable receptacles or strips and it should be able to test for hundreds of thousands of issues initially and millions of issues in the future. A pocket fictional Dr. House so to speak. This would distribute the testing workload to the people at home or work and give doctors something better to start with than only perceived symptoms and would allow prioritizing patients during triage. It should find everything from the obvious emergencies to the most obscure anomalies that could be addressed at a later time. It should also be able to tell me what to start or stop eating.

There should also be a high-risk-accepted mode where the device can make guesses about what is occurring based on all the data it gathered even if it does not have 100% of the scientific data to do so. All this in a sub $1000 device I can buy online without a prescription. This device must not have any way to leak patient data. Connect it to a printer via USB and it just makes a hard copy to hand to a doctor or they could just read this device with their eyeballs. No cell phone required. No wifi or bluetooth. No dial home. No uploading to LLM's. No cloud. Patients can download an updated OS and firmware image for updated medical data and update using a thumb-drive or micro-SD.

If only we could get some funding for this, and maybe get a charismatic college dropout as CEO to lead it bravely, whatever the results?

Maybe someone with a really interesting voice?

Yes, you should have a medical tricorder. It’s a scandal these don't exist.

I said in another thread that I think that the first step can at least be that an LLM helps triage/prioritize the test results so docs can at least know an order in which to review them, especially with radiology/imaging results which are mostly words-based, rather than the numerical results that often come out of blood tests.

I can't tell if you're trying to say that ChatGPT hasn't added anything here?

Yes I'm sure a human doctor would have noticed the low platelet count -- if they bothered to look at it. Which they didn't (or at least hadn't got around to yet).

Sorry, I edited my comment with a final statement. I think chatGPT added a lot and I think the future of AI in medicine is with patients bringing their own AI to help interpret the results.

On the provider side, it might be useful to automate the analysis of various patients, particularly ones who have elusive diagnosis. I hear stories of some patients who sent from specialist to specialist trying to figure out what's wrong with them where the symptom is obvious but the cause is not. I had a colleague who had vertigo and it took 6 months to figure out what was going on. She couldn't work or drive. AI/ML might help in that scenario on the provider side. I understand it's pretty good at analyzing imaging as well.

If the patient agrees to it (and is informed of it), I can definitely see value in automatically recording, transcribing and logging a visit (as long as the doctor does a final approval / editing phase). Part of me wants to enable that by default in some of our Teams meetings too because for some reason they often end up chaotic and emotional (enthousiastic / energetic, I mean, instead of businesslike), and it's difficult to bring it back to some core points.

How does she disclose that to her patients? How many refuse?

Her legal team said they are allowed to get verbal consent. She has a sentence she says verbatim something along the lines of “ I am going to use this computer to record this visit and take notes. Is that OK?”

Almost no one refuses.

If I heard that, I would think the person asking is going to take the notes, not feed the recording into a LLM.

I do NOT want a transcription ERROR in my medical records.

I bet if she says something like "AI will transcribe" then there will be more hesitation, although maybe not more refusals because most people don't know how it can make mistakes.

Anecdotally, she has reported it’s _much_ better than her human scribes. And she is still ultimately responsible for the content and reads it all before signing the note.

Are audio recordings of patients shared with third parties?

I believe so, yes. This is nothing new however as "tele scribe" companies have existed for a long time. These sort of cases are exactly what HIPAA regulations were designed to enable.

That sounds like the old trick of pulling out a pen and paper and saying "I'm going to record this, okay?" with a tape recorder in your briefcase.

Are they informed that their personal medical information will be shared with an unknown set of third parties / LLMs subject to manual QA / cloud services with telemetry?

Honestly sounds like she's opening herself up to massive lawsuits (depending on State law). I would reach out to the legal department if any doctor pulled this kind of shit on me.

If you think the clinic's lawyers haven't already reviewed this setup with a fine tooth comb then you haven't seen how medical lawyers work. This isn't a rogue PCP surreptitiously recording visits. Its a contractor that was selected and negotiated with from on high that was subsequently deployed to all clinics in her large company.

Also, I guarantee you signed a paper when you first were enrolled with your PCP giving them permission to share your medical records with third parties when needed (subject to HIPAA compliance of course).

I'm sure they have.

I don't have a PCP.

https://www.forbes.com/sites/daveywinder/2025/02/02/medical-...

https://www.gartner.com/en/newsroom/press-releases/2025-02-1...

https://www.usatoday.com/story/news/investigations/2024/04/0...

Reply on news.ycombinator.com