Hacker News Reader

Comment by dogmatism | Hacker News Reader

Call me when an LLM can ingest a pts medical record from 3 different sources in multiple formats ranging from structured data to free form text stored (I shit you not) as TIFF files, extract the actual information, and present it for review prior to a visit

and for a follow up visit, read the prior visit's note and orders, search/track down all the test results (may be at more than one institution with multiple EHR systems that don't interchange) and summarize

or even just be able to take verbal orders like it used to be: "get a CBC, CMP, TSH, and CXR"

The NEJM CPC's are after the hard work has already been done to get the data to present the case

View on HN

> Call me when an LLM can ingest a pts medical record from 3 different sources in multiple formats ranging from structured data to free form text stored (I shit you not) as TIFF files, extract the actual information, and present it for review prior to a visit

LLMs were born to do this.

> or even just be able to take verbal orders like it used to be: "get a CBC, CMP, TSH, and CXR"

GPT-4 of course knew what that meant, although I didn't. I told it to roleplay a potential follow-up:

> "Patient with fatigue, weight loss, mild fever. CBC shows anemia, elevated WBC. CMP reveals elevated liver enzymes. TSH normal. CXR clear. Consider ESR, CRP, ANA, and abdominal ultrasound. Possible infection or autoimmune condition. Referral to a hematologist may be warranted."

I know an LLM can do this when given the data in an ingestible form

it's the taking of data from disparate sources including literal paper faxes that's the hard part

You’re being dogmatic about your, what seems to me, objections to LLM skills.

Making data ingestible is just laborious and unappetising to an engineer. So hand it over to a data entry operator who’ll do this happily for a wage.

I’m not sure why the issue of making data available to an LLM is a failure of the LLM.

Btw, OpenAI’s LLM can very well parse images, read handwriting, fix sloppy spellings, and extract objects from unstructured data.

There's no need for paper faxes - fax is a digital format. But of course, if you intentionally and pointlessly output to formats that make input to an LLM more work, it will be hard.

Other than that dealing with disparate data formats is something LLMs can already do very well, and that can be improved fairly significantly by putting a "straightjacket" on it (making it use APIs or selecting only tokens that fits a given grammar)

I mean, have you used GPT-4? Free form text is its bread and butter, and yes, it can absolutely ingest images and read text in them, and combine that with other information from multiple sources in structured or unstructured text or image form, no problem!

What you are describing sounded decades away last year. Today it's not something you can rely on quite yet if you need accuracy. But if you still believe it's decades away then you need to update your perception of what's going on in AI right now. Forget the free GPT-3.5 or Bard, you really need to try GPT-4 to understand what's about to happen.

see my comment above

Pt sees Dr in office A, has labs and scans ordered. Labs order is printed out and given to pt, who takes it to lab A. Scan #1 is ordered, and order faxed to facility B, who independently calls, schedules, performs, and interprets scan #1, same for scan #2, but at a different facility. Office A has no electronic link to any of these facilities

How does LLM get the data?

A human knows the patterns in the area, and calls around for it, then collates it and feeds in to office A's EHR, where if lucky, LLM can do it's thing

LLM is only saving the bare minimum of the work

In an industry that is still reliant on the fax, I'm not holding my breath for the advent of the LLM savior

In the situation you describe humans fail constantly. My experience with the medical system has decidedly not been that magically competent back office people call every office you've ever visited to assemble your complete medical history from scraps and effectively organize it for the doctor. It's a constant game of telephone where you have to do most of the work yourself, calling around and giving instructions to receptionists to give to other people to get this stuff done. They can't even communicate reliably between the nurses and doctors in the same damn office to avoid asking the same questions multiple times every time you visit.

What you describe, if it worked reliably, would be an advance over current human-level average performance. And AI will absolutely be able to do it, too. Probably within ten years, though it will take longer to deploy after the capabilities are there.

No, you’re right, it doesn’t work reliably

I just think the y combinator crowd underestimates the human obstacles and ancient tech in the path of getting AI implemented

That only concerns the time needed to implement it though, not capabilities. If an AI is known to be significantly better at diagnosing than your GP, and you know and believe this (this of course has yet to happen and then take time to be noticed by people), then you will use it. You will demand it is used. People actually care about their health.

Not hard to strap GPT voice to a phone line, and provide it with some information of local labs and their contact information.

Also you said the office doesn’t have an electronic link but your example workflow includes the phone ;)

What you're describing is a process and information management failure that ought to be addressed first, sure.

I book a video consult. I talk to a doctor, who has an initial description of my symptoms from my booking on her computer. She takes notes on a computer (I have access to them). If needed, she orders labs on a computer. I go in, and there is no paper, just an electronic record, and while there may well be paper in the process in some instances behind the scenes, data entry people ensures it is all added to the electronic record. If labs are not needed, she'll submit my prescription electronically to my local pharmacy.

This is how it's worked when I talk to my doctor for years (UK; not nearly all GPs have gone this far)

It may well "only save the bare minimum" of the actual labour happening behind the scenes, but with a well functioning system, it saves the expensive, resource constrained part of the work.

> How does LLM get the data?

* Run database query to see where the type of scan is typically ordered (we know they can generate sql pretty alright)

* * Alternatively someone at one point needs to just make a list

* Call the office (voice conversations is clearly possible, it's a feature in chatgpt as an app)

* Receive fax

* gpt-4-vision or OCR then GPT-4 or however you want to interpret the data

It's not like it's not work to build something like this, but all the pieces are right there.

This reminds me of what happened in self-driving. It went from "decades away," to "it works but you can't quite rely on it for accuracy" in a year and then it stopped cold for some reason.

Because it kept crushing peoples knees.

Useful Full-Self-driving is arguably harder than "useful automated medicine" in two ways. First, you need insane reliability to match human performance. Much higher reliability than doctors even. Second, it's not "just" information processing somewhere in the cloud. It has to run in real time inside a car. Both are very hard in that the conditions are very context dependent and vary a lot (a.k.a. the real world is messy).

> and then it stopped cold for some reason

By stopped cold you mean "there are actual real world self driving cars delivering actual real world passengers in select locations"?

So you describe a couple of things LLMs seem near ideal for, and one thing that is a horrific failure of information management that it's shocking if doctors are wasting their time on, and a case for other automation.

At my GP at least, all of this is presented to doctors digitally, all notes are digital, and all requisition of follow up tests are digital, and my initial consultations are all phone or video and only follow up visits are in person and only if a physical examination or tests are needed.

Reply on news.ycombinator.com