Healthcare org with over 100 clinics uses GPT-4 to write medical records

I'm surprised to see that healthcare providers would be allowed to send patient data to OpenAI under HIPAA.

According to OpenAI's security page[0]:

OpenAI has experience helping our customers meet their regulatory, industry and contractual requirements (e.g., HIPAA). Contact us to learn more.

But there are no further details. Does anyone know what they're actually doing here? Do they have a way to legally process PHI, or are they depending on some sort of anonymization method to claim that it's no longer PHI?

[0]: https://openai.com/security/

The article is missing some details so I won't comment on the legal and privacy issues. But from a purely technical standpoint, this realizes part of the vision of an AI medical scribe that former Google CEO Eric Schmidt laid out in a healthcare conference keynote speech in 2018. I think his predictions are generally proving fairly accurate.

https://www.healthcaredive.com/news/himss18-eric-schmidt-key...

https://youtu.be/ACQes9erfsw

The company in question:

https://en.wikipedia.org/wiki/Carbon_Health

Not surprisingly, this company seems to have a knack for intellectual dishonesty. For example:

"The use of scribes and transcription services is standard in the healthcare industry, and a majority of patients provide consent to have their visit recorded by their provider," a spokesperson told The Register on Monday.

To be recorded by an AI, maybe. To be rewritten (summarized) by an AI? No.

Generative AI models aren't perfect, and often produce errors. Physicians therefore need to verify the AI-generated text. Carbon Health claims 88 percent of the verbiage can be accepted without edits.

Sounds like pretty bad odds, actually. Given that -- especially in a penny-pinching environment like we have in this company -- we have to reckon with 12 percent of the summaries being broken in some way (perhaps very significantly). Because we know that, inevitably -- many, if not most of these summaries will never get edited.

A clinic testing the tool in San Francisco reportedly saw a 30 percent increase in the number of patients it could treat.

What they meant to say was: "A 30 percent cost savings for the company, at the expense of patient safety and expectations of personalized treatment".

Speaking as an LLM fan... This seems like a bad idea. Maybe its technically less error prone than a human summarizer, but the kinds of summarization errors GPT4 might produce are different and potentially harder to spot than a human.

> Generative AI models aren't perfect, and often produce errors. Physicians therefore need to verify the AI-generated text. Carbon Health claims 88 percent of the verbiage can be accepted without edits.

So the AI is only wrong 12 percent of the time!