"You wouldn't believe how much of a relief it has been. In your last visit, you saw me typing everything you were saying, right? I don't have to. I can listen to you and take very specific notes as necessary as opposed to focusing on both typing and listening to you at the same time. It has bought my stress levels down to here." (Indicated by his hand lowering)
Too many AI tools are built hastily for me to give my doctor’s (visibly awful) software the trust.
It's a shame, if they had an AI transcription tool that kept everything in-house I'd be much more comfortable with it. Or perhaps some kind of zero-knowledge cloud-based product where I hold the keys.
On the one hand, pre-LLM there was plenty of software out there for medical use, and they had to be HIPPA compliant, etc. I've worked with people who used to write that SW (they hated the job because HIPPA was so strict). The rational side of me is saying I shouldn't be biased against these, and that there's no way they would relax the requirements.
On the other hand, the emotional side of me screams: It's LLMs! Huge privacy concern!
I try to let the rational side win. I've always given consent. Especially because I liked my doctor pre-LLMs, and it seems silly to suddenly mistrust him and go to a doctor I don't know at all.
I've seen summaries that implied the opposite of what was actually said, and even the transcripts themselves often contain egregious errors. I can so easily imagine these tools summarizing "I drink coke often" to "I use cocaine often", as a random simple example. All nuance gets lost, all context gets lost and unless you're vigilant and looking out for these types of errors it's so easy to slip by the cracks.
They're amazing in a support role, but not equipped as well for primary care roles.
If you present with anything other than healthy and fine, the ones I have had to see are less than helpful.
They simply do not have the depth or breath of training as a doctor. They're very good at executing a plan as part of a care team, and even making changes to that plan based on patient need. But they just simply aren't as good as doctors.
There's a reason they're replacing doctors. . . They're cheaper. That's it.
Which is great, because anyone suggesting AI should replace clinical judgment and work is an idiot.
I made a single presumption: that the PCP's use of AI for this situation is not mandatory. Is it mandatory or is it not?
In 2024 when the AI empathy snakeoil machine was inflicted on the nurses, the nurses had no choice in the matter. And management has already established that they may do something like this again.
Your situation is completely different. It's a doctor overseeing their own tool. If the AI makes a mistake, the doctor is responsible and would presumably take responsibility. Who is responsible if an AI tool makes a mistake in a performance evaluation of an employee? The employee! The employee is guilty until proven innocent. But the employee gets no choice in whether that AI tool is used against them (absent a union agreement which dictates how AI tools can be used).
The reason I fear the misuse of AI is because it is so useful.