Hacker News Reader

Comment by pythko | Hacker News Reader

I can tell you that the current focuses of AI implementations are around real, impactful issues: sepsis risk, readmission risk, deterioration index, etc.

The problems with AI in healthcare are:

1) People don’t want it to be a black box - that means quantifying the factors that go into a recommendation

2) Operationalizing AI recommendations is hard. AI tends to give gradiated information on binary decisions (e.g. there’s a 68% chance this patient is septic. Should someone go check on them? What if they were 49%?). The challenge becomes deciding how that information should be shown to people and what the acceptable false positive and false negative rate are.

3) The same problems of AI everywhere. Things like garbage in garbage out, unrealistic user expectations, feeling like it basically tells you what you already know, the challenge of getting insight from a pile of data.

View on HN

No, the problem with AI in healthcare is that like much of healthtech is that it further reduces the ability of providers (especially in hospital settings) to respond to fluid and evolving situations that may fall outside the dotted lines that the AI understands or scenarios the system allows you to work within. Specifically, it creates further red tape that providers need to worry about, more checkboxes on an iPad to be clicked, more time required per patient on administrivia.

It could be done well but it will be done poorly, will increase the burden of front-line workers while making administrators feel like they can say they accomplished a big project this year. At the end of the day rather than making healthcare more auditable, practitioners will learn to just quickly fill in bogus data on the new system so they can go deal with the patient that's coding and when the AI gives a recommendation a provider doesn't like they'll just ignore it anyway.

In a good system that wasn't falling apart at the seams, AI in healthcare would be a boon, but in a broken system that's falling apart and failing its front-line workers, it will just serve as a distraction and another burden.

I think what you described falls under #2 in my reply. “Doing it well” is not a trivial option that people are ignoring; “doing it well” is the thing people are trying to solve. AI is not a magic bullet that always makes everything better.

Honestly that's worse than I thought. I work in the field, particularly in relation to accountable AI, and it's not OK to have models that tell you whether to check on people to make sure they're not dying unless there is also a human checking every case, which I hope is what's going on. How would you like to be different than the training data and deemed "no risk, 100% confidence" when you actually have a life threatening problem?

In a hospital setting, nurses and doctors round regularly. No one is talking about using AI as a replacement for that, because no one has anything approaching that much trust in predictive models.

Predictive models are most often used as either an alerting mechanism or an additional data point on a dashboard. You need to careful of alert fatigue, where too many false positives cause humans to disregard all alerts from the model. And if you don’t get people ignoring alerts, you can waste a lot of people’s time and energy by having constantly having them run to check on someone who is actually fine.

Reply on news.ycombinator.com