A popular algorithm to predict sepsis misses most cases and sends false alarms
statnews.com
statnews.com
I can go for a run on a hot day- I'll be pyrexic, tachypnoeic, and tachycardic- enough to trigger a sepsis alert. Whereas an elderly patient who is a bit more confused than normal may have a very severe raging infection with few changes in their markers.
The problem here then, i'd guess, is how many of those extrinsic factors are unmeasurable in practice; and how little data we'd have on them.
Not that "Epic" is the name of an Electronic Health Record system that embeds this proprietary algorithm for detecting sepsis.
Key Points
Question How accurately does the Epic Sepsis Model, a proprietary sepsis prediction model implemented at hundreds of US hospitals, predict the onset of sepsis?
Findings In this cohort study of 27 697 patients undergoing 38 455 hospitalizations, sepsis occurred in 7% of the hosptalizations. The Epic Sepsis Model predicted the onset of sepsis with an area under the curve of 0.63, which is substantially worse than the performance reported by its developer.
Meaning This study suggests that the Epic Sepsis Model poorly predicts sepsis; its widespread adoption despite poor performance raises fundamental concerns about sepsis management on a national level.
Results We identified 27 697 patients who had 38 455 hospitalizations (21 904 women [57%]; median age, 56 years [interquartile range, 35-69 years]) meeting inclusion criteria, of whom sepsis occurred in 2552 (7%). The ESM had a hospitalization-level area under the receiver operating characteristic curve of 0.63 (95% CI, 0.62-0.64). The ESM identified 183 of 2552 patients with sepsis (7%) who did not receive timely administration of antibiotics, highlighting the low sensitivity of the ESM in comparison with contemporary clinical practice. The ESM also did not identify 1709 patients with sepsis (67%) despite generating alerts for an ESM score of 6 or higher for 6971 of all 38 455 hospitalized patients (18%), thus creating a large burden of alert fatigue.
Conclusions and Relevance This external validation cohort study suggests that the ESM has poor discrimination and calibration in predicting the onset of sepsis. The widespread adoption of the ESM despite its poor performance raises fundamental concerns about sepsis management on a national level.
Here they just replicated all the same issues the current model of diagnosis faced.
After the surgery my parents(both doctors) told me to watch for fever as a sign of reinfection. I had to remind them there was no fever in the first place and that was likely a bad proxy for infection in her case.
She already had a lot of burst veins from the hospital, so we decided to just do what her surgeon said, basically take amoxicillin for another 7 days and just about it.
Ultimately doctors have to take the whole clinical picture into account instead of relying on a couple markers or vital signs.
You’re right that a big problem is the inability of our system to do testing at home, which might greatly expand the scope of the clinical picture. But it still wouldn’t be a cure-all
The biggest issue is the costs of false positives, which with any early intervention protocol are likely to be high and, questionably, on balance a net negative.
I have been in a similar situation where after several days of illness my condition didn't appear bad enough and the doctor sent me home. This was some 15 years ago and they didn't have quick tests back then, but luckily they did prescribe a lab CRP test and called me back in for treatment immediately after they got the lab result later that day.
A lot depends on your index of suspicion, how well the person is, what the risks are of getting it wrong (in both directions), what safety nets you have available to you (i.e. are they home with someone sensible who can keep an eye on them).
In summary - there is no perfect test or perfect heuristic to apply to everyone.