Should Epic's algorithm be open to critique or is it IP that they can keep secret? If challenged should they be forced to explain it or at least provide the inputs? It's worrisome to me that the process behind these numbers is fairly opaque and is going to get even so with ML. It's often fairly difficult even for data scientists to precisely explain their own algorithms and weights that produce an output.
My money is that they will have to disclose inputs, if for no other reason than to address the uncertainty about race as a possible input.
The article suggests that they're already providing the inputs, "a list of the variables used to calculate the index and a rough estimate of each variable’s impact on the score".
There is an interesting Star Trek Voyager episode Critical Care where people were helped by an AI depending on their social status. I believe they hacked the weights so everyone got a treatment in a part of the episode.
I worry about this. It's easy to let things slide in terms of data quality and statistical rigor when building machine learning models, even in the best of times. When it's a problem people would actively rather not have to think about (which of these patients should get a ventilator, for example), that's even more true. Add to this the fact that Epic isn't exactly known for the quality of their engineering.
Triage for mass-casualty incidents (MCI) is already a rule-based system [0]. Modifying the rules appropriately for COVID and encoding them in an expert system would provide both efficiency and the transparency that ML lacks.