Hacker News Reader

Comment by gbjw | Hacker News Reader

Part of the problem with the reporting on death rates and, to a lesser extent, many of the tests and treatments of COVID-19, is the almost nonexistent emphasis on uncertainty quantification. A view might only be ‘different’ in the sense that the mode of the belief is shifted significantly, which is to be expected if existing beliefs are quite diffuse (i.e., uncertain).

View on HN

Former actual scientist here.

This! This, exactly! This, a million times over!

If I could teach people one scientific concept to help us get through this, it wouldn't be immunology or epidemiology. It would be an understanding of uncertainty, where it comes from, what it means, and how to handle it.

The funny thing is that they actually do this -- they show that based on their measurements of the assay's specificity, the confidence interval for the number of true positives includes zero. That means, even discounting all of the other biases, their study cannot provide a lower bound for the number of infections, which was the whole point of the study.

They then proceed to ignore this and use a point estimate for their further calculations.

The right thing to have done would have been to test the assay further, or move the study to an area where there was higher positivity and they could get a more reliable signal.

What I actually had in mind with my comment was the hydroxychloroquine trials. The interpretation of those results has been just saddening to the scientist in me. They've all been weak studies, for various reasons (many of them quite valid!), and so the results just don't have much power. And yet, the partisans jump on everything they like to push their agenda: Miracle cure! Worse than drinking bleach! Et cetera.

My reading of the data to date (note: not including anything that might have shown up in the last few days) is more nuanced. Hydroxychloroquine most likely has a small positive effect, but the data are consistent with anything from a modest negative effect to a modest positive effect to absolutely nothing at all. From its long history of previous use, the drug has known negative effects, but these are unimportant or manageable for most COVID-19 patients. However, any positive effect it may have is limited; the studies to date, albeit limited, have been powerful enough to detect a "miracle cure".

None of this says a word about mechanism of action. Someone is bound to say, "But what about zinc? You need to consider zinc!" The thing is, we did. These analyses are independent of mechanism of action. It's entirely possible that hydroxychloroquine's small positive effect is mostly or entirely caused by zinc interactions. Or not. We don't know! We'd have to do a study designed around zinc to test that theory out.

In my opinion, again as a science-y but not doctor-y person, is that giving hydroxychloroquine to COVID-19 patients in most cases will not hurt them, and might help a little bit. The biggest danger, I believe, is that people think it's more powerful than it is. We should not stop and think that hydroxychloroquine is good enough or a "cure". Because it isn't. And if a patient receives hydroxychloroquine instead of better care because people misunderstand science, that is a failing.

(The above is definitely a tangent. But getting it off my chest somewhere was good for my mental health!)

Problem is that most patients will recover from Covid 19, so unlike a more deadly disease, need a high bar for treatments.

Do you actually understand uncertainty?

I've, uh, spent significant portions of my life trying to understand just how well we know some particular measured values. And that's not counting the coursework.

So, yeah, I'm going to set myself up for a fall here by saying I've got a pretty decent professional understanding of the topic.

Reply on news.ycombinator.com