Comment by timr | Hacker News Reader

Comment by timr | Hacker News Reader

Parent Full thread

> Evidence-based medicine, more than the indeed important issues raised in the article, is mostly impaired by the absolutely abysmal average quality of clinical research papers. The absolutism of higher-ups combined with their unrivalled (in the scientific world) statistical ignorance is hurting science more than anything else.

I couldn't agree more. Throughout the pandemic there has been an infuriating pattern of terrible research coming from medical doctors, followed by dismissal of any/all critiques by people much more qualified to make these critiques, because "they're not doctors".

The average medical doctor has next to zero ability to evaluate a study, and is easily fooled by elaborate statistical techniques into concluding that up is down and left is right. Meanwhile, scientists and statisticians with enough experience to see the methological flaws are dismissed, and policy makers turn to MDs -- who are often wildly out of their depth -- for pronouncements on "science".

To do high-quality research you need science/statistic knowledge, and domain knowledge. No matter how good a scientist you are, data is really hard to interpret correctly without domain-knowledge

Yep. But all the domain knowledge in the world doesn't help you when you mis-apply a statistical analysis, use a bad experimental design, or don't properly control for confounders. These are most often the limiting factors in medical research.

Regardless, the bigger problem comes when we're told that we can't possibly be seeing that fatal flaw, because we don't have an MD.

Absolutely agree. Data do not have conscience. We cannot apply absolute data collectively to all individual human beings. Which is especially happening over two years now.

> The average medical doctor has next to zero ability to evaluate a study

Yes, but...

> dismissal of any/all critiques by people much more qualified to make these critiques, because "they're not doctors"

... unfortunately, no. While it's true that most MD clinical researchers are depressingly bad, it is also true that pure scientists who never set foot in the clinics are very limited when evaluating practical aspects. So the situation cannot be trivially solved by allowing scientists to do more. In my experience, the problem has no real solution right now and I expect we'll have to wait on technological progress to allow improvements.

Yeah, this is the really hard part about medical research. At the individual level for those who are really qualified to do it, the incentives lean in favor of practice, where your comp is 4x-10x higher

Exactly, yes, a hundred times yes! This is the major flaw in our system. But I don't see a way of rebalancing to saner priorities. And apparently, neither do our supreme leaders.

Don't get me started on the loans either -- sometimes those incentives aren't just incentives, but necessary outcomes given your financial state exiting med school. I know folks who emerged from residency (where they moonlighted too!) with 500 credit scores and debt that can be described as multiple new Ferraris. They'd laugh at the idea of anything but practice

> it is also true that pure scientists who never set foot in the clinics are very limited when evaluating practical aspects.

Yes, but most medical/epidemiological papers fall over under the lightest methodological scrutiny. Even advanced scientific concepts like "have a control group" is seemingly beyond the analytical capacity of a fair number of MDs. When you get into things like "comparing small effect sizes with confidence intervals", they're positively adrift. We really are talking about table stakes, here.

I'd love to live in a world where MDs were sufficiently knowledgable of experimental design so that domain expertise is the limiting factor, but I can count the number of such papers I've read in the last two years on one hand.

I could write a much longer post on the arrogance of researchers when interpreting medical data or extrapolating lab results into humans...but that's a different subject.

Any books so I can be better interpreting paper? Thanks

How about an example, please.

Reply on news.ycombinator.com