Hacker News Reader

Comment by general_ai | Hacker News Reader

Great entertainment for anyone with basic knowledge of statistics: read medical papers. Lots of laughs to be had. Most experiments are bogus. Most conclusions are drawn with little in the way of supporting evidence, often without controlling for obvious confounding factors. Correlation is implied to mean causation. Extrapolations abound, both for concentrations (i.e. if I give this mouse a megadose of X it'll die, hence X is harmful) and between species (mice grow tumors if I give them a ton of X, so humans will too if they consume a tiny fraction of the amount per unit of body weight). Most of it is basically bro science, except expressed in much longer words.

View on HN

I disagree. This is a bit like browsing open source repos on github and laughing because of bad algorithms, issues or quick fixes.

There is indeed bad medical research. Typically, animal research that focuses on just one marker and then extrapolates on what it means for humans. You should not rely on these. At best, they give you cues on what might work for humans. But there are multiple studies that reviews all medical research. They study 5 million people over 20 years, follow their health as well as their habits. Variables are adjusted. In such studies, if you discover that green tea decreases risk of type 2 diabetes, then evidence is high. Much, much higher than the simple animal research where mice were given green tea.

The biggest problem is not medical research. It's the army of bloggers, writers from NYTimes and overall the general population hungry for the latest superfood or cure of cancer. They think that throwing billions of dollars will find a revolutionary food, herb or medicine for cancer. So medical researchers present their results in a way that's pleasing. And then most bloggers are just too guilible. That's amplificated by social media, or even boards like Hacker News (see the upvotes for effect of coffee on mortality yesterday. That study is non-conclusive but the story got hundreds of upvotes. People are just too happy to discover that their daily drink will extend their lifespan, even though evidence is limited)

> and between species (mice grow tumors if I give them a ton of X, so humans will too if they consume a tiny fraction of the amount per unit of body weight).

Not only that, but to add to the injury using mice or rats which are known to be sensitive to growing tumors due to factor X (which is henceforth completely ignored, and fails to be mentioned).

Sad, very sad. I wasted a lot of energy & focus in my life on this.

Sadly, this applies beyond just medical research. Most of the biology papers we discussed in Journal Club (a weekly meeting of grad students who go over a single paper in detail) turned out to have at least one invalidating error (one that makes the conclusion unsupported).

As for dose relationships- a lot of drug dose responses are linear over a range. But they tend to not be linear outside that range; the error is in not correcting for low-dose effects.

Just to put your rant on more sound statistical footing, there is the whole Replication Crisis issue:

https://en.wikipedia.org/wiki/Replication_crisis

Holy cow! I've been looking for a 'clearinghouse' site that addresses all the issues swiftly and clearly. It's obvious in retrospect, but I never though Wikipedia would be that place. Thank you!

Actually sourcing the data via PubMed has been eye opening for me, and I saw all those things. I am amazed by the number of news articles that cite conclusions as factual for humans, but on reading the source, they have only been shown in mice studies.

And quite often they're obviously bogus even in mice, especially if the author seems to have a political axe to grind (e.g. anti-GMO, anti-sweetener, anti-herbicide, etc).

Extrapolations

Technically, going from "megadose is lethal" to "smaller doses are harmful" is an interpolation, not an extrapolation.

Only if an even smaller dose was tested too.

Well, technically anyone with a pet mouse is doing the experiment "Does a dose of ~0ppm of cyanide/arsenic/etc. kill mice?" :)

That is called a control group. You need to actually have a formal one and big enough too.

It has to be kept in as similar conditions as the group under treatment and have genetically similar makeup.

Most importantly, the composition of both groups and the conditions the experiment is ran under should be rigorously described.

It does, but it seems to be very slow acting.

The dose is the poison.

There are infinite interpolation possibilities, one decides it is linear by extrapolation.

Don't forget all the BS drawn from epidemiological "research". We ask 10,000 people how many cheeseburgers, eggs, tomatoes, etc. they eat in a week, at the start of the study, extrapolate that out for 10 years, see how many died, break the participants into arbitrary quartiles, and report everything as relative risk to pump the headlines: "Eating 2 cheeseburgers a week increases your risk of death by 75%"

None of this research passes even the most basic statistical sniff-test, and even if it did, the hazard ratios are so small as to be easily influenced by noise or confounders.

My favorite, especially because this ends up being reported in the media as infallible truths.

I mean, medicine is not science, it is an applied science. Applying biology, chemistry, and physics to the attempted betterment of health. Akin to the difference between physics and engineering.

It would be nice to see a meta-study on this.

I agree. But peeing in other people's cereal is a perilous endeavor.

Cochrane reviews frequntly say "the available evidence is poor, more research is needed", and "we found X relavent papers, but had to exclude Y which were low quality."

Reply on news.ycombinator.com