Hacker News Reader

Comment by coldtea | Hacker News Reader

And this comment is a prime example of what's wrong with internet comments on scientific research.

Usually all the basic concerns have been accounted for in the actual study.

Whether an article in the mainstream press conveys that or not, is not important, since the article is just supposed to give a summary of the findings.

It's seldom an issue of "those stupid PhDs and experts in their field, they didn't think of a confounder a random internet person can come up with in 5 seconds".

Whoever wants to delve in can read the actual reports.

And whoever wants to start the same practice (e.g. sauna here) would be foolish to do it based on one article anyway, they should consult with their doctor, and/or wait until a fuller consensus has been reached after several meta-studies (which can take decades).

View on HN

The parent comment is a valid criticism of the quality of reporting, especially in the context of the overall standard right now.

Claiming a causal relationship is an incorrect summary if the study did not show this. This is a very serious problem. If the particular study did in fact show good evidence of causality, that's worth specific highlighting because it's not that common.

The parent is not saying they've thought of some obvious confounder that the study authors didn't. They've emphasized the importance of unknown unknowns and correlation vs causation.

Raising the standard of science journalism is a noble goal, if you ask me. In this case, the authors are not making as strong a claim as the summary does.

> Usually all the basic concerns have been accounted for in the actual study.

Often. Yes. But I often go try to find the original publications when a study interests me, and I have yet to, once, find one where the causal claims made in the reporting article are actually supported by the research. Often the original paper explicitly mentions the fact that the claimed causal relationship can't be inferred.

The claim here isn't that the science is bad. No one's saying the PhDs are stupid (there may be an argument there, but that's not what anyone here is saying). The complaint is that the science reporting is terrible.

And it is.

> And whoever wants to start the same practice (e.g. sauna here) would be foolish to do it based on one article anyway, they should consult with their doctor

You are suffering under a severe delusion if you think the average doctor has the interest or competence to provide useful advice on this sort of issue. The average doctor will not have read any relevant studies, and has no interest in being informed about them, and I can say with some confidence that having a medical degree does not give him a magical ability to judge the usefulness of a practice without consulting any evidence.

Science is great, but science reporting is terrible, and for that matter unreasoned faith in the academic establishment is anathema to real science.

That's far too optimistic. I can't find the full text of this dementia study, but the one where they concluded sauna improves cardiovascular health barely gives any thought to the idea that perhaps men with a worse subjective assessment of their health are more likely to find it unwise to go to sauna too frequently. (In the closing paragraph they do a disclaimer "It is possible that underlying diagnosed or undiagnosed diseases may affect saunabathing habits, but our subgroup analyses according to various clinical characteristics were consistent.")

>Usually all the basic concerns have been accounted for in the actual study.

No, I don't think so: http://www.economist.com/news/leaders/21588069-scientific-re...

THe summary is actually wrong or at least severely premature, if it doesn't tell you whether this was about correlation without any real proof or analysis of causation.

> Usually all the basic concerns have been accounted for in the actual study.

No. It's an observational study. If they'd account for the basic concerns they'd do an RCT. Yes, that's more work and takes longer.

> It's seldom an issue of "those stupid PhDs and experts in their field, they didn't think of a confounder a random internet person can come up with in 5 seconds".

No, you don't understand the problem. Confounders everyone can come up with can be controlled for (although it gets complicated the more you consider). But the major problem are unknown confounders. You can't control for things you don't know.

"Observational study" does not mean "bad science". There are good and bad observational studies, and without reading the paper, you can't know which one this is. Moreover, your assertion that only random trials can be controlled is just completely wrong - there are plenty of ways to do controlled studies on observational data. You're engaged in the scientific version of the middlebrow dismissal.

In health sciences, a lot of good work starts as well-controlled observational science. In fact, for a question like "does sauna use lead to a reduced risk of dementia over a human lifetime?", pretty much the only way you're ever getting the funding to do an "RCT" is to start with lots of observational evidence.

Yes, observational study does not mean bad science. But claiming or inferring that causation was found when it was not, that's bad science reporting.

Never said anything about the reporting. The headline is certainly bad. But a bad article doesn't mean bad science.

"Usually all the basic concerns have been accounted for in the actual study."

Fair enough ... but there could be simple and nuanced elements that they didn't or couldn't measure.

For example: maybe Sauna dwellers are far more socially active than others? Social interaction is maybe they driver of longevity, the sauna may simply be a part of that process.

Anyhow, just a crude example ... but I'd suggest it's very hard for researchers to rule out most such things.

Reply on news.ycombinator.com