Comment by joshgel | Hacker News Reader

Comment by joshgel | Hacker News Reader

Parent Full thread

> I understand that it's difficult to control for various factors when doing large scale studies of health and behavior, but my perception as a layman is that there is some willful ignorance or absence of rigor.

I perhaps understand the sentiment behind this, but if this is really what you think then probably you don't really 'understand' how difficult it is to do large scale studies of health and behavior. As a thought experiment, pretend you are the researcher studying a glass of wine a night and write the script you would pitch to potential research subjects. You have to try to convince them to be randomized to either completely abstain from alcohol for a period of 10-20 years OR to drink 1 glass of wine a night for that same period. Do you think you could get 10,000-50,000 people to participate and that those people would reliably stick to their randomized groups? How would you monitor compliance? How will you keep track of those people for 30-50 years to see differences in mortality or other outcomes?

The 'answer' may be here that it's simply best to acknowledge that we cannot realistically study some, and arguably many, things. I expect your response would be that at least trying to do the best with the resources we have is better than remaining blissfully ignorant. But the problem is that coming to wrong conclusions far more dangerous than simply acknowledging we do not yet have the resources to come to any conclusion. And science is built upon the shoulder of giants. But what happens when the shoulders you've built upon collapse? One mistake decades ago can end up destroying decades of further work that was built upon it.

Science is never 100% on practically anything. Even in physics there are keystone works that are mutually incompatible indicating errors in one or both. But there is a difference between models that are based on predictiveness, direct experimentation, and falsifiability - and models that are based on correlation. Newtonian mechanics is a good example. The equations that determine a body's position given a set of inputs and time could be shown to be 100% correct in every single scenario. Well at least until Mercury came along with it's 1/100degree per century orbital aberration. And that 1/100th aberration per century of a single body was enough to inform that something was wrong. With correlational studies, actual results don't really matter and entire views become practically impossible to falsify. And when they do provide predictions, rarely is it the case that the failure of that prediction to come true would falsify the study. That's just not science by any stretch of the imagination. It is the difference between astronomy and astrology.

I mostly agree with this. But part of it is scientific illiteracy on behalf of most people, who want these 100% answers, not grades of evidence and certainty. An observation study is definitely helpful, but should be taken as what it is, an observational study, not truth, which is often how it is reported and understood.

But I don't think that coming to wrong conclusions is necessarily 'far more dangerous'. People who drink 1 drink a night are not likely to have dramatically different outcomes than people who abstain. (perhaps unless it contributes to binge drinking, but science already says that is bad)

I guess what I mean is that the results of these types of studies are useful, but only in pointing where to look for a more direct mechanism. In other words, we found a correlation, now we can look more closely at the elements involved to find causality. Mostly it seems like the correlation is accepted and then untested theory is exposed as fact to explain it.

Reply on news.ycombinator.com