Comment by amgreg | Hacker News Reader

Comment by amgreg | Hacker News Reader

The author explains the following, regarding Vitamin D studies:

> Most studies follow this pattern: Two sets of people are evaluated. One set has a certain disease (diabetes, for example). The other set does not have the disease. Vitamin D levels are measured in both groups. Vitamin D deficiency is found to be much more common in the group of diseased individuals.

If this is true, I wonder if most studies aren’t falling short for failing to control for Vitamin D deficiency. Couldn’t the studies be structured differently?

Let’s say we want to know the effects of Vitamin D on Covid. What if, instead of measuring Vitamin D deficiency in a group with the malady and a group without (analogous to what the author suggests most studies do):

We had two groups made up of Vitamin D deficient people. We gave the first group a Vitamin D supplement and the other a placebo. We then observed both groups out in the wild (ideally a place with a high R), measuring for infections. If the supplement folks were infected at significantly lower rates than placebo folks, wouldn’t this be better at demonstrating causation?

A quasi-fix could be to control for the most obvious sources of vitamin D: how often do you go outside, how long do you go outside, do you take vitamin D supplements, how often and with which strength. You can do this kind of analysis alongside any type of study, don't need full coverage and get some preliminary results. All you need is a Bayesian on your team (: partially in jest) or the willingness to open up your data after you publish.

In my opinion as someone who practices non-medical statistics the trope of correlation vs. causation and interventional vs. observational are somewhat overblown. Andrew Gelman posted somewhat snarky about MDs and statistics recently [1]. I see that this post is in (very) good faith, but it does have the MD-bias to statistics in it.

[1] https://statmodeling.stat.columbia.edu/2020/08/11/that-not-a...

I'm the one who wrote the blog, and really appreciate your comment. As a physician, I can confirm that most physicians have only the tiniest grasp on statistics - and most will not have even heard about Bayesian stats. I do not want to imply that observational studies of correlation are useless - or even that observational studies are "less valid" than randomized controlled trials (something that doctors usually assume, but it incorrect). I actually want to write about this issue in future posts. There is a lot I want to cover, and stats in medicine is one of my favorite topics. One blog at a time, though.

I'm looking forward to future posts as well! Thanks for writing this. You gave me pause to reconsider the only supplement I give my children and perhaps even more so in future episodes.

You’re describing observational vs. interventional studies.

Such Interventional studies can be done, but would be more difficult and expensive, and might be difficult to get an answer from.

To your point, a confounding variable could be that diseased people go outside less (and therefore have less vitamin D)

this!

In slovenia, most of the deaths are in old-age/nursing homes, where most of the patients have severe health issues and a lot of them are bed-bound. Yes, they have lower vitamin D, but also many other health problem, so looking at a single variable (vitamin D) proves corellation but can't prove any causation without interfering (eg. giving half of them vitamin D supplements, to compare to the other half).

Great point. Low Vitamin D is common in very sick people, partly because they are not as active, not going outside, and may not be eating the best diets. There are trials like you describe, which I'm hoping to get to in part 2... or 3? There is a ton of information to cover!

The quote you cite is interesting if not a little worrisome.

Maybe its just acting as a first-filter of things to (not) investigate at more expense later?

Otherwise you'd get a ton of studies promoting solutions as cause-and-effect when they're only coincidentally related:

- U.S. Spending on science correlates @99.8% with suicide by hanging

- Drownings correlates with Nic Cage films @66%

- Japanese passenger cars sold in US correlates with suicide by motor-vehicle @ 93.57%

*source: https://tylervigen.com/spurious-correlations

I love this Spurious Correlations site. And you are correct - a correlation can be a first-filter, especially if you have some reason to believe that there would be a causative connection. And though "correlation does not equal causation" is a cliche, it is still so easy to get sucked into equating the two. Part of the human brain's desire to make sense of things.

Correlation does not equal causation, but it correlates with it.

> failing to control for Vitamin D deficiency

Do you mean failing to control for the disease causing vitamin D deficiency? Because it seems that's the issue you're addressing with your proposed study design.

As a very specific example I think you're looking for this vit D study.

https://pubmed.ncbi.nlm.nih.gov/23032549/

As a very general example I think you'd have a lot of fun looking at examine.com pages that link to medical studies.

https://examine.com/supplements/vitamin-d/

Two axis of engineering problem definition not handled by the article are definition of normal and lifestyle variation.

The concept of normal blood chemistry levels is vary vague. Certainly for mariners over 200 years ago the level of vitamin C was always normally very low. Running a nutrition program for Columbus-era mariners to optimize their diet to produce the most numerically average possible vit-C level would be possible but would not be healthy at all. There's a giant subculture of both doctors and average people running all kinds of semi-long term experiments on diet and health. Another side dish is the definition of normal for diet, some consider paleo to be normal diet and some consider twinkies and hot pockets to be normal diet. And of course that "normal" diet interacts in peculiar combination with "normal" concept of blood chemistry making it a very complicated problem.

The lifestyle issue is an interesting problem. Due to local weather I don't go outside and due to exercise hobbies I'm extremely large from weight lifting for many years, also I sweat out a ton of electrolytes and presumably water soluble vitamins every other day. A stereotypical elderly petite sedentary zero-exercise desk worker would likely turn into a pillar of salt if they consumed the same salts I require to prevent muscle cramps, but perhaps this theoretical person is old enough to still falsely consider sun tanning as a healthy activity. Meanwhile our evolutionary ancestors evolved to work as physically hard as me yet do it all out in the sunlight naked. Given this incredible diversity in lifestyle, you'd think we'd all take different vitamin/supplements much like we drink varying amounts of water, perhaps a 5 to 1 ratio of individual variation if not more, but instead the bottle of vit D pills in front of me claims we should all take 125 mcg aka 5000 IU which is 625% of the normal RDA. That seems highly unrealistic.

I'm sure there's potential startup ideas to both gather data and analyze the data. Rather than running a trial on 322 people and hoping its good enough, you could gather less accurate data from 1e6 people, maybe 1e7 people, using some kind of cloudy app statistical sampling thingie.

Reply on news.ycombinator.com