Hacker News Reader

Comment by cynicalpeace | Hacker News Reader

40-60% of nutritional studies cannot be replicated.

You can't reliably draw any conclusions from them. You have to use common sense and rules of thumb.

But some are better than others. The NIH is currently running a study (N=36, expected to complete in 2025) on ultra-processed foods where the participants are sequested as inpatients at the National Institutes of Health Clinical Center's research facility and strictly monitored 24/7. They can't leave without a chaperone that ensures they're not cheating. They've done prior studies such as this one [2] (N=20) in 2019. In these studies, they switch the person's diet halfway through, in order to see if the effect is real. The participants were allowed to eat as much as they wanted, but the diets had the same amount of calories total, and the same calorie density. The results are striking; participants eating ultra-processed foods consumed more calories and gained weight while the other group lost weight. [1] https://www.nytimes.com/2024/07/30/well/eat/ultraprocessed-f... [2] https://pubmed.ncbi.nlm.nih.gov/31105044/

Even though I'm a scientist and thoroughly trained in statistics, the idea that we can to sequester 36 people and monitor diet 24/7 and made general conclusions doesn't sound completely right to me. Partly in the technical sense and partly in the "why do the folks working on human health get away with sample sizes that would be laughed about in any other field?"

I don't know enough about medical statistics to say, but I often see small sample sizes in studies where the effect size is expected to be high. That may be the case here.

the effect size will not be high

There are too many variables in diet. If they study steak every meal vs rice and beans every meal they can come up with one. However most people are not that one-tracked either way. Sometimes the rich eat rice and beans, sometimes even poor manage to afford steak. For steak, did I mean beef, lamb, goat, pork... - this might or might not matter. There is also chicken, turkey, snake, deer, elk - and dozens more animals people eat which might or might not be healthy. OF each of the above there are different cuts (does it matter?), different fat levels (does it matter?). And that is just meat, how many varieties of beans are there, what about rice? What about all the other things people eat?

Do these other fields also study humans in controlled experiments?

I think it has to do with the sample to staff ratio. It's not enough to observe human subjects. You have to actively prevent them from going off the rails. It doesn't scale well when you increase the sample size. I guess we could replicate a similar experiment n-times and then do a meta study, but it's not ideal either.

How would you tackle the logistics of scaling up the above experiment?

Yes, the most common example would be clinical trials for drugs and other medical treatments- often have thousands of patients (with recruitment being the limiting factor). There are tons of ways that studies can go wrong, for example when patients don't take the treatment and lie (this is common) or have other lifestyle factors that influence the results, which can't be easily smoothed out with slightly larger N.

I don't know how to fix the nutritionist studies- I'm still pretty skeptical that you could ever control enough variables to make any sort of conclusion around things with tiny effect sizes. This isn't like nutritional diseases we've seen in the past, for example if you look at a disease like pellagra (not getting enough niacin), literally tens of thousands of people died over a few years (beri beri, rickets, scurvy are three other examples; these discoveries were tightly coupled to the discovery of essential nutrients, now called vitamins).

From my reading, that's not generally true. It all depends on the methodology. Safety or feasibility studies can use very small sample sizes. I've been reading safety studies on monoclonal antibodies like Cimzia, for example:

https://pubmed.ncbi.nlm.nih.gov/29030361/ (N=16)

https://pubmed.ncbi.nlm.nih.gov/28814432/ (N=17)

https://ard.bmj.com/content/83/Suppl_1/1145.2 (N=21)

Of course, these are not nutritional studies.

You should try reading the FDA approval for the drug; it was already approved before these publications (which aren't so much clinical trials as just medical research). The FDA approval has a whole paragraph about how the effect size was too small to demonstrate statistical significance, and the trials had n=300.

It's also indicated for use in a disease we don't understand, for people who didn't respond to all the previously approved drugs. Not a good example at all.

I'm not comparing these to the FDA approvals process, but to your claim that trials use thousands of patients. These three studies are ascertaining the pregnancy safety of a drug, irrespective of whether we understand the disease or what the response rate is.

Cimzia has been well-studied, and we understand why it works on autoimmune diseases like inflammatory arthritis. It has 6 FDA approvals for different indications, so your description of the drug itself is incorrect.

The sample size doesn't concern me as much as what does that force on their lifestyle and in turn do they apply. They probably are not getting the same exercise as a normal person (which runs the range from "gym rat" who gets too much to "couch potato" who barely walks).

> why do the folks working on human health get away with sample sizes that would be laughed about in any other field?

Because it's really hard and expensive to do such studies with more participants

>The participants were allowed to eat as much as they wanted, but the diets had the same amount of calories total, and the same calorie density. The results are striking; participants eating ultra-processed foods consumed more calories and gained weight while the other group lost weight. [1]

Did they eat the same amount of calories or more calories? People will eat more of tasty food and less of bland food. You could get the inverse result by giving bland ultra-processed food and tasty unprocessed food.

"use common sense" lol - the same common sense that people use when confronted with dyhydrogenoxide? the same common sense that people used if asked about sodium chloride? The same common sense about that tomato, mushroom, seaweed extract called MSG?

Can you please give your reference for that definitive statement. And what are 'nutritional studies'? Why wouldn't they include the research that led to the list of nutrient recommendations issued by USDA and similar publications in the UK, Norway, France, Australia and many other countries. No conclusions from them? I think there are. There is a truly vast literature on subjects nutritional so it's vital to be very specific.

Separately, when using the term 'ultraprocessed' we should be precise about the processes used. There are many different ones with undoubtedly different effects to different degrees on the nutrients therein.

The RDA and nutrient recommendations are the bare minimum so you do not die. Vast literature is ad populum fallacy.

Also consider that genetic background matters in nutritional matters and well... The populations under study have changed, and that's assuming you have a fairly similar background to a population and not very mixed.

And we are not even getting into how these things go down in practice, with heavy industry lobbying and what not...

TLDR, you are on your own in terms of optimal nutrition but as another commenter said "eat food, not too much, mostly plants"

They are not, they are recommended averages.

RDA for vitamin C is 60mg, but you can survive without getting sick on 5-15.

Potato potato. Familiarize yourself with their history and then your opinion might differ

Eat food, not too much, mostly plants.

My favorite plant is wheat, deep friend in peanut oil, covered in corn(syrup).

is the last one even widely accepted anymore

What's not widely accepted?

https://nutritionsource.hsph.harvard.edu/healthy-eating-plat...

That's pretty good. Though it should be balanced with avoiding factory made food.

I personally think non processed meat is good for you, but that's a minor point compared to ultra processed vs not really processed foods.

I once argued with med students that Oreos (which are vegan) are not healthier than a steak.

Absolutely crazy and tbh frightening that anyone (let alone med students!) would think Oreos are healthier than a steak.

The reasoning of course is that processing and sugar content don't matter as much as any level of saturated fat.

The avoidance of factory food is part of the point. GP is invoking Michael Pollan from the Omnivore's Dillema, among others. By 'Food', Pollan specifically means to exclude the sorts of chemical-engineered vague nutrient simacrula you're talking about.

Eating most plant starches would be a terrible diet.

It's saying most of your food should be plants, not that most plants should be your food.

(Michael Pollan) https://www.goodreads.com/quotes/18508-eat-food-not-too-much...

40-60% - that's a pretty large p-value and reasonable proxy for thumbs and sense.

Reply on news.ycombinator.com