Hacker News Reader

Comment by johnday | Hacker News Reader

n=17, so read nothing into this result.

Also, what they tested is having no fixed eating schedule. The effects (if any) of skipping breakfast every day consistently are not measured.

View on HN

The problem is not N=17. If the effect (difference between skipping breakfast vs dinner) was large enough, it would be obvious and significant even in a small sample. The effect seems to be very small.

I agree that what is missing is to measure skipping meals for a longer time period, like a month, at least.

I have an identical twin brother; I began IF (eating only 1/3 of target calories on Mon/Fri) in 2012. We do functional similar levels of cardio, weight-training; we're both programmers, with similar hours.

Right now, I'm 1/2 taller, ~70 lbs lighter, I have ~10 lbs more LBM, blood pressure and blood work is spot on. He's got high blood pressure, and is diabetes adjacent.

I have a good friend who's an identical twin who did CFR (over 30 years, now); similar situation.

Here's the summary: getting fat is bad; losing fat is bad; being fat is deadly.

Why is losing fat bad?

Losing fat requires calorie restriction; calorie restriction will lead to loss of both fat and lean-body-mass. When you stop your calorie restriction, you gain back fat much more easily than lean-body-mass (this is a result of the Milwaukee starvation study). Lean-body-mass includes things like muscle, bone, and other things you'd rather keep. Furthermore, it can cause significant stress on other tissues.

The healthiest thing to do is just never be fat in the first place.

But the entire field of medicine is based on small studies with high power to achieve statistical significance, that lead to sequentially larger studies and eventually regulator approval or rejection. Are you saying the entire field is a sham? I urge you to write to the FDA immediately them to inform them that their methodology is gravely flawed.

I am saying that lay readers on Hacker News should not draw conclusions from this study on 17 people. Do you disagree?

You shouldn't draw conclusions from this small study but it increases the probability of one view being correct

Yes. The entire field is a sham.

I agree that we shouldn't read much into this. But I do suspect that, if an n=17 study came to the opposite conclusions, supporters of intermittent fasting would never stop citing it.

I doubt so, since the study is not about intermittent fasting, but intermittent meal skipping at best :-).

I think that skipping a meal every once in a while, and regularly skipping meals according to a specific pattern for one, two or ten years, are completely different activities with very different outcomes

i don't get why some studies have so few partipants. How hard is it to recruit people for something like this? We're not talking rare diseases.

From what I have read in the past this is just how the funding tends to work. You write a grant proposal for a study with 20 people, get funding, get results, use that to get funding for another study with 200, then 2000, etc.

Frequently they don’t manage to make it past the first couple levels because the results aren’t interesting.

Any study less than 200 people is essentially false advertising unless the population size is incredibly small (e.g., rare diseases) or the intervention has a crazy high prior (e.g., does getting shot in the chest lead to blood loss leading to passing out) so I consider these grants completely counter productive.

There is a real benefit in having fewer pieces of junk to sort out and so many of these medical research studies are just noise that well meaning members of the public cite and adopt into practice because they don't know any better.

It's the modern day version of leeches, essentially.

Rather than “well meaning members of the public”, I think the larger problem is with health and science reporting which frequently reports findings of very small studies as if they are definitive fact, with very prominent and provocative headlines.

> health and science reporting

Is part of "well meaning members of the public" just as "YouTube commentary" is and "random celebrity that Googles something and repeats it on Twitter."

And quite frankly, so are scientists themselves. I have met far, FAR too many scientists that do not know the very first thing about statistics and yet feel competent to reference other people's research.

This is why I say it's a big pile of leeches for more than half this stuff. Sure the incentives are misaligned, sure its reporting, sure this and that. But what we really need here are people to say "no you idiot leeches don't do dittly squat to fix this problem."

All that screening and testing costs a lot of money.

Nutritional studies are notoriously difficult to conduct. You either have to isolate subjects and monitor everything they eat and when. Or you have to rely on self reported data and people do an atrocious job of tracking food, estimating portion size, etc.

This study did the former:

> The duration of the study is 7 days including 3 days with controlled diet and 4 days (including 5 nights) in a metabolic chamber at the Institute of Nutritional Medicine at the University of Hohenheim.

I imagine it was prohibitively expensive to have more participants.

Try plinking around on a sample size calculator a bit, the results might be surprising. For example:

https://www.surveymonkey.com/mp/sample-size-calculator/

If you are ok with a 25% margin of error and a 95% confidence level, 16 people is good enough. You have to make sure there isn’t some skew in your population, but if your method of sampling the population produces a skew, adding people won’t really help there.

If it's clearly not worth HN readers' time, how did it get published in "The American Journal of Clinical Nutrition"? Or am I just naive in thinking published in a Journal still means something? Why did it pass peer review?

It's a scientific journal - a place for the researchers to discuss and discover science.

This is a lay forum - a place more suited to the discussion of decided science.

Yes exactly. If your average HN user can see sample size is too small, Journal reviewers should be able to. Your not going to discover or decide any science if your sample size is too small.

> If it's clearly not worth HN readers' time, how did it get published in "The American Journal of Clinical Nutrition"?

This comment is extremely funny.

I heard a rumour that the American Journal of Clinical Nutrition might be aimed at scientists working on clinical nutrition, who could make meaningful use of this result - and not the readers of Hacker News.

That would be a strange rumour hear.

What I'm saying is, if some rando on HN can see see that the trial size is too small to be statistically significant how did it pass peer review?

The trial size is not too small to be statistically significant. It may warrant further investigation by scientists in the field of clinical nutrition.

However, it is too small and insignificant for Hacker News readers to make meaningful use of, and it would be incorrect for them to read anything into the results presented. That is the difference.

So it's a practically baseless conjecture within the field of clinical nutrition, clinical nutritionists now publish such in peer reviewed journals, and the wider scientifically literate public have no business ease dropping on all this. Got it.

If we give substance X to 17 people and 100% of them die, I would recommend against taking substance X.

Of course, it's not as black and white here, but sample size and effect size should be considered in relation to each other. Previous research and theoretical expectations should also play a leading role.

In this case, previous research and theoretical expectations would indicate only a small effect, if any. This is covered in the introduction to the paper, which I'm sure you have read.

These comments are always so hilarious. It demonstrates a significant lack of understand of how statistics actually works. You see this on reddit all the time, you'd think it'd be better here.

Power and sample size are determined by numerous factors depending on the question under study. You could have N=10 be statistically powerful and N=1,000,000 be statistically meaningless. It depends ENTIRELY on the subject under study. More is not always better and in many cases completely unnecessary.

https://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Power/B...

Here is a decent guide. You may wish to read it so you can learn when you can dismiss a study based on N. You never mentioned, did you back out the study numbers and determine the N=17 number was outside the range of statistical power? If so, would you mind posting your calculations?

This is clinical nutrition. There are so many confounding factors that it is not reasonable to draw any conclusions about how the body works (especially as a layman reader on HN) from a study that tested 17 people. Indeed, their own introduction claims that previous larger studies have found no correlation between meal plans and effects on the body, at least wrt caloric expenditure.

> These comments are always so hilarious. It demonstrates a significant lack of understand of how statistics actually works.

I do have a PhD, but thank you for your input anyway.

Would this be an intuitive way of putting it? If I pull a random 17 phones off an assembly line, and 16 of them are defective, I can pretty confidently say there is something wrong with the whole product line even though n=17?

Reply on news.ycombinator.com