Hacker News Reader

Comment by tomerico | Hacker News Reader

Why do you say that the results aren't impressive? 49% reduction in all cause mortality when High intensity interval training is compared to moderate intensity continuous training seems like a very strong result.

View on HN

Mostly because they admit those results are not statistically significant (for example, at the bottom of the summary diagram). That said, I admire their honesty, and I hope these results suggest other kinds of experiments to other researchers. Maybe the study just needs to be larger, or longer.

Focusing on specific kinds of mortality might also give stronger results than measuring "all cause" mortality. I say that because it seems like cancer was the biggest killer in these groups. I am not a doctor, but I wouldn't have thought cancer was causally related to (lack of) exercise, the way cardiovascular diseases are believed to be. I'd be interested to see larger studies with enough non-cancer deaths to say something statistically significant about the effect of exercise on those outcomes.

Why even bother doing a study with a sample size so small that a difference as large as 49% is still not significant?

They had 1567 participants. Compared to control (HIIT-like), HIIT reduced risk by 1.7% while moderate intensity training increased it by 1.2%. The overall mortality in the control group was 4.7%. For a controlled trial, that seems like a large number of participants (correct me if I'm wrong), they just were too healthy ;-)

I assume the significance problem is inherent to studies looking at mortality as the outcome is very binary and can take a long time to manifest. As an effect on all-cause mortality can be seen as the ultimate metric of how healthy something is, it's probably still worth it to investigate. In this case, they made it quite difficult for themselves by comparing active people to other active people.

It's also possible that they made other observations during the study that are or will be published separately.

Also, just adding some sugar to this, don't forget that this study was in Norway. Overall health levels would be higher than in, say, USA. I'm talking about weight specifically.

Assuming healthy / not overweight would be a very big assumption to make for the US (as a Norwegian I went to the the USA, once, in 1999, and what I saw there shocked me), where another study could be to monitor mortality outcome of getting weight under control and exercising vs not.

I'd also love to see impact of weight resistance training added to the mix surveyed.

And not just Norway, but specifically Trondheim. Correct me if I am wrong, but I expect their lifestyle to be even less sedentary than the average Oslo resident. I expect the results to have slim pickings (pun intended)!

Roger that. But even by e.g. Australian standards (which are a lot closer to the US ones), Oslo is far from sedentary; I didn't even think about buying a car in Norway before I left it, being happy to walk/run, ride or PT it everywhere. Bad diet, alcohol use etc is a lot more common than in Norway too.

HIIT reduced risk by 1.7 percentage points, mortality down to 3% for the high intensity group, from 4.7% for the control group.

The percentage decrease was 37%: Percentage points are not percent!

I read that wrong. I thought that the overall mortality rate in the groups was 1.2% and 1.7%. Didn't realize those were both differences from the control.

Because running a large enough study is expensive and it's cost-effective to perform a smaller study to determine an approximate size of the effect. Then, you can design the next study to have enough power to distinguish the expected effect size from the null hypothesis.

Additionally, these results could be aggregated with comparable results to yield a stronger result.

"an absolute risk reduction of 1.7 percentage points was observed after HIIT (hazard ratio 0.63, 95% confidence interval 0.33 to 1.20) and an absolute increased risk of 1.2 percentage points after MICT (1.24, 0.73 to 2.10)."

So, 1.7% vs 1.2% seems just above noise level. The OP is right, the numbers are not that impressive.

We are not talking about reducing overall mortality by double digits...

Summary: looks like exercising is good, and adding HIIT has an increasing effect of reducing mortality, but the overall effect is small.

Those are absolute risk reductions, not relative ones. The 1.7 percentage point reduction equates to a 37% reduction in all cause mortality, and a 1.2 percentage point increase is a 25% increase.

Those are huge numbers. The problem is that the confidence interval is really wide.

>but the overall effect is small.

quibble: non significant, wide confidence doesn't mean "small", it means _unknown_. It means the data is too sparse and/or too noisy to tell.

Doesn't this also imply that the effect is "small" even if real? After all if there would be a very strong correlation (eg a true and exclusive casual chain) then we would see a huge signal even in small and noisy datasets. Or am I missing something?

No. If you look closely at the data you might be able to draw such conclusions but lack of statistical significance often doesn't suggest or imply a small effect. Notice that in this case in particular, the confidence intervals are consistent with very large positive or fairly large negative effects. Don't underestimate the amount of noise often found in studies. Lack of significance usually just means that data is too noisy to tell us anything. If you get significance you get to say: the data is probably not pure noise but the effect could still be very tiny or caused by systematic measurement errors. Null hypothesis testing is pretty useless really.

Thanks for your detailed reply! I meant that even this dataset puts a limit on the effect size, if viewed as an "evidence of absence of clear and large effect".

Of course since "everything is correlated" [0] expecting such truly simple signals might be nonsensical/pointless.

I was just lamenting the lack of simple magical treatments basically.

[0] https://www.gwern.net/Everything

That gwern page is excellent!

>Participants were randomised to two sessions weekly of high intensity interval training at about 90% of peak heart rate (HIIT, n=400), moderate intensity continuous training at about 70% of peak heart rate (MICT, n=387), or to follow the national guidelines for physical activity (n=780; control group); all for five years.

I can't quite tell by your summary whether you are saying "exercising is good, but the overall effect on mortality is small", or "exercising is good, and the effect of adding HIIT to baseline exercise is small".

You cannot make the former statement from this study as control group were not non-exercisers (and adherence was decent). The latter statement, does seem to be supported.

Regarding exercise in general, the literature shows the mortality gap between exercisers and non-exercisers is absolutely massive.

2.9% seems like a significat risk reduction to me. HIIT results in a 1.7% risk REDUCTION and MICT results in a 1.2% risk INCREASE.

No difference between people managing their own exercise program compared with those who had a trainer.

"These differences were not statistically significant" according to the study.

The study itself says that they observed no difference. That is not a strong result. Which is OK.

Reply on news.ycombinator.com