Comment by tgv | Hacker News Reader

Comment by tgv | Hacker News Reader

Parent Full thread

ACTIVE doesn't have much to show for it, does it? Effect sizes are small, and some are almost absent, while there are differences in other factors between the groups with the same magnitude (drinking, cholesterol, diabetes, subject withdrawal, etc.). The calculation of the effect size is also dubious: it assumes the changes are linear and independent of the starting position, and the baseline is not within the group, although I can't see an easy way to get an objective one either.

As to the statistics: it's p-values all over, with all of the associated problems. The p-value for article [4] gets just below the 0.05 mark, but only for speed training. They also do two other comparisons, which are nowhere near significant. That is really suspicious, and there's no correction for multiple comparisons.

All in all, this short inspection doesn't convince me the OP article has it wrong.

For folks who dislike p-value reporting (and I'm one of them), we can focus on the effect sizes for speed of processing training in ACTIVE:

[1] 0.36 in slowing decline in functional abilities, equivalent to ~3 years of delayed decline [2] 48% reduction in at fault auto crash risk [3] 30% reduction in the risk of experiencing serious [0.5 s.d.] worsening of depressive symptoms [4] 29% reduction in dementia incidence [hazard ratio]

These are all clinically meaningful effect sizes.

Regarding the dementia incidence study, it's correct that two of the cognitive training interventions did not show effects, and speed of processing training did. In my view, a straightforward interpretation is that different types of cognitive training are different (much like different small molecule pharmaceuticals are different), and consequently they have different effects on endpoints like dementia incidence.

I don't disagree with you, generally, but if you had hit the first of the 5 quoted papers the OP linked, you would have seen that the effect size at least in a study was medium. Not small.

I did check [1]. That's where the differences between the four groups can be found. There are 12 effect sizes, only two or three medium (≥0.5). Some of the others are even negative. It's pretty suspicious that the type of training matters so much. It's as if it's training to the test.

But if this is what it is, then by all means, do cognitive training for the elderly: it can't do any harm. But there doesn't seem to be any point in further research.

Reply on news.ycombinator.com