Edit: Jesus, this thread is proof than HN techies think they know math but actually have no clue about statistics. They are easy prey for this kind of journalism.
(Just saw your rebuttal about "significance" below. Statistical significance is not how big a movement is, it is the chance that such a movement could be due to random chance. A very small movement could be statistically significant if your sample is big enough to show it's probably (e.g. 95% probably) not due to chance.)
And yet you are the one confusing the standard deviation of an effect size with the standard error of a point estimate...
> Point estimates for effect size on all cognitive tasks were 0.164 > (SE = 0.028, n = 96, p = .05) for control groups and 0.478 (SE = > 0.029, n = 101, p = .01) for exercisers, suggesting that both groups > improved between Times 1 and 2. However, the control groups’ im- > provement was about 1/8 a standard deviation, whereas the exercise > groups’ improvement was nearly 1/2 a standard deviation, on average. > Both values are significantly different from zero, and from each other > (see Table 1).
Earlier in the paper they define their effect size as:
g = (M_Post - M_Pre )/SD_P , where M_Pre is the preintervention mean task performance, M_Post is the postintervention mean, and SD_P is the pooled standard deviation.
I think the standard deviation mentioned is relative to overall population spread of cognitive performance, given that standard errors of the effect size are given separately?
But it's not about how big the effect is. It's how likely that the effect was or was not due to chance. A very small effect can be statistically significant. You missed "sample size" in your list above - a big sample size can help you show a small effect is probably not chance.
I can't read the original article because it's paywalled, but assuming that the original claim is correct ("increased performance by approximately .5 of a standard deviation across all tasks tested") and that by "significant" you mean "statistically significant" then you are spectacularly and confidently wrong.
Imagine we have a normal distribution of IQs around 100 with a standard deviation of 15.
Imagine there were some intervention which resulted in a uniform increase in IQ by half a standard deviation across the board. Perhaps the intervention is "literally add 7.5 points to every single score in the treatment group."
You seem to be asserting that because the effect size is "only" half a standard deviation, we have no way of determining whether or not the intervention actually increased scores, no matter how much data is collected. We could collect 100 million IQ scores in the control group, and 100 million IQ scores in the "treatment" group, and see that the average score in the control group is 100 and the average score in the treatment group is 107.5, and we'd have to say "nope, the null hypothesis looks completely plausible here, because the effect size was just half a standard deviation."
Does that seem like a reasonable claim?
I imagine the average HN user to look like Richard Stallman.