Hacker News Reader

Comment by wheaties | Hacker News Reader

6!? They used an N=6 to make such a bold claim? If I had ever thought about that as a sample size for any scientific research my professor would have run me off campus.

View on HN

With 6 subjects, it feels a bit silly to even turn on the math. 5% is about two legs of a participant. I think humans can process those low numbers instantly, intuitively; we can do it by pattern recognition, without the need for abstraction. The statistics of a dinner party. I think even some animals could do some "math" on 6 food.

Should be called "collected case studies" or something.

I also find it a bit hard to believe that going from 102±7 to 109±5 has a p<0.005. If I run a simple t-test, it says p-value (two-tailed) = 0.07.

Unfortunately a lot of medical science involving expensive laboratory resources and trials currently rests on sample sizes around this range (~5-20). Then whenever there's any kind of technical innovations that makes it cheaper to collect the biosignals there is a sudden series of breakthroughs in what is really happening. All science is a work in progress.

It is a paired t-test as the test takers are the same individuals. This is confirmed in the statistical analysis section of the actual paper. Paired t-test have more statistical power because we don't correct for group variation (as they are the same group measured twice).

Edit: One way to think about this is assuming the effects are random and symmetric then the probability of going up is 1/2. If all 6 participants had increased scores, then the p-value is at least 1/2^6 and this is not taking into account the magnitude of the shift.

You're right about the paired test (although I think there's an issue with the group variance). But your final calculation is for a sign test. For a t-test where every point shifts a very little bit (e.g. 0.01 stddev), p is larger, I think.

Yes, my edit was more meant to illustrate how 6 data points can yield a significant result. Paired t-test is all about the differences. So if all test subjects shift up by 0.01 (no variance in the differences) then the statistic will actually be infinite (i.e. very large) and so p is theoretically 0 (very small). I also agree that the paired testing needs more convincing from the researchers.

Reply on news.ycombinator.com