Hacker News Reader

I don't think the graph is particularly good either, but I think you're maybe reading it wrong too. The y axis is significance level which makes sense would normally improve as visitors go up. Their line arguably is a moving average regression.

View on HN

He's saying that as new data arrives, it has to adjust both ways, not only toward the correct answer.

(If that wasn't the case, you could just figure out which was the only direction it would move and then stop collecting data. You've already got your answer)

Their graphs don't only adjust up. That one does, but that's because the only really significant downward movement is while their regression is still trailing behind.

It's a statistical significance test not a mean. You expect statistical significance to go up over time. Unless the effect size is zero, it doesn't have an asymptote.

Reply on news.ycombinator.com