Hacker News Reader

Comment by klausnrooster | Hacker News Reader

My (mis-?) understanding of the Central Limit Theorem is that the average of >30 measurements increases the accuracy. https://en.wikipedia.org/wiki/Central_limit_theorem

View on HN

No, that's increasing the precision, not accuracy. Many imprecise measurements can be averaged to a more precise measurement.

Say we had a perfect ruler to measure the length of something. It's absolutely precise and accurate. All measurements from it return the exact, same value, and preternaturally we know it's the "correct" value.

Now say some bandit comes in while we're not looking and adds a small chunck of diamond to the end of the ruler without telling us. Our ruler is still precise, but no longer accurate. If we take many measurements with it, they always come back with the same value. Averaging those values does not improve the accuracy at all.

Alternatively, say the bandit starts randomly changing the temperature of the room we are in. Thermal expansion is constantly changing the length of the ruler. The average length of the ruler is still the same, so it's still accurate, but it's no longer precise. We could average the measurements we take with it and get a more precise measurement.

What the central limit theorem says is that the averages of our measurements will be normally distributed, even if the change in temperature is not.

This really doesn't make much sense in the context of the measurement which is bounding the value(0) with 90% confidence[1].

1. https://arxiv.org/pdf/2212.11841.pdf

I think op is talking about bias vs variance. In some cases we have a bias variance trade off.

Yep, and I'm talking about hypothesis testing which is what's being done in the paper.

Hypothesis testing cannot rule out systemic bias errors present in both control and treatment populations.

Are you pontificating about systemic bias or is this a specific critique of the techniques used to simultaneously detect the upper and lower doublets in a single shot of the experiment?

I am trying to explain to you what moron4hire is talking about. Hypothesis testing doesn't magically fix the issue he is discussing, as hypothesis testing works by collecting many imprecise measurements to average it out to a more precise level.

That's all true. I am biased because I am responsible for keeping hundreds of devices calibrated, and keeping a lab within 2C of 23C. The CLT implies that the sample mean is more likely to be close to the true population mean as the sample size increases. Not the same as more accurate. Shame on me!

The number of people in this thread not understanding the difference between bias and variance errors is too much.

http://scott.fortmann-roe.com/docs/BiasVariance.html

Reply on news.ycombinator.com