Comment by SiVal | Hacker News Reader

Comment by SiVal | Hacker News Reader

Parent Full thread

Imagine you have a population that is NOT normally distributed. Take a large sample of it, and calculate the mean of your sample. Then take another large sample of that population and calculate its mean. Keep doing this. Think of each sample mean as a data point that gives you an estimate of the mean of the whole population. The Central Limit Theorem says (approximately, and there are conditions) that those sample-mean data points of yours will be normally distributed, even though they were obtained by sampling a non-normal distribution. This also works when the population IS normally distributed, but that's not a surprise. The surprise is that the sample means are normally distributed even when the population isn't. (And again, there are some qualifications I'm not mentioning.)

So, sample means (estimates of the population mean) tend to be normally distributed even when the population they are sampling is not.

The D3 demo here doesn't show multiple sample means, it shows that as you take more and more items from a normal population, the distribution of your sample of items gradually comes to resemble the distribution of the population as a whole.

No, it does show multiple sample means. Each landing spot is the sum (scaled average) of the n - 1 bernoulli rvs, where n is the number of bins. The issue is that n is small.

You are correct in pointing out this error of the parent comment (by SiVal). Each ball is indeed the sum of "n-1" bernoulli RVs, and the CLT does apply to these sums.

As btilly points out elsewhere, to actually obtain the correct limit, you have to normalize the sum correctly. Because of the way the scaling is done in this graphic, as you increase the number of levels, it's in effect normalizing the sum by dividing by "n". To get the right limit, you need to divide by sqrt(n).

In this sense, the CLT is a high-resolution version of the SLLN ("law of averages"). If you normalize the sums by 1/n, the resulting average converges to a number, the mean.

But if instead you subtract this mean, and normalize by 1/sqrt(n) rather than 1/n, the result converges in distribution, and looks like a normal random variable.

By less aggressive normalization, you get information about the fluctuations rather than just pounding it down to a number, as the SLLN does.

I used to TA a probability class for undergrads, and I found the demo to be perfectly reasonable.

If we're going to pick nits, it would illustrate the "Weak" LLN, not the Strong LLN. The Strong LLN still holds here, but you'd need to plot the sample paths to illustrate it.

I was actually trying my best to not pick nits, because HN has somewhat too much of that.

The reason I posted is to defend the OP against some assertions that I felt were picking at details of what started out to be a simple and fun visualization. I wrote what I did about the LLN to supply some intuition to the person who created the visualization, in case they wanted to put these two results in perspective.

Thanks for this explanation. I understood most of it but could you explain why you should normalize using 1/sqrt(n) and why doing so makes the result converge in distribution?

For a sequence of independent random variables with the same variance, X_1, X_2,..., we have

  var( (1/sqrt(n)) * (X_1 + X_2 + X_3 + ... X_n) 
    = (1/n) * (var(X_1) + var(X_2) + ... var(X_n))
    = (1/n) * n * var(X_1)
    = var(X_1)

This holds for any n, which means that, if you normalize by 1/sqrt(n) instead of 1/n, the "randomness" never vanishes even when n gets infinitely large. If you normalize by something bigger than 1/sqrt(n) the variance blows up, and if you normalize by something less than 1/sqrt(n), the variance collapses to zero so you get something concentrated at a single point.

The CLT tells us more than that, it actually tells us how the randomness is distributed when n gets very large, which is pretty remarkable when you think about it. (and it holds under much weaker conditions than what I mentioned above, it's just that those assumptions are probably the easiest to understand).

Thanks a lot for the explanation.

Reply on news.ycombinator.com