Hacker News Reader

Comment by fyp | Hacker News Reader

Can someone give me an example on how to properly use the central limit theorem in the real world?

For example with stuff that are commonly assumed to be bell-curved like test scores, IQ, etc. What are the iid variables being averaged? Each test question?

View on HN

He's the example I have my students do in class. Roll a die repeatedly and tally the results. You'll get a (roughly) uniform distribution of 1s, 2s, 3s, 4s, 5s, and 6s.

Now to illustrate the CLT, you roll a die 50 times, and average the result. AND your 300 classmates do the same. If you tally the 301 averages, the distribution of the averages will not be uniform but bell-shaped, with average (approximately) 3.5.

The CLT says (roughly) the distribution of the averages will be approximately normal, regardless of the original distribution.

Short and concise, as things should be. Thank you. :)

Here’s something that’s bothered me for a while. Why is it necessary to roll it 50 times? Why can’t I roll it just once? Does each X_i have to be a sample greater than 1 and if so is there some kind of minimum required for each random sample? Does CLT still hold if each random sample’s size is 1?

As the number of samples approach infinity the mean of all the iid samples will become normally distributed.

So it will neither hold strictly for n=50 or n=1, but n=50 may be a better approximation of infinity :)

As I understood it each X_i is a random sample and the n refers to X_n, that is, n random samples. But what is the size of each sample? In the parent it was 50. The dice is rolled 50 times by each person i, and the outcomes are given by the random variable X_i. Also, there are 300 people rolling a dice, so n=300, thus 300 random variables each composed of 50 dice roll outcomes. I’m curious why each X_i has to be a sample of size 50 and can we just have each person roll the dice 1 time? Maybe we need 50*300 people to roll the dice one time now? Does the CLT work when each X_i is a random sample of size 1?

Each person has to roll their die a number approaching infinity times, then the reported means will be normally distributed. Now you will almost always get something quite indistinguishable from normal with n=50 throws, but not with n=1.

If you plot the means of the 300 dicethrowers for n=1,2,3,4... dice. You will see the distribution of the means take on more and more of a bell shape. But fully normal it becomes only when n approaches infinity (and then it will be a quite spiked bell of course :))

So, to be clear: the CLT talks about behavior when n approaches infinity. The CLT can be used approximately before that, but it's a very bad approximation when n=1.

That's a good question, and it goes back to the definition of the CLT and the distinction between a distribution vs. a sample. We get to pick how we define the terms for our problem!

So in the original example, we know that if we choose X_n to mean an individual die roll, then the random variable X is not uniformly distributed. However, the CLT tells us that the average of many rolls is indeed normally distributed.

So if you pick n=50, then the average number of pips will resemble a normal distribution. What's the point of having 300 students do it then? It's so we can gather evidence that we're correct :)

Let each individual student do a 50-roll run. We know each run is i.i.d, so let Y represent the distribution of the average number of pips, and Y_1 ... Y_300 represent 300 samples - we can graph the empirical distribution of Y and test it for normality.

However, going back to your actual question, what if we had each person roll just 1 time? Is the outcome normal?

It depends on what you mean - how do we aggregate the 300 individual dice rolls? If you take each individual dice roll and tally it up (how many 1s, how many 2s, etc.), you will find that the distribution is still uniform - so no, a 'sample size' of 1 was not sufficient.

However, what is normally distributed is the average of those 300 dice rolls. However, in this case, we only get a single 'sample' (something close to 3.5), so we lack evidence that the CLT holds (even though it does).

Same goes with your other question - what if you had 50 * 300 people roll the dice? The average number of pips over 15k die rolls is approximately normal, but you've only drawn a single 'sample' in this case.

Going a bit further, this implies that what matters is not just how many dice we throw, but how we choose to define the meaning of those throws - for 15k throws, you can choose to think of it as 15k samples if a single throw, or 15 samples of (the average of) 1000 throws, or anything in between - just pick the definition that's useful.

Ok great explanation. Going to your last paragraph, what does this mean “in practice”. Say I have a dataset of 15k people throwing a dice 1 time. Can I group it into 300 random samples of 50 dice throws and now say I have 300 random samples and apply the CLT? Each random sample should be i.i.d no?

I was just curious if one can go the other way around, because usually you only have 1 sample of size 15k and not 300 samples of size 50. If you have the raw data it’s just 15k samples of size 1 or 1 sample of size 15k, depending on how you look at it.

Then I could be wrong here, but doesn’t the proof of the CTL also assume that each random sample is the same size? So you can’t have one sample of size 30 and another of size 20 and another of size 25, etc? Each of the 300 samples must be size 50?

Yes, each dice roll is i.i.d, so you can choose to look at it as 300 random samples of 50 throws each, or any other choice of N.

Put another way, what you have is 15k dice throws, and the fact that they were thrown by 15k, 5k, or 300 people can be ignored, if you choose to. In fact, it may be useful to 'shuffle' dice rolls into new 'samples' - that gets in to the use of resampling and bootstrap techniques.

And also yes, for CLT to apply, each sample must have the same N.

> I’m curious why each X_i has to be a sample of size 50 and can we just have each person roll the dice 1 time?

Smaller samples have higher variance and their means probably won’t converge as fast to a normal distribution.

“Samples” of one, in isolation, have no defined variance and you can’t use them to infer anything.

By the law of large numbers, the sample mean approaches the population mean as the sample size gets larger. The law of large numbers goes hand-in-hand with the central limit theorem.

Ok, but the real pdf is cut off at 1 and 6, while the normal distribution extends to +- infinity and can never model the cutoffs, no matter how large N is (though of course those tails get less significant).

like the other comment points out, the measured rv is shifted and scaled. re tails: there's a sqrt(n) that scales the measured distribution so that it in the limit it has tails at +/- infinity.

The CLT can be used to justify modelling an effect with a normal distribution and raises questions about model assumptions when normal distributions don't appear where they should. The Normal Distribution is special because it is the highest-entropy distribution that is completely characterised by mean and standard deviation [0].

So what the CLT is saying is that, surprisingly, sums of i.i.d. random variables lose information from the individual variables quite quickly but reliably retain a little data about mean and second moment. Initially a given X_i has all sorts of information associated with it (higher order moments, other distribution characteristics, etc) that disappears as many X_i are summed together. All that is left is information about a mean and variance.

So say I have a situation where a large number of i.i.d. variables are going to be summed together. The CLT tells me that summing N of these variables together is going to be similar to summing together an equivalent number of normally distributed variables (!!). This is because the sum variable is equal in value to n * \bar{x}, but the CLT imposes a distribution on \frac{\bar{x}}{n}.

This justifies why the normal turns up everywhere in practical measurements. A lot of measurements (say, number of people at the beach) are probably really measures of a sum of random variables (maybe there is some non-normal variables that captures the chance a given person goes to the beach). So if the number of people at the beach turns out to be normally distributed (maybe I measure it each day for a few weeks) it isn't shocking. If the total number isn't normally distributed then that implies that there is no i.i.d. variable representing probability a given individual goes to the beach (eg, high correlation between the individual variables).

I didn't quite fail any of the statistics courses I've ever done, YMMV.

[0] https://en.wikipedia.org/wiki/Normal_distribution#Maximum_en...

Suppose you have a school with 30 groups of students. All students are randomly assigned to the groups (so independent of their skill at IQ tests). The distribution of measured IQ within classes can be any distribution. But let’s take the 30 averages of the groups. The mean of these averages of groups (asymptotically) follows the normal distribution according to the CLT.

not the mean of the means - just the means themselves.

The distribution of the original population of iqs in the school is what you're sampling. Each group is a sample. The distribution of the sample mean approaches a normal according to clt as the sample sizes increase.

there's a distinction between random variables that are assumed to actually be normal and RVs that we conceptually apply the clt to and then treat as normal.

in the case of test scores it could be either. for an individual test it's possible the distribution is roughly normal actually. but if you, for example, look at subsets of all SAT scores and take their means (the mean of the subset) then those averages will be Bell shaped because of the clt.

Just to directly answer your question: the iid random variable is the test score or the iq.

Reply on news.ycombinator.com