The Normal Distribution: A derivation from basic principles [pdf] | Hacker News Reader

The Normal Distribution: A derivation from basic principles [pdf]

courses.ncssm.edu

28 pointsnkron6 comments

Thanks for posting this. I've been looking for intuitions about what's so special about the normal distribution.

While this is a nice little derivation, the reason the normal distribution is important (and common) is the Central limit theorem[1].

This loosely means that if you have n samples that are independently drawn from the same distribution, their sum approaches a normal distribution for large n. The generality comes from the fact that this happens (almost) independently of distribution the samples come from.

[1]https://en.wikipedia.org/wiki/Central_limit_theorem

This is often given as the reason in the textbooks, but it actually is a sloppy argument, for there is no reason to presuppose sums of independent variables (they do not have to be drawn from the same distribution in some variants of the CLT) are so ubiquitous in nature. A more convincing reason is given by:

http://en.wikipedia.org/wiki/Maximum_entropy_probability_dis...

Well, I think textbooks often give the maxent justification as well, but probably neither is strictly superior to the other.

Often, where a measurement is perturbed by a litany of physically unlinked disturbances (e.g., antenna noise), the CLT argument is compelling. In a case like antenna or front-end electronics noise, justifying the maxent argument would require buying in to an almost mystical belief by comparison.

Interestingly, the two seem to have a deep connection related to thermodynamics, e.g. http://en.wikipedia.org/wiki/Thermal_fluctuations, where you can see both as expansions of a configuration count up to second order in the exponent.

This is manifest in the detectable remnants of the Big Bang, which are a particular realization of a Gaussian random field that was present due to quantum fluctuations of a primordial system. It's funny to think of the universe as a draw from a random number generator.

Isn't the key point here about isotropy, similar to the velocities in the ideal gas?

I think you're right, but it's really isotropy in combination with independence of x and y. Isotropy means you really are dealing with a function of r where r^2 = x^2 + y^2, and independence means the resulting function p(x,y) must factor as p(x) * p(y). This then forces the exponential form.

Reply on news.ycombinator.com