Hacker News Reader

Comment by sevenless | Hacker News Reader

They also have more variability. A tiny nation that gets one medal gets a huge per-capita frequency. You'd want to regularize it a bit.

View on HN

Definitely. A similar example: at one point studies started coming out that showed the best performing schools were small. Cue a lot of noise on how small schools are better, if you want your kids to be successful, send them to a small school, etc. I think these findings even showed up in government reports/recommendations.

Problem is, the worst performing schools were also small. Almost all of the effect could be explained by higher variance.

Unfortunately I don't remember where where I read about this, wish I could provide sources.

EDIT: found it. It wasnt government, it was the Gates foundation, who spent billions into supporting smaller schools.

http://press.princeton.edu/chapters/s8863.pdf

Oh my God. They wasted a billion dollars and made education worse because they didn't understand variance.

While the research wasn't performed by the government, I think its the basis for the move toward small "schools-within-schools" as an educational "reform" in a number of public schools.

This is a beautiful article. Though I've read a number of reports that the higher male variance isn't all it was thought to be, but can't find it again. There should be done interesting studies about the variance in different species whose gender differentiation is done by a different mechanism, namely where the female has the two different chromosomes.

How would one regularize to account for such variability (in a way that doesn't discount and favor the performance of small and large countries respectively)?

Genuine question as I'm admittedly not very familiar with the mathematical/statistical aspects of this. I am however mildly satisfied with my home country's per-capita performance in this and prior summer olympics, and curious to learn if/why I maybe shouldn't be :)

You could use a Bayesian model to incorporate prior information about the distribution of global medal rates into a conclusion (a posterior) about a particular country. This explanation will be simple and empirical, but ideally you'd integrate over all country-level data and the prior at once.

Count data is nice to model with a Poisson distribution, and a conjugate Gamma prior distribution. We'll assume a lot of things about the distribution of medal rates and medals which probably aren't true in reality.

For the global prior, eyeballing the medals-per-capita list for all countries, it has mean value around 3E-7 and standard deviation about 5E-7. That corresponds very roughly to the distribution Gamma(0.25, 1E6) for the prior. A Gamma with parameters (a, b) has mean value a/b and standard deviation sqrt(a)/b, so I just matched the numbers.

For a country with N medals and P population, we want to update our posterior distribution of their medal-per-capita rate based on a Gamma(0.25, 1E6) prior and a Poisson distribution. The updated posterior distribution is conveniently, Gamma(0.25+N, 1E6+P). (That's the conjugacy property).

So Grenada's medal rate, with N=1, P=100,000, has posterior distribution Gamma(1.25, 1.1E6). This has mean value 1.25/1.1E6 = 1.1E-6. Compare that mean value to the raw medal-per-capita rate of 1E-5 - in other words, if this model is to be believed, we should heavily discount Grenada's great performance. On the other hand Australia's medal rate barely changes from 9.2E-7 to 9.7E-7 by using the prior, close to Grenada's Bayesian rate. So it takes a larger population into account quite naturally.

Reply on news.ycombinator.com