Department of Awful Statistics: Small Schools Edition

And that's not even the worst of statistical illusions. After reading about, for example, Simpson's paradox

http://en.wikipedia.org/wiki/Simpsons_paradox

I concluded I'll always be very wary of using statistics for anything. It's a powerful tool, but also very easy to shoot yourself in the foot with. (Or to get deceived by. And they don't even need to know statistics better than you in order to fool you with some reasonable-looking analysis!)

At the least the Gates Foundation has enough mathematical sophistication to understand what they got wrong.

Why link to the link? This is what you should have pointed to: http://www.marginalrevolution.com/marginalrevolution/2010/09...

Oh! I thought the claim was that there was an error in the blue-grey box somewhere, and I was feeling dumb for not seeing it. Then I finally read the sentence at the end about the Gates Foundation, and I understood. :)

Aside: based on the percentages, it looks like there were 85 schools per decile, or 850 schools total (give or take a few); in which case the total percentages indicate that a total of 58 or 59 distinct schools made it onto the Top 25 over a four-year period (23 of them from the smallest-decile schools). So there really is a lot of variability in the winners. I'd be interested to see a similar decile breakdown of schools that managed to get in the top 25 for at least two of the four years studied. That would seriously damp the noise.

Hopefully the funding will be diverted into support for smaller class sizes.

The WTF here is how anybody drew a conclusion without performing a simple statistical test for significance.