Data Visualization Disease

jgc.org

30 pointsjgrahamc7 comments

Lies, damn lies, and charts.

It's the sad case of the amateur statistician, who drowned crossing a stream with an average depth of six inches.

I once got told off by a very senior person in a company during a client meeting for presenting data where all the values were positive but the mean - 1 sigma was negative because he claimed that could never happen.

A few days later I emailed him three numbers: 1, 2, 100.

He apologized.

Not to say your boss was totally in the right, but if he thought the summary statistics could never be negative (due to knowledge of the source and meaning of the data) that can be very informative.

If I were told the numbers {1,2,100} and that the data was definitely non-negative, I'd fit the thing to an exponential perhaps and thus set my 95% error bounds as (1.74, 101).

Which isn't a whole lot better than the mean/sigma stuff, but it would take into account the non-negativity.

If all of the data is positive, but the mean - 1*sigma is negative, the gaussian approximation probably isn't very good....

For the first time in my life, I see what that quote meant.

Hmm....appears to discuss confirmation bias.

Reply on news.ycombinator.com