Hacker News Reader

Comment by ssivark | Hacker News Reader

My personal opinion is that statistics textbooks usually come from a prescriptive perspective, and that makes it challenging for the reader to get visceral intuition for what is actually going on. Any reader would be far better off just visualizing the damn distribution / samples and using reasonable judgement, instead of implicitly assuming a Gaussians distribution and blindly memorizing tests / formulae. Making the distributions explicit allows us to model them and get an intuition for what the samples are telling us. I would whole-heartedly recommend the Model based machine learning book to anyone (online version is free) https://mbmlbook.com/

View on HN

How do you make 'reasonable judgements'? How do you tell whether someone else made reasonable judgements? How do you judge other people's intuition?

Modelling distributions explicitly sounds nice, yes.

Look at the histogram and think about what distribution one could reasonably impute from samples. And what you would set as bounds for "outliers", per your needs. While we're at it, let me also say that it might be useful to specify outlier bounds not just based on the spread in sample values, but the costs/payoffs they imply for your application.

If you are not doing something crazy, most reasonable people would agree with your judgement. Conversely, if you are making non-obvious inferences where reasonable people disagree, you are in murky water and no sophisticated statistical method will save you. Math is not magic; theorems merely recycle (launder) modeling assumptions into results.

>Look at the histogram and think about what distribution one could reasonably impute from samples. And what you would set as bounds for "outliers", per your needs. While we're at it, let me also say that it might be useful to specify outlier bounds not just based on the spread in sample values, but the costs/payoffs they imply for your application.

Only people with prior education/training in statistics are capable of doing this. The people who don't need a textbook.

Something like 60% of US adults read at or below the 6th grade level, and 25% of US adults struggle to comprehend graphs or charts entirely. Someone who has no idea what a standard deviation is can't intuit about distributions. I think you're dramatically overestimating the average person.

> Someone who has no idea what a standard deviation is can't intuit about distributions.

I disagree vehemently with this claim. I could cite my experience in teaching this topic to liberal arts / humanities college students in the US, but it is really more obvious than that. Anyone can understand a histogram easily and far more intuitively than they can understand the formula for a standard deviation and whether it must divide by N or N-1. Statistics courses and textbooks get stuck on that kind of pedantry, and most students end up missing the forest for the trees.

Yup. And thinking through the physical realities or whatever real world constraints exist.

My larger issue, any time I have tried to learn statistics, is how fast the notation moves. You end up flipping back pages and pages just to double-check a definition that was given once and is now being extended syntactically. It's infuriating.

Reply on news.ycombinator.com