Hacker News Reader

Comment by kevinalexbrown | Hacker News Reader

If you're looking for things to add big-picture-wise, it might be helpful to specify what assumptions go into various tests/methods. In my experience, this is the biggest hangup and mistake, because a) it's more difficult to understand and b) ignoring it gives the appearance of rigor even if the test used is inappropriate for the data.

I should emphasize that this is not a nitpick or even a criticism, just a feature I would love to see. It's also what I spend a large portion of my time trying to track down, so having it in a convenient location would be nice.

View on HN

This guy has written up many of the most common statistical tests, their interpretations, and their assumptions in a very human-readable way:

http://udel.edu/~mcdonald/statintro.html

I point a lot of newbs to pages on that site so that they can develop a better intuition for the methods.

This is an amazing resource thank you.

I wish I had this last semester during my statistics course.

If you're looking for more of this, but in greater depth, pick up a copy of Biometry.

It's written for biologists, but you don't really need to know much biology to work through the examples, and the focus is inherently practical.

Just as a rule of thumb, in general the most important assumption that is violated is independence. If the data is not independent and is instead positively correlated (the most likely form of correlation in practice) most equations will overestimate statistical significance.

A small amount of correlation can lead to very biased estimates of significance.

Correlation between observations will generally appear in any time series data or data that is arranged spatially.

Again, just a rule of thumb, but you should be very wary of the lack of independence between observations.

To add to this comment, large correlation between (or among) variables will often inflate estimates of standard error, leading to underestimates of statistical significance.

Great suggestion. I've been amazed to find out that many coders and amateur "data scientists" don't realize that testing the assumptions is an important part of conducting statistical analyses. Part of this may be due to the recent emphasis on machine learning techniques, which tend to be assumption-free (often just assuming independence of cases in the sample).

Part of this may be due to the recent emphasis on machine learning techniques, which tend to be assumption-free...

No statistical technique is assumption free, unless it is purely descriptive.

Some of them are free of explicit assumptions known by the practitioner, but that's not the same thing. In much the same way, my code is all bug-free.

    machine learning techniques, which tend to be assumption-free

ML should be a rigorous exercise in Bayesian and classical/frequentist stats, computational methods, dataset integrity, visualization etc, if you've been thru the texts by Murphy or Bishop. It often happens that people a couple years out of their last stats class only retain that high R-squared, p-, t- and f-values are what they're looking for, and heteroskedasticity and sphericity are just big words.

My evidence that ML is a rigorous exercise: the free texts listed (Barber, Mackay and Smola's are excellent, ESL not as accessible)

http://metaoptimize.com/qa/questions/186/good-freely-availab...

Thanks, @gtani, great resource. Yeah didn't mean to imply that ML techniques are free of ANY assumptions, just that several of the popular ones like logistic regression don't have distributional assumptions. (actually, I really want to understand the VC Inequality at some point, as it seems to allow us to make conclusions about out of sample error rates without depending on distributional assumptions)

Reply on news.ycombinator.com