The Probability and Statistics Cookbook
statistics.zone
statistics.zone
[1] Gigerenzer, G (November 2004). "Mindless statistics". The Journal of Socio-Economics. 33 (5): 587–606. doi:10.1016/j.socec.2004.09.033
[2] 'The phrase "errors of the second kind", although apparently only a harmless piece of techinical jargon, is useful as indicating the type of mental confusion in which it was coined.' -Ronald Fisher. "Statistical Methods and Scientific Induction." Journal of the Royal Statistical Society. Series B (Methodological) Vol. 17, No. 1 (1955), pp. 69-78 https://www.jstor.org/stable/2983785
[3] 'no test based upon the theory of probability can by itself provide any valuable evidence of the truth or falsehood of that hypothesis.' -Neyman, J; Pearson, E. S. (January 1, 1933). "On the Problem of the most Efficient Tests of Statistical Hypotheses". Phil. Trans. R. Soc. Lond. A. 231 (694–706): 289–337. doi:10.1098/rsta.1933.0009.
Initially, I wrote this cookbook/cheatsheet in order to structure and retain the material in these courses, not to challenge them. Most of the content comes from the cited references, all of which have a very terse and mathematical presentation. It would be great to augment the current document with pointers to the literature that offer a critical discussion. As a non-statistician, I lack the historical perspective, but I always appreciate contributions from experts in the field. (The document is open-source: https://github.com/mavam/stat-cookbook)
[1] https://en.wikipedia.org/wiki/Statistical_hypothesis_testing...
Anyway, you don't think power analysis à la Cohen is useful?
Also, this isn't really about what I think, rather I would hope people check the Fisher 1955 ref and go from there.
What I think though is this whole idea of testing vague/vagrant hypotheses (eg the example we used here) is wrong in the worst way possible. The null hypothesis should be deduced from some theory, or at least correspond to what you care about. I have shared this paper on the site many times, I think it should be standard reading in high school: http://www.fisme.science.uu.nl/staff/christianb/downloads/me...
Sample size, effect size & power are related concepts in the context of power analysis -- see also Cohen's "A primer on power", which is available on the Internet. The concept of power has nothing to do with "degrees of evidence" or vague hypotheses.
Sorry for the miscommunication. The point is that power is a Neyman/Pearson concept, Fisher said it didn't make sense. On the other hand a gradient of evidence is a Fisherian concept, Neyman/Pearson said that didn't make sense.
What people have been teaching as stats is a mismash of the two that makes sense to no one who thinks these types of things through. Gigerenzer reviews this strange phenomenon and offers some entertaining commentary, it is a decent starting point.
Yes it does. To properly assess the probability of incorrectly failing to reject a hypothesis you need to know how likely the data would be under various rival hypotheses. This depends on the precision of the various hypotheses. This is explained by Fisher in my original ref.
is the author using a null value to inform this perception?
Clearly he doesn't think it had exactly zero effect, since it affected him!
"Estimating parameters from sample" (on the right) would be his observation that there was little discernible effect. Thinking that 1000 reprints of the paper would have a larger effect on practice would more correspond to "theory" (on the left), although that is a pretty vague one.
It is no accident that statistics is usually presented as arising anonymously and forming a monolithic paradigm, when in fact the exact opposite is true. Stats must be one of the most controversial areas of intellectual activity around. Those refs are to start off the curious, some of whom will manage to break free of the brainwashing and think for themselves.
Covers: Hypothesis testing (single population), Analysis of variance (multiple populations), Control charts, Design of Experiments (A/B testing and beyond).
Edit: Used in the undergraduate industrial & systems engineering program at USC when I was a student. The 7th edition has various cook book style walkthroughs.
I appreciate anyone can look "sample space" or "parametric inference" up on Google, but it'd take some time to find some authoritative source (especially for people like who do not work with stats every day). It'd be awesome if I could see a "[1]" and a reference (or list of references), either online or offline, where the concept is defined.
From a presentational point of view, I wonder if it would pollute the plain/clean presentation of the formulas. Perhaps very small footnote-like citations could work, but it has to be unobtrusive.
The hardest part, however, will be coming up with the authoritative source for each concept. As it's outside my field of expertise, we would have to rely on the community to fill in these details incrementally.
Everything I've tried has been absolutely horrible except for "An Introduction to Error Analysis" by John Taylor (yeah the classical mechanics guy). Unfortunately it's a bit basic...
Intermediate: Rice, "Mathematical Statistics and Data Analysis"
Wasserman's "All of Statistics" is also a very good book except that it is too terse to be a primary text.
Goes over the commonly used methods in science but only explains the intuition behind the method and what you can and cannot expect from the results, drawbacks etc. Written in a very conversational and opinionated style, I enjoyed it.
It rarely explains how any of it works (you'd be hard pressed to find the formula for a probability distribution function, for instance), so it's just a one-stop collection for a lot of useful tests and the circumstances under which they should be used.
It's less of a textbook, and more of a reference for someone who needs to occasionally work with statistics and can't remember offhand when the T-test is appropriate or the procedures to undergo for a chi-squared test or whatever.
Bayesian Data Analysis by Gelman et al.
Second, this is almost useless for multi-columns PDFs. It just works fine with very simple PDFs. Everything with a minimal of complexity becomes junk.
Reading PDFs in the kindle is a terrible experience. The text doesn't reflow. You get lost in multiple columns. Even the next doesn't work fine.
Since it is something open, with the original content published in GitHub, I thought someone should have generated a mobi file.