What Is P-Value – A Complete Beginners Guide from Scratch

p-values are commonly misinterpreted as the probability of a type-I error, i.e. that the null hypothesis is true but is being rejected. However, the p-value is actually a random variable. A nice paper covering this can be found in [1].

p-hacking also seems to be an abundant problem, i.e. the replication crisis in psychology [2], and it's not necessarily done on purpose [3].

Regarding the threshold for statistical significance, there are arguments for abandoning the concept altogether [4].

[1] https://amstat.tandfonline.com/doi/abs/10.1198/000313008X332...

[2] https://en.wikipedia.org/wiki/Replication_crisis

[3] http://www.stat.columbia.edu/~gelman/research/unpublished/p_...

[4] http://www.stat.columbia.edu/~gelman/research/unpublished/ab...