I'm completely lost here. How is 0.005 "dead center"? Are you assuming p = 0 is the center? Are there negative p-values I'm not seeing that somehow balance the positive ones?
How can a random variable that's strictly between 0 and 1 even follow a bell curve?
[0] http://www-ist.massey.ac.nz/dstirlin/CAST/CAST/HtestPValue/t...
Am I severely lacking sleep and going crazy or something? Maybe I should check back in like half a day to see what people have said, I feel like I must be completely confused right now because literally nothing I've read so far makes sense to me.
Here's an R example to play with:
pvals <- replicate(10000, {
x <- rnorm(100)
y <- rnorm(100)
t.test(x, y)$p.value
})
plot(density(pvals))
That will plot you a nice uniform line on [0, 1].(NB: I have no idea why OP talked about p values following a normal distribution. That doesn't make sense to me, and I think the post has been deleted.)
<< HypothesisTesting`
With[{n = 10000000, dist = NormalDistribution[]},
Histogram[Last[NormalPValue[RandomVariate[dist, n] - RandomVariate[dist, n]]], 500]]
Why is your x variable though? If H0 is true shouldn't your x be fixed?I think if you remove the subtraction though then you do get a uniform distribution -- in which case I see what the claim is, yeah. Wasn't really clear to me earlier but indeed, getting p = 5% means you have a 5% chance of getting observations that extreme, so I guess it is uniformly distributed!
(Disclaimer: I am not a real statistician....)
That's the very definition of a p-value! The mapping of data to p-values is chosen to have a uniform distribution of p-values when the data is distributed according to the null hypothesis. That's the property that makes p-values interesting.
> a P < 0.05 means that there is less than a 5% chance that the null hypothesis is true.
In other words, P(H0 | X) where H0 is the null hypothesis being true and X is the data observed. But that is not what a p-value is, they actually represent P(X | H0).