The coin example makes perfect sense to me and I had no idea what a p-value was before I read this article.
Here's an intuitive explanation: a p-value is the probability of getting your experimental results given that your hypothesis is wrong.
Here's an intuitive explanation: a p-value is the probability of getting your experimental results given that your hypothesis is wrong.
That's a common misconception. Actually it's the probability of getting your experimental results given that your null hypothesis is right.
Consider an experiment where your hypothesis is that cold temperatures cause the common cold. This is a good example for a thought experiment because we "know the answer" in a way (there have been a lot of experiments on this). The null hypothesis in this case is that cold temperatures are uncorrelated with incidence of the common cold.
You place people in isolation in cold areas and a control group in warm areas, and study how many get the common cold. None of the people who didn't already have colds get the common cold: because they are in isolation and the common cold is caused by rhinoviruses (which they can't get because they are in isolation), you get exactly the same results.
This disproves the hypothesis, but it does not prove the null hypothesis, that cold temperatures and the common cold are unrelated. Cold temperatures are, in fact, related to the common cold.
Try a second experiment: you place people in groups of five in cold areas and in warm areas, and discover a moderately high correlation between cold temperature and incidence of the common cold. This disproves the null hypothesis. But the simple hypothesis that cold temperature causes the common cold has also been disproven by your first experiment.
The reason for this is that the correlation between cold temperature and the common cold is a dependent correlation: given that rhinovirus is present in the system cold temperature is correlated with incidence of common cold (rhinoviruses reproduce ideally at temperatures significantly lower than human homeostatic temperature).
The null hypothesis is not just a statement that there is no independent correlation, it's a statement that there is no independent or dependent correlation. As such, the null hypothesis is an extremely broad hypothesis which is impossible to practically prove. This is why there's such a focus on finding correlations rather than finding non-correlations: you aren't going to prove the null hypothesis.
Then research with high p-value would just mean "we tried, and we still know practically nothing".
The p-value won't tell you if the coin is fair or not, but it can tell you the probability that the coin is fair.
I think that's sort of the point that the article is making actually. That high probability does not imply truth. There are other non probabilistic ways to verify that coin is unfair, for example by looking at the density throughout the coin.
Even if you look at the density of the metal throughout the coin, there's still a chance I've altered your device to report the coin is fair. Or a passing microsingularity decided to play games with the scanning beam. Or you're just imagining the whole thing.
That's not to say one should despair that the world is unknowable. One only has to get used to the fact that, in practice, "true" just means "extremely, extremely likely".
But yeah truth is tricky.
Suppose I have a coin and flip HHHHH. Can you tell me the probability the coin is fair? No, it's fundamentally unknowable. We can say that a fair coin would have a 3% chance of flipping HHHHH (the p-value), but we can't say with what probability our coin is fair.