> A null finding might not be great for the researcher
The fix for that would be to change science funding so a null finding doesn't hurt the researcher's career.
The fix for that would be to change science funding so a null finding doesn't hurt the researcher's career.
There's something to be said for brute force on rare occasions though.
Let's say you try a particular educational intervention. it turns out that if has no effect on children's learning.
This is really useful information, because it means that we now know that there is no point in schools trying it.
By publishing null results you avoid 100 scientists going the same endearing but unproductive routes, which can also help build good hypotheses and speed up progress.
A null result (no significant evidence for anything, a p value above 0.05) is truly null. It doesn't even confirm the null hypothesis.
Common wisdom says that Statistical Hypothesis Inference Testing doesn't work because researchers engage in "P Hacking". That's a half truth, researchers really don't understand the meaning of the p-value.
The stronger "absence of proof is proof of absence" would be fallacious.
A more accurate way to state this is “Absence of evidence is not always evidence of absence”. Or “Absence of evidence may or may not be evidence of absence depending on the circumstance”, though it doesn’t roll off the tongue as easily.
"When trying to reject the null hypothesis (sensu Fisher), absence of evidence (for its invalidity) is not evidence of absence (of its invalidity)."
Fisher wasn't the only statistician, he wasn't the only confused one either. We can go with Neyman and Pearson instead:
"When trying to reject the null hypothesis with a sufficiently powerful experiment, absence of evidence is evidence of absence."
This requires a power analysis, which is something most studies lack. Constantly reminding everyone of these details and misguided philosophical differences is tiresome, so I prefer to go with Laplace, Bayes, and Jaynes:
"Everything is evidence."
Today, most science is funded by government agencies, like the NIH. This means, bureaucrats who know nothing about science allocate other people's money to research they don't care about. Since they don't understand the research, they need objective measures of "research quality", and they picked number of publications and impact factor of the journals as measures. What you measure becomes your objective, and we got "publish or perish" and all sorts of scientific misconduct.
Science needs a free market, too.
Five million euro of tax payer money have been blown on a Neanderthal genome. All we got out of it is a hyper sensitive analysis that picked up an artifact (source: first hand experience with the raw data) and called it admixture. But it's a high profile publication thanks to the catchy headline and more tax payer money flows in the same direction.
This wouldn't have happened if the bureaucrats who presided over the grant money knew anything about the science and looked into the methods. Or maybe it would have happened anyway, because this was never science but just a publicity stunt. Either way, funding "science" this way is bad.
A great introduction to the topic and a great general guideline. It's not a collection of recipes, more like a nudge in a direction where you don't need canned recipes anymore.
"Surely that only works if your prior belief in your hypothesis is neither exactly 0 nor 1?"
And that's correct. Prior distributions are never completely concentrated at either extreme, for those could never be updated. If your hypothesis is now totally wrong, your confidence in it (the posterior probability) will converge toward zero.
In other words, its generally much easier for your head to come out of the barrel without an apple.
I get what you're saying, but the counter to that is it would be too easy for scientists to conjure up studies that they have no reason to believe is true, do an experiment, publish the null finding (that frankly, they were expecting), and ask for more money.
It's almost trivial to think up experiments that will give you a null finding.
Unless of course you want to reward null findings which might, depending on the circumstances, open another can of worms.