And that's without knowing how the study was really executed. Where I worked, there was a relatively successful postdoc who presented the results of his p<.05 significant pilot study. When asked, he said it wasn't the first pilot. It was the 20th.
When people asked around, informally what was said was that the grad students in the other areas (especially one area, in the experimental molecular biosciences) would leave after having to "redo" their dissertation over and over again. Essentially what would happen is they would propose a dissertation study, it would be approved by the area committee, the student would do the study, and it would produce null results. So they would be told to redo it a different way, or to pick a different topic, it would get approved, and the same process would happen again. After this happened a few times, with the student being told they had to produce significant results, the student would grow despondent and leave the program.
What's sad about this is that it's formally reinforcing p-hacking basically, as part of the degree program. But it's even more absurd than what's often alluded to in meta-science writings, because in these cases you would have a formal graduate committee, composed of faculty, deciding that the dissertation thesis is a good one -- that the hypothesis and design are solid, and formally approving the dissertation proposal -- and then because the results are null, it's unacceptable. If this was being done so casually in that forum, I can't imagine what goes on behind the scenes.
Getting a null result doesn't invalidate that in any way.
If you have a committee of experts who carefully evaluate a proposal and decide it's good, the results are as they are.
Broadening the discussion a bit, it seems one feature of science, as opposed to, say philosophy, is that the conclusions regarding a hypothesis are not knowable a priori. I think in contemporary academics there's some implicit idea that the quality of a researcher lies in their ability to identify hypotheses that are "correct", as opposed to simply following through with good but ultimately "incorrect" hypotheses. There's a bit of a roll of the dice involved with science; if there isn't, it's not science.
That should be a defining characteristic of any academic inquiry, regardless of whether it's science or not.
I have no training in non-quantitative fields, and my academic experience is from CS where the "science" part is often so-so. As such this should be taken partially as a layman view. However, my impression is that while in non-scientific academic fields the research isn't necessarily taking the form of explicit hypothesis testing, more or less similar criteria for intellectual inquiry should apply.
The research might be more about observation and critical (often non-quantitative and non-absolute) evaluation of arguments, and as such the validity of the methods (such as whether the hypothesis is assumed or genuinely questioned) might not always be as easy to judge [1].
The process might not be as easily formalized or judged as in science, but the mentality of critical inquiry should be similar. If the hypothesis is assumed and not questioned, that's no longer any kind of academic inquiry. It becomes politics, in the pejorative sense.
> I think in contemporary academics there's some implicit idea that the quality of a researcher lies in their ability to identify hypotheses that are "correct", as opposed to simply following through with good but ultimately "incorrect" hypotheses.
I think that's partially just psychology and human nature. We like results that make us directly know (or think we know) more, and results that basically tell us we still don't know less. Few people like uncertainty.
The society outside of the academia certainly values the former more than the latter, and funding and other external incentives probably exacerbate the underappreciation of negative results.
[1] Or perhaps it is, to an expert, but having that judgment would require the kind of experience in those fields that I don't have.
A medical student worked hard to analyze, say, 40 x-rays out of hundreds available. He found no significant evidence for some hypothesis. When he told his supervisor, the reply was: "Well then you should just analyze some more x-rays. I'm sure you'll have a statistically significant result at some point."