Hacker News Reader

Comment by learnstats2 | Hacker News Reader

I understand that people generally mean 'null results' here, by your definition, when they say 'null results'. That's the intent.

Null results are also important.

Suppression of null results allows for p-hacking and confirmation biases to creep into research, and greatly reduces the power of literature reviews.

View on HN

True, but I'm not really arguing against what you're saying. It is true that, when you have a positive (or a negative!) result you should also report on the nulls you obtained on the way (most likely in the supplementary materials) as a compendium of the result, to put it into context.

What I'm arguing against is publishing a null result as a stand-alone publication. This creates the illusion of it being somehow a "result", which is not (in fact, we should stop calling them "results" altogether). With a null you haven't proven anything, and thus it is not a sufficient basis for a publication.

I see. Thanks for adding to the clarification. I think that the presentation of nulls as "results" can definitely be disingenuous. Ideally, science would have a better database to keep track of what people find, where we could add nulls in a way that doesn't highlight their "importance". As the person above says, reporting nulls is still useful to prevent p-hacking and publication bias.

(Of course, ideally I think we'd be better off focusing on reporting the data in a Bayesian approach, but that hasn't really gotten traction in the broader community.)

Reply on news.ycombinator.com