Scientific Reproducibility: Begley’s Six Rules

From the article:

----------------------------------

1) Were studies blinded?

2) Were all results shown?

3) Were experiments repeated?

4) Were positive and negative controls shown?

5) Were reagents validated?

6) Were the statistical tests appropriate?

----------------------------------

Let me add:

7) Did the published work address the same topic as the original grant proposal?

8) Did the researchers commit to publish regardless of the study's outcome?

9) Did the granting source agree not to interfere with publication regardless of the study's outcome?

10) Did the researchers publish their data along with their results?

These are all serious problems in current research practice.

This is a very important post about the many gee-whiz reports we see each week about some new medical breakthrough. Stories like the ones analyzed in the post kindly submitted here regularly make it to the front page of Hacker News, and we all need to practice fact-checking such stories. The low rate of replication of eye-catching stories suggests that if a story reports a great breakthrough, it's better not to submit it here until the breakthrough, if it really exists, is just one more well established fact of medical practice found in medical textbooks.