> It’s also led to an undercurrent of anxiety among scientists who fear that their labs and their publication records might come under attack from a feisty cadre of freelance critics.
Formerly known as "peer review".
Formerly known as "peer review".
That being said, laypeople can (and should) absolutely call out research malpractice like what's described in the article.
You've got worldwide universities churning out masters and pdh-level educated persons, who mostly go to the private sector for employment. Many of those would be perfectly able to perform peer review. And with this p-hacking crisis, mostly due to abject statistical knowledge, there are literally millions of technically able non-scientists able to review this nonsense. Any physicist, econometricist, statistician (and >100 more technical fields) could kindly take a shot at the social science methods in dispute here.
Sounds about right. How about we crowdsource the task? Since all the qualified specialists are busy writing grants against a 15% payline, we can farm it out to people without those pressures.
Oh... wait...
But you do make a convincing point. The 'stink' that is uncovered nowadays is mostly in those sciences where the layman / statistically schooled can make a difference. We hear a lot less about the replication crisis in those sciences where one indeed needs to have a lot of background knowledge before being able to do peer review. That's a problem.
There is no reason to assume that the same problems are not paramount in cell biology and other fields. We actually have enough reason to believe that many trials in medicine are flawed. Given that researchers in all fields face the same incentives (publish or perish) we are probably looking at at least 20 years of false, peer-reviewed and published results in all fields. That's a huge problem for mankind.
An in-depth review of a paper in advance of publication is common (although that system isn't as reliable as had one been assumed). Such an exhaustive review after publication is rare, for multiple reasons.
One serious problem with post-publication review is that the original data are rarely made available to reviewers. For example, academic critics of the work of Diederik Stapel couldn't justify revoking his Ph.D. because the original data that might have proved his fraud no longer existed. He eventually voluntarily gave up his degree[1], possibly to try to avoid a deeper investigation into his behavior.
1. http://www.science20.com/cool-links/dutch_psychologist_diede...
As opposed to something much closer to "LGTM!" With some "but be sure to cite my vaguely-relevant work in Foo et al, 1952".
Your objection is to a position I never took -- in fact I suggested the opposite. The depth of a review is gauged by its length, not its accuracy or objectivity.