From the article:
> Retractions remain exceedingly rare for scientific papers: Just eight out of every 10,000 are retracted, according to a Retraction Watch database.
Think about how much attention has to be paid for retractions to happen, how long the work existed without retraction, how difficult it was to get a retraction for outright fraud (ignoring work that itself was poor quality). This maybe says more about the incentives against retraction which then explains the low retraction counts better than whether or not the remaining papers are good. After all, the papers that have been retracted are high citation count papers that are foundational for certain avenues of research (this one for Alzheimer’s, the other famous fraudulent work about honesty pledges, etc). What does this say about all the papers that then cite these papers as work they’re basing theirs off of. What about the papers that then “agree” with papers that turn out to be completely fraudulent?
You're just making the data fit your preconceived beliefs, which would is how we get fraudulant research.
What? this is literally what happened (someone pushing hard to challenge the results) for these 4 retractions related to this guy's work. You don't get a retraction just by posting a comment on HN or something.
You’re the one making definitive statements of a position and then claiming the data backs up your position and then arguing with anyone who posits that your interpretation of the data may be flawed.
Elizabeth Bik has said:
“Science has a huge problem: 100s (1000s?) of science papers with obvious photoshops that have been reported, but that are all swept under the proverbial rug, with no action or only an author-friendly correction … There are dozens of examples where journals rather accept a clean (better photoshopped?) figure redo than asking the authors for a thorough explanation.”
I wrote an essay about this in 2021. The photo at the start is indicative of the level of problem we're talking about here: you can literally just look at it and notice it's a Photoshopped photo, yet the journal contacted the authors who gave them a transparently nonsensical explanation, fully accepted without question. The people who did the fake research of course suffered no penalties and continued to be employed, most likely their employer never even found out.
https://blog.plan99.net/fake-science-part-i-7e9764571422
The retraction rate in science is small because standards are low, and whenever random outsiders stumble across problems the system closes ranks. Getting even one paper retracted is a giant struggle and succeeding doesn't cause other papers based on the first to be retracted either.
Consider that in psychology papers, basic statistical consistency checks of the numbers in them fail about half the time. Were half of all psychology papers retracted when this fact was discovered? Definitely not.
Also, bear in mind that journals and universities have very narrow criteria for retraction. They care a lot about plagiarism and data forgery, but if the paper is merely filled with nonsense then nobody cares. The DCM paper I call out in the blog post "calculated" that the average Brit lives in a household with 7 other people. This problem was noticed during peer review but never fixed, and the paper has been cited 75 times so far. It would certainly not be retracted if anyone asked.
There are many more papers that are less cited that will never get looked at in enough detail to discover fraud.
Shoddy science is (currently) not a reason for retraction. So just because a paper's results aren't replicable doesn't mean it should be retracted. Deliberate falsification is. But you'd need to distinguish those from not-perfect science.
Last, a sanity check: your comment implies/suggests that 1/3 to 1/2 of published science is fraudulent because it's not replicable. But fraudsters might accidentally hit upon the truth - at least sometimes. Moreover, plenty of research answers a binary question (yes/no, which of the two has a greater effect, etc). For those research questions, just guessing the answer gives you a50% chance of being right.
All of that implies that, following your logic, a huge portion of published research is based on fraudulent practices - not mistakes, but deliberately falsified contents. Add to that the "meta-fraud" (playing with authorship, citations), and one has to wonder if every second scientist on the planet is cheating and faking everything they publish.
To me, that is an obvious sign that the reasoning went off rails somewhere - much like the result of a sine function being outside of [-1, 1]. So no, you shouldn't expect 1/3 to 1/2 of all published research to require retraction.
The reality is we don't know. Should it be more than 8 out of 10,000? Almost certainly, since quite a lot of papers barely get any attention after publication.
But should the number be 10 out of 10,000? 20 out of 10,000? 100 out of 10,000? We don't know.
And boring papers without any interesting conclusions published just for CV reasons also don't really matter, and the 10,000 figure is also wrong, and we also don't really know what the correct value for that figure is.
To put a figure on it, consider that about 0.04% of papers are retracted but about 2.5% of scientists admit in polls to having engaged in fraud. Obviously though most fraudsters won't admit to it (they often convince themselves it's not fraud via mental gymnastics), so another way to approach it is to ask them to estimate what proportion of other scientists commit fraud. Then they answer 10%.
It's a bit hard to know how many papers are published per year, I've seen estimates of anywhere from 2 to 5 million. If 10% of scientists are committing fraud and they were all detected, that would yield on the order of maybe half a million to a million retractions per year, as we'd expect fraudulent scientists to be much more productive than the ones that play fair.
But fraud isn't the only reason that papers should be retracted. You'd expect serious non-fraud problems to be a reason too. If papers were retracted for all the reasons non-scientists imagine papers are retracted, e.g. being foundationally based on another paper/dataset that's been retracted, using invalid methodologies, internally inconsistent statistics, serious software bugs etc, you could probably get up to 50% or more in some fields.
This sounds absurdly high but is the number estimated by Richard Horton, editor of The Lancet. And according to Marc Andreessen, when he asked if it could be true that half of all biotech research was fake the head of a major US funding agency actually laughed at this figure and said no, in biotech it's more like maybe 90% is fake. So the problem does seem to be huge.
It has become a LOT easier for non-experts to hunt down fraud as electronic dissemination has widened.
> Retractions remain exceedingly rare for scientific papers
This retraction actually happened because someone BOTHERED to check the results and push for hard questions.
You can bet that this does not happen for every publication... So...
> [Elisabeth] Bik later told The [Stanford] Daily in an interview, relying on her experience of investigating more than 20,000 papers for research misconduct and causing nearly 1,000 retractions and roughly the same number of corrections to make that claim.
https://stanforddaily.com/2022/11/29/stanford-presidents-res...
Let's assume Bik chooses papers to investigate because, to a significant degree, her expert judgment is that they are most likely to have fraud, and Bik may know that topic better than anyone in the world.
Of the 20K most likely candidates, 5% were retracted and 5% corrected, and 90% apparently needed neither (though possibly they were inconclusive or she couldn't convince publications or authors to retract).