This really undermines the "trust the science" narrative. It has become much more about cushy white collar jobs, than the brightest minds being laser focused on understanding and bettering mankind.
Not really, but it does mean you shouldn’t trust individual papers blindly.
Anyone who follows research already knows this. Individual papers appear all the time with remarkable findings which seem revolutionary, but then nobody can replicate or commercialize it.
Some communities eat these isolated results up, like supplement and health podcasters (Rhonda Patrick, Huberman). They should know better than to take some random mouse study at face value, but it’s too good of a story to pass up.
In medicine and the industry, anyone experienced knows not to get excited about singular results unless it’s from a trusted source or until it’s replicated.
In my opinion, the three research frauds of the past 30 years that have had the biggest impact outside of academia are:
1. Andrew Wakefield's autism paper
2. Elgazzar's paper on ivermectin as a treatment for COVID
3. Marc Tessier-Lavigne's Alzheimer's papers
The interesting thing about 1 and 2 is that yes, they reduced public confidence in science, but only because there are large blocks of people who remain convinced that the research wasn't actually fraudulent.
But there are many scientists that love hearing themselves on TV etc. that pull out a paper to shove whatever agenda they have.
First problem: the sort of people who popularized Follow The Science™ actually meant "follow the people who run public sector institutions". Not "follow single studies" or anything that deep. The catchphrase was deployed many times during COVID to browbeat people into obeying people like Anthony Fauci, a man notorious for claiming he literally was science. "Attacks on me are attacks on science", he said. It was pure authoritarianism dressed up as intellectualism, as was so often the case throughout history (see: Lysenko).
Second problem: the heuristic that you should only trust a claim if it appears in several papers will also lead to you trusting many bogus claims. Academics regularly cite work they haven't even read, let alone replicated. It's easy to find invalid citations in the literature. Most of them just take claims on trust or even cite claims clearly advertised as made up fiction (sorry, "modeling assumptions").
Third problem: many results that replicate shouldn't:
https://blog.plan99.net/replication-studies-cant-fix-science...
This in turn allows certain people in institutions to trade legitimacy for agenda pushing, creating the current crisis of institutional trust, which the legitimate "Trust the science" crowd suffers from.
Science itself is all about testable hypotheses and facts. If we can't rely on it, chances are we are doing something wrong and science itself is perfectly fine.
Why not move back into a cave and stab your own meat with a sharpened stick next.
In many cases, citing articles still go to print media.
Almost all of the actual lab work requires statistical determination of repeatability & reproducibility to be calculated between different labs, and the summary is included with each document.
I would say there is way less than a kilo without this.
And the amount of supporting raw data on file amounts to kilos which dwarf the pages published. Formally accessible so everything can be thoroughly reviewed at any time in the future, allowing for complete reconsideration if called for.
Scietific instrumentation doesn't stand still.
So it definitely can be done. Even if it's to the extreme not suitable for everybody else.
The less-reproducible documents are there, they did the best they could, but have a smell not shared by the good stuff. You know "exactly" how good or bad the underlying science turned out to be in the real world.
Paradoxically, or intuitively, as the case may be, if you're going to utilize the less-reliable techniques (most likely because they're the best there is), you may need to know how bad they are most of all.
Maybe other publications should raise the bar on statistics as appropriate, I figure zero statistics is about as far as you can get from ASTM "standards".
Some places probably have a lot further to go than others and it would be nice to have a whole lot sweeter smell all around.
People can probably appreciate that lot of the kilos are more figurative than ever, but there's still enough hard copies made to fill big trucks though.
I'm not in academics professionally myself, how universal would you estimate it is among their journals these days to require a statistical study between an adequate number of different labs before final publication?
If they can’t reproduce the original, they should get called out eventually
Junk science has been around since, well, even before we coined the word science really. In some ways I think the situation has improved. People seem far more aware of misconduct, and willing to make retractions than they used to.
The idea that science is self-correcting is a nice fiction spread by academics to absolve themselves of the need to reform. "Don't make us change, any problems will go away if you wait because science is naturally self correcting", they say. It's a silly word game. Academia, which is what they mean when they say science, is a human institution whose incentives lead naturally to rewarding bad behavior. Nothing in that system self corrects, corrections must be forced from the outside (or we give up on reform and dissolve the whole thing, a solution now actively being talked about in hardline right-wing forums).
Example: pre-registration does happen sometimes today, and it always happens for corporate clinical trials. It's a good idea that works. In the private case the FDA is the auditor and there's some circumstantial evidence that pre-registration may be the reason why pharma productivity flatlined around ~2000. But in a surprising number of cases in academia a study is pre-registered, the final paper doesn't match the pre-registration and nobody notices. Why? Well, who is responsible for systematically checking these things? It'd have to be auditors, but universities point the finger at journals and journals point the finger back at universities.
Papers calling out problems: it's been done for much longer than I've been alive. Doesn't work. People nod their head and say something should be done, then go right back to the bad old ways.
The only thing that can help is external force. Governments have to either defund academia and let science be done by the private sector, or they have to tie funding to passing rigorous external audits (by private sector actors paid to find fraud, not academics themselves). The latter has been tried a few times, the US ORI is an example of that, but it doesn't work. The ORI was set up on the assumption that academic malpractice is rare so its methods don't scale.
But the findings are often not replicated.
But I think the worst are the Science Fans who go around saying "Science, bitches!" just because something has a two-column layout and never actually read the shit. Then they go around saying "There is some evidence that" and they've never read the paper.
In fact, there is so much bullshit that you would never accept if you were the one making a decision. It's why so many outsiders frequently make the right decisions on things.
e.g. Bezos closed flights to/from China to/from his offices a month before the pandemic. Bezos isn't a scientist. Trump tried to do the same. Trump isn't a scientist. But what did the scientists say? That it would hinder a coronavirus response: https://www.statnews.com/2020/01/31/as-far-right-calls-for-c...
The "far right" non-experts said one thing while the scientists said another thing and now we know that the "far right" non-experts were right. It's not even because the "far right" has some exclusive access to truth. It's just that there is a certain kind of blind spot imposed upon science by the fact that scientists have certain political persuasions. This causes them to self-censor and blatantly lie so as to preserve those politics.
Even if some racist view were true, you would be hard pressed to find a liberal scientist who would espouse it. Even if some sexist view were true, or xenophobic view were true, or whatever. Which is a dirty stain on science as practiced today.
If a thing is true, it is true.
Even more like undermining trust in journals.
With Science.org doing it to its own self.
Which should be living up to a higher standard than most so they can set an example.
At Science.org I think they have the responsibility to reverse the forces that are trying to build hatred for scientific endeavors across-the-board.
Looks like the haters are most concerned with climate and vaccines, and are perfectly willing to let everything else go down the tubes if that's what it takes to purge every last trace there is concerning progress in those areas past, present, and future.
So when you get the direct opinion of the most hateful where they explain why they don't trust vaccine and climate research, they can often further explain why they don't really trust very many scientists at all, not even NASA, so this is where the most concentrated anti-scientist effort can be found.
The worst of the haters are the ones painting with the broadest brush.
This fraud needs to be exposed, but the way it looks to me it just gives haters another arrow in their quiver.
This article looks very well researched itself and seems like the authors are on the right track and will only gain more expertise on the subject if they try.
What bugs me is that they're painting with as broad a brush as you can get, themselves.
With the resources at their disposal, they should be breaking down in great detail which fields of science are the most effected and least effected by the growing fraud industry. The numbers of scientists in these fields and the locations their research was conducted in, as well as the estimated fraud in each of the fields. We want University and institution affiliations, correlations with educational histories, numbers, why not?
If they try, and stick with it for a number of years, progress in one direction or the other should become measurable for each field.
So the most egregious fraud can be targeted first & strongest.
If there was a time when there was no fraud "industry" at all, I would say that some of most questionable findings were still heavily concentrated on the social sciences, and least of all on the natural sciences.
It would be good to know if this trend still holds based on true statistical data, this could indicate if or when the fraud industries are disproportionally targeting natural science. It would be good to have a sign whether fraud in natural science is on the increase or not independently, whether initiated for the purpose of promoting the lesser scientists or their institutions, or perhaps a source of uncertainty & doubt that can be put on steroids without any intention to promote scientists or their work at all, just the opposite effect could be intended.
You know natural sciences like climate and vaccines are in their cross-hairs so you can expect those to take a big hit without any proven or rumored fraud, but if you're not careful everything else will be equally destroyed even though it was not the primary target of the hate.
One of the worst ways to discredit natural science is to lump it in with social science or anything else.