Researchers taking on fraudulent science
analystnews.org
analystnews.org
At the same time I don't think any of this really addresses the root causes of the problems, which are career, university administrative, and field (i.e., recognition and attention) incentives. The whole paradigm we use, and the way we treat scientists and fund research, is broken. It feels a little bit like bailing water out of your boat with teacup on one side while ignoring the gaping hole letting rushing water in on the other side.
Whenever you have this situation you inevitably lead to selection criteria which can, and will, be gamed.
The only effective solution I see is making fraud harder to hide. (There’s also making the penalty for fraud higher and trying to instill integrity, but the problem is some people will still take the risk.)
This is absolutely false in real science and that you say it seriously is proof of just how far gone the academy is. Needless to say no matter how much you fake your data or results, nature is not so easily fooled.
The reality is these negative results are just as useful and can either help other researchers avoid dead ends or they might have a not great result but with some refinement to methodology might be worthwhile for further study.
Researchers should be judged on the quality of the research they perform, not their outcomes. A well crafted study that shows that further research shouldn't be done should be just a well received as a well crafted study showing more research should be done. The focus should be on the design of the study first, since no matter what the results it adds to the corpus of knowledge in a meaningful way
2. Letter-writers have no control over the work of the person they recommend. They cannot prevent the submission of a manuscript or a grant. They would be taking a lot of risk for no gain.
3. Letters of recommendation are a form of corruption and nepotism. If you would have a conflict of interest for reviewing a manuscript or a grant, you should not be allowed to write a letter of recommendation either.
Only in academia is fraud treated as a faux-pas rather than a crime. We have accepted a status quo in which academics and academic institutions play by a different set of rules to the rest of society. As a result, hundreds of millions of dollars are being wasted and patients are suffering real harm. Academic independence is valuable and should be protected, but independence does not mean impunity.
We just need to make sure the salesman scientists and parasitic beuracrats don't squash the efforts of those doing real work.
Just the same as in business. We need a new more prestigious stock exchange for companies that are willing to open their books up to continual audit.
The government can step in to aid businesses and researchers that act openly, in the form of grants to researchers or tax breaks for members of the more restrictive stock exchange, and shift the cost of policing destructive fraud to rewards for constructive conduct.
Real science is full of messy exploratory work. It's sometimes done remotely, and often in short stretches regardless of what time and day it is. And you usually don't know in advance if it's going to be relevant.
At some point, you may start believing that you have discovered something other people may care about. Then you start thinking about the normative experiments that are supposed to demonstrate the result and perhaps even validate it. But at that point, most of the work is often already done. The discovery has already been made (or at least you believe so) and you are just trying to replicate it. Maybe you could record those experiments if you are really into it. But the recordings would likely be meaningless without the actual written report.
I don't think it's fair to paint recordings of exploratory work as meaningless. You obviously have some interest in it or else you wouldn't be doing it. If you don't share that experience, wouldn't someone else spend time trying the same thing?
Computers have made science boring to watch. Many things are now automated and run 24/7. Even if you happen to be on site, there may not be much to see.
In many situations, automation also makes doing things on your own easier than trying to determine if someone has already done similar experiments. If you are doing exploratory work, there is often no established terminology for the topic. Even if a report of prior experiments exists, it can be difficult to determine if it's discussing the same thing. (AI could plausibly help here.)
Scientists need to care about the truth and most do care. The problem is both that as "publish or perish" becomes more and more literal people feel a strong incentive to cheat and moreover we have increasing cynicism as the system treats a given scientist as disposable.
Constant surveillance wouldn't address that, if anything it would increase the cynicism. Just much, no brilliant scientist is going to work under constant surveillance so you'd drive the most serious thinkers out of any given field.
I think streaming or recording would decrease cynicism. The openness would allow scientists to avoid false idolatry of people who juiced their citations with fraudulent research. And the cost of dealing with the fallout from fraud would be eliminated. I suspect that the fraud itself is a big accelerant of the hypercompetitive publish or perish phenomenon.
Maybe some funding of a transparent program could be allocated toward severance for scientists who honorably choose to "perish". Of course, perishing in this case means still being able to live as a civilian outside of academia. And that person can still speak to those in charge of research projects and benevolently influence the arc of human understanding, which for most good people is enough of an incentive to continue living and contributing.
Folks who wouldn't want to be transparent in their methodology could still do science the old way, they just wouldn't have access to the extra funding.
Especially if that recording is publicly accessible.
There are basically 3 sources of funding for universities - government, military, corporations. These 3 have aligned interests. Where their interests align (or are not in conflict) you will have funding of one side only. If you fund one side only, you will generate lots of information to support whatever-it-is, but this will fail to be a fair representation of the issue.
I'll just run through a simple, silly example. If I wanted to prove cats cause stormy weather, I could fund say, 10 studies. Perhaps 2 are supportive (cats and storms correlate in some way). I can then fund even more studies in that direction. I would soon have a huge body of supportive data .. but I haven't said anything meaningful, I haven't got to the truth of the matter. I just have a huge body of evidence.
To me, this is chicken feed. It's not that the information is a lie necessarily, it's just that it's isn't important. Nevertheless, the body of data can be referred to, in order to help steer public opinion. Everyone is getting paid, the thing is true .. what's to be upset about?
Anyway, whoever has the money determines the outcome. Truth is a secondary consideration.
Permit all the cronyism and gaming around university hiring and admissions, and don't hold universities responsible for unethical behavior of their faculty and graduates, then maybe this is what you get.
And since this is HN casting stones in glass houses: Permit all tech industry fratbro conventions around hiring (e.g., some that originated as naive and classist "did you take the same Stanford classes that I did", then were cargo-culted), and don't hold the companies responsible for the growth pump&exit investment scams, nor the criminal incompetence around infosec and correctness, then maybe that's what you get there.
Maybe, maybe not. It feels possible at this point that the entire top of the competitive curve is actively juicing. Ie, with such fierce competition, there could be a vanishingly small cohort of researchers who both operate with integrity and whose CVs are competitive for faculty positions.
At best, we might expect a removed fraud to be replaced with someone marginally more honest.
The other front is to let the fakers keep faking, but change society so that everyone who wants to research out of personal interest has the free time to do so. They'll have less incentive to fake.
I think it's fallacious to compare the CV "competitiveness" of a falsifying 'scientist' with a legitimate scientist.
Not that that's what you were suggesting, but maybe we haven't fully accepted what it means to combine the typical daisy-chained/chain-reaction growth of CV opportunities (one prestigious-looking line item opens a door to one or more others), with cheating offering additional boosts as needed at each step.
When an athlete "juices", they can perform with greater strength or endurance.
But when a scientist falsifies research, they aren't performing greater science -- they're performing greater CV (and terrible science).
Competition for faculty positions shouldn't be CV, but the potential for great science.
If an institution wants to use CV as the proxy for science potential, then they have to address that the CV metric is being gamed like crazy, including by falsifying (i.e., terrible) scientists.
Because cheating without getting caught will always be the "best" strategy within the confines of the game alone.
However, that's likely the worst strategy for society as a whole.
I'd go so far as to say that the mere existence of an "exit plan" is an indicator that the purported product may be flawed (the business may not be flawed, from an investor's PoV, but to them, the "product" is actually the business, itself, as opposed to what the business says is their product).
IMO: Real businesses are meant to be ongoing enterprises. Being sold to others, is supposed to be an unplanned (possibly fortunate and hoped-for) event. It shouldn't be an inevitable business plan milestone.
Sure, rules that make it harder to buy companies might slow down the volume of businesses being created due to less investor interest, but I'm not sure that decline would surpass the slowdown in fradulent exit schemes.
The lack of training social scientists in cell life for a bachelors, or the biological sciences in electrical signaling, or the physical scientists in their necessary inferior perspective of natural changes because of measurement imperfections by their instruments and their body’s senses, is all telling for a community which has been avoiding the successes of electrical engineering for over a century, likely because they don’t want to do that math ;)
This is a huge problem. When fraud is rewarded over legitimate research we lose so many times. Once when we reward the fraud, again when the fraudulent research is propagated into the corpus, again when we recognize that this has squelched legitimate scientific research in exchange for resources being sent to the fraud. And finally when trust in science falters.
There's every chance that the next great scientist is flipping burgers at McDonald's.
The problem is much bigger than anyone wants to admit. In some domains a coin flip is just as good as relying on peer reviewed research.
After all, being anti-establishment is often associated with scientific quackery.
I’d generalize this further and say we have an underlying economy and society that rewards fraud over what should be happening widespread. It’s happening everywhere and lying seems to be one of the best strategies to success. With competitive pressures increasing, it’s often even worth the risk. This may have always been the case but the population at large seems to be more cognizant of it and taking action on it. Politics, business, everywhere deceit is rampant. It may in some cases be “allowed” forms of lying or skirting around the truth but underlying behavior is the same here: avoiding the truth for one’s own self gain. There’s few if any establishments I trust anymore. I just assume everyone everywhere is lying for their own goals.
Sorry, but that's tin foil hat territory. Everyday I trust my bank to handle my money fairly, the gas station to give the right amount of gas, the labels on food and medicine to be accurate, the weather report, etc. etc. I am sure there are problems with virtually every institution that need to be addressed. But I am surrounded by reliable information and trusted actors.
You can totally inspect your olive oil yourself. First a primer: Extra virgin olive oil is the highest grade. This is unregulated and mostly nonexistent in most US grocery shelves.
Here's the criteria I recommend:
- Source. Most Italian olive oil is repackaged Tunisian, Turkish, Spanish olive oil. Which isn't a bad thing but consider why they're not consuming it themselves (Italy and Greece do not produce enough to meet domestic demand). You'll see this by abbrvs like Source: "TN SP TK" (Tunisia, Spain, Turkey). Look for single source oils. Consider Californian oil (if in the US). A country does not produce better olive oil than another, it's all up to individual producers. Some of the best oils I've had are from Australia, California, Chile
- Smell! Does it smell like wax/waxy crayons/Crayola? It's probably a bit rancid/old. Not necessarily bad for you but lacks the health benefits (it's not extra virgin at this point if it ever was). It should smell fruity/grassy/green. - Taste: It should be bitter, fresh, and tickle your throat (that's the antioxidants!). It shouldn't leave your mouth feeling greasy.
- Harvest date: Oil degrades quickly. If there isn't a harvest date, it's probably not great. Should be harvested within the last 2 years, ideally within 1 year.
Extra virgin has the maximum taste and health benefits. Most grocery store oils are simply olive oil - not the highest grade but still significantly better than seed/nut oils.
Source: I'm an olive oil somm
Online is always an option :) The Olive Truck Tuscan blend is a personal favorite: https://bestoliveoils.org/search
The owner is this awesome Turkish dude who drives around California in a mobile olive press and makes oil from olives on the spot!
BUT BUT BUT make sure it's NOT the "global blend" - they got in a lot of trouble in California for trying to pass it as good oil. It's trash. Blends are usually made to maximize profit which minimizes the flavors and benefits of good oil. Look for their oil from California in the glass bottle.
If you're in California, the Bariani brand is affordable and awesome.
Labels on food? For ingredients? Again, sure. But where is the label for "the taste of this product has been engineered to increase your consumption of it. Your health? Ha. Forget about it."
And so on.
You're trusting the sheeps' clothing and are all but ignoring the wolf that hides beneath it.
What I'm getting from the article so far is that "the establishment" is actually a bunch of establishments (plural). And thus there are many systems to game, and many incentives to game them.
Otherwise the fraud would be so pervasive that anybody with a PhD attached after name should be distrusted unless proven otherwise.
1- The article give pure numbers and does not report ratios. So when they report
> Last year, more than 10,000 research papers were retracted, marking an all-time high
The source is nature publication and while it gives rate of retraction per year it doesn't show the ratios. So if this is 10,000 out of 5M or 10,000 out of 10M is huge different. Yes in both cases it will be a problem but also there are no information about re-traction cause. There is no statistics on cause of re-traction. Not every retraction is necessarily because of fraud.
Actually in the nature paper they reported the retraction rate (without any other information) to be 0.2% in 2022 but again without more information.
2- calling it epidemic is clickbait title. Depending on retraction as a measurement of fraud problem in science is subjective. There are problems with citations, paper mills and many other things. But you can't put all of these things into one basket. Retraction is a standard process that even legitimate researchers are using sometimes to fix problems with the paper ranging from discovering malfunction device after publication to method not working. Associating retraction with fraudulent activity only would mean that a researcher wouldn't use it if the perception that if you do it yourself then you are trying to cover something or you have ethical problems.
This shouldn't be treated as a mathematical problem. The consequence of just one of this 10.000 articles can lead to thousands being killed or suffering traumatic useless treatments that seem designed by sadists. A single article and people still are told decades later that vaccines cause autism. Other, and parents will start believing that bleach enemas will cure their autistic children. Other and millions of dollars will be lost. Spreads like fire and is simply evil.
And when you'll eventually convince everybody that those ideas are nonsense you'll have still 9997 fake articles more to fix.
https://freebeacon.com/?post_type=post&p=1854123
They haven't even made a case that it is at an all-time high as a rate of anything. The article also says that output has spiked.
But you can’t tell anyone this because they’ll circle the wagons. “Oh yeah, you think science is blah blah blah”.
See also https://datacolada.org/98
I had read GP's link prior to this. It's about a researcher claiming that signing a truthfulness statement beforehand reduced lying, but that very research was tampered with.
It was assumed by default that scientists don't lie (and researchers are signing their articles since thousands of years). Innocent until proven guilty is a better system, but... lets add even more bureaucracy to publish, is not insulting enough yet.
If you haven't yet, GP's link is humorously topical.
Scientific fraud isn't going anywhere, and neither is academic fraud. In fact, it will only continue to worsen as most of the Global North is in a post-Truth, post-meritocratic society. Heck, tenured professors and administrators often will not even be censured except in the most extreme cases, let alone lose their jobs, let alone ever see the inside of a prison cell.
"Hmmm, these experiments that I have spent several years working on, and are the basis of my entire thesis, didn't work out. If I report the correct results, my entire project will be worthless and I will not earn my PhD. Too bad I wasted 5 years of the prime of my life pursuing this hypothesis which my supervisor insisted I investigate. I hear Costco has good benefits, I wonder if they're hiring?"
or
"Hmmm, if I just delete some outliers, cherry pick some experiments, ignore others, nudge a few of these data point in the right direction, and choose this specific statistical analysis, I can show a positive result, which means I can graduate, start my life and family (finally), get a job (maybe in academia, maybe not), and fulfill my dreams."
As I said, I am not condoning fraud, but the discussion of solutions need to focus on the right problem. Most discussions seem to imagine a professionally established scientist whose prestige and job promotion depend on good results. In reality the stakes are a lot higher if you realize that these results of literally life-changing for the graduate students doing the work. I guess I'm saying there is a human side to this issue beyond greed and prestige seeking (although there is plenty of that too).
From now on, all scientists who want to publish science, they publish their work in the form of a video. They film themselves doing the science and then put that video on YouTube, or maybe some science specific version of youtube. Other people can then watch the video and determine for themselves if the science is valid or not.
Its much harder to fake science if there is a video involved. Its too easy to make fake science if all you have to do you "prove" it is write some words.
On second thought, maybe I could make a video of how I manage all of those dirty LaTeX tricks to squeeze my paper into less pages, at least there's some tangible payoff there...
The point is to show your work. You film yourself setting up the experiment to prove that you actually did the experiment. You also film yourself collecting the results, proving that you actually got the results you claim.
The only kind of science that consists of "just writing the paper" is math. Math will be exempt from this requirement. Fake math is not an epidemic
Folks don't have the time to read the literature thoroughly as it is. That's one of the reasons the junk gets published. So this is not practical.
Plus, imagine reporting to your funding agency, Dean, or boss, that all you accomplished this year was to watch a few videos. No new work of your own. I guess you would make a video of you watching a video. And then someone would watch a video of you watching a video ...