Fraud, so much fraud

"Science" nowadays is mostly BS, while the scientific method (hardly ever used in "science" nowadays) is still gold.

What are you talking about? You _are_ successful. You're not a fraud like all those other tossers.

As I've gotten older, I understand that Charlie Munger's observation "“Show me the incentive and I will show you the outcome.” is applicable everywhere - including science.

Academic scientists' careers are driven by publishing, citations and impact. Arguably some have figured how to game the system to advance their careers. Science be damned.

I still feel that my bio work was far more important than anything I've done since, but over here the work is easier, the wages are much better, and fraud isn't table stakes. Frankly in exchange for those things I'm OK with the work being less important (EDIT: that's not a swipe at software engineering or my niche in it, it's a swipe at a system that is bad at incentives).

Oh, and it turns out that software orgs have exactly the same problem, but they know that the solution is to pay for verification work. Science has to move through a few more stages of grief before it accepts this.

Fraud I suspect is only tip of the iceberg, worse still is delusion that what is taught is factually correct. A large portion of mainstream knowledge that we call 'science' is incorrect.

While fraudulent claims are relatively easy to detect, claims that are backed up by ignorance/delusion are harder to detect and challenge because often there is collective ignorance.

Quote 1: "Never ascribe to malice that which is adequately explained by incompetence"

Quote 2:"Science is the belief in the ignorance of experts"

Side note: I will not offer to back up my above statements, since these are things that an individual has to learn it on their own, through healthy skepticism, intellectual integrity and inquiry.

My suggestion stands: Give true scientists the ability to hunt fraudsters for budgets. If you hunt and nail down a fraudster, you get his funding for your research.

On the other hand, shysters always project, and this thread is full of cringe vindications about cheating or faking or whatever. As your "emotions" are probably telling you, that kind of generalization does not feel good, when it is pointed at you, so IMO, you can go and bash your colleagues all you want, but odds are the ones who found results did so legitimately.

It happens everywhere.

The only solution to this is skin in the game. Without skin in the game the fraudsters fraud, the audience just naively goes along with it, and the institution collapses under the weight of lies.

Even in my immediate circles, I know many industry scientists who do scientific work beyond the level required by their company, fight to publish it in journals, mentor junior colleagues in a very similar manner to a PhD advisor, and would in every way make excellent professors. There would be a stampede if these people were offered a return to a more accountable academia. Even with lower pay, longer hours, and department duties, MORE than enough highly qualified people would rush in.

A hypothetical transition to this world should be tapered. But even at the limit where academia switched overnight, trainees caught in such a transition could be guaranteed their spots in their program, given direct fellowships to make them independent of their advisor's grants, given the option to switch advisor, and have their graduation requirements relaxed if appropriate.

It's easy to hem and haw about the institutional knowledge and ongoing projects that would invariably be lost in such a transition, even if very carefully executed. But we have to consider the ongoing damage being done when, for example, biogen spends thousands of scientist-years and billions of dollars failing to make an alzheimers drug because the work was dishonest to begin with, or when generations of trainees learn that bending the truth is a little more OK each year.

Naturally, they are now the head of AI where they work.

https://archive.org/details/Fantasy_Science_Fiction_v056n06_...

The good news IMO: more recent MolBio methods produce data that can be checked more rigorously than a gel image. A recent example where the evidence in form of DNA sequencing data is contested: https://doi.org/10.1128/mbio.01607-23

The only option for someone who wants to double check research is to completely replicate a study, which is quite a bit more expensive than double checking the researcher's work.

???

I think this statement is either meaningless or incorrect. At the very least your conclusion is context dependent.

That being said, I ran gels back in the stone ages when you didn't just buy a stack of pre-made gels that slotted into a tank.

I had to clean my glass plates, make the polyacrylamide solution, clamp the plates together with office binder clips and make sure that the rubber gasket was water tight. So many times, the gasket seal was poor and my polyacrylamide leaked all over the bench top.

I hated running them. But when they worked, they were remarkably informative.

But given that many people are saying that they noticed and quit academia, how about also creating a more direct 'whistleblower' type of system, where complaints (with detailed descriptions of the fraud or a general view on what one sees in terms of loose practices) goes to some research monitoring team which can then come in and verify the problems.

It seems amazing that they are pulling off what seems impossible.

Years later, we learn they really aren’t. They unjustifiably made a name for themselves by burning VC money instead of running a successful business.

In my experience no, most fraudsters are not evil people, they just follow the incentives and almost non-existent disincentives. Scientist has become just a job, you find all kinds of people there.

As far as I know no-one goes to jail, worst thing possible (and very rare) is losing the job, most likely just the reputation.

This kind of fraud happens because scientists are rewarded greatly for coming up with new, publishable, interesting results. They are punished severely for failing to do that.

You could be the department's best professor in terms of teaching, but if you aren't publishing, your job is at risk at many universities.

Scientists in Academia are incentivized to publish papers. If they can take shortcuts, and get away with it, they will. That's the whole problem, that's human nature.

This is why you don't nearly as many industry scientists coming out with fraudulent papers. If Shell's scientists publish a paper, they aren't rewarded for that, if they come up with some efficient new way to refine oil they are rewarded, and they also might publish a paper if they feel like it.

Once it became a distinguished profession the incentives changed.

"When a measure becomes a target, it ceases to be a good measure"

As a quick example, any book about electricity from the early 1900's will include quite serious sections about the positive effects of electromagnetic radiation (or "EM field therapies"), teach you about different frequencies and modulations for different illnesses and how doctors are applying them. Today these devices are peddled by scammers of the same ilk as the ones that align your shakras with the right stone on your forehead.

I guess it also reinforces the supreme importance of reproducibility. Seems like no research result should be taken seriously until at least one other scientist or group of scientists are able to reproduce the result.

And if the work isn't sufficiently defined to the point of being reproducible, it should be considered a garbage study.

For most of the people who end up in these scandals, this is just the day job that their various choices and random chance led up to. they're just ordinary humans responding to ordinary incentives in light of whatever consequences and risks they may or may not have considered.

Other careers, like teaching, medicine, and engineering have similar problems.

Indeed, I believe many academic careers were kicked off in this manner. Where it all goes wrong is when other more diligent researchers fail to reproduce said fraudulent research - this is what brought down famous fraudster Jan Hendrik Schön in the field of plastic-based organic electronics, which involved something like 9 papers in Science and Nature. There are good books and documentaries on that one. This will only be getting worse with AI data generation, as most of those frauds were detected by banal data replication, obvious cuts and pastes, etc.

However, when you add a big financial driver, things really go off the rails. A new pharmaceutical brings investors sniffing for a big payout, and cooking data to make the patentable 'discovery' look better than it is is a strong incentive to commit egregious fraud. Bug-eyed greed makes people do foolish things.

This is why I think the most effective way is to empower good actors. Ensure open debate, limit the power of individuals, and prevent over concentration of power in a small group. These efforts are harder to implement than you think because they run against our desire to have scientific superstars and celebrities, but I think they will go a long way towards building a healthy community.

Anytime you see champagne bottles up on a professor's top shelf with little tags for Nature publications (or something like that), then you know they are a glory hound.

When you see beer bottles in the trash, then you know they're in it for more than themselves.

Jobs at top institutions are worth much more than their nominal salary, as evidenced by how much those people could be making in the private sector. (They are compensated mostly in freedom and intellectual stimulation.) Unambiguously faking data, which is the sort of thing a bad actor might do to get a top job, should be considered at least as bad a moral transgression as stealing hundreds of thousands or perhaps a few million dollars.

(What is the downside? I have never once heard a researcher express feeling threatened or wary of being falsely/unjustly accused of fraud.)

Are you going to go to jail for fudging some numbers on your paper, not likely.

It's bothering you a rational amount, actually. These people have done serious damage to lots of lives and humanity in general. Society as a whole has at least as much interest in punishing them as it does for financial fraudsters. They should burn.

Studies that have not been replicated should be published clearly marked as preliminary results. And then other scientists can pick those up and try to replicate them.

And institutions need to give near equal weight to replications as to original research when deciding on promotions. Should be considered every researchers responsibility to contribute to the overall field.

Yes and ... we're already there.

Removing the government from science could help, not the other way around.

But there was no golden era of science to look back on. It has always been a wonderful productive mess—much like the rest of life. As least it moves forward—and now exceedingly rapidly.

Almost unbelievably, there are far worse crimes than fraud that we completely ignore.

There are crimes associated with social convention in science of the type discussed by Karl Herrup with respect to 20 years of misguided focus on APP and abeta fragments in Alzheimer’s disease:

https://mitpress.mit.edu/9780262546010/how-not-to-study-a-di...

This could be called the “misdemeanors of scientific social inertia”. Or the “old boys network”.

There is also an invisible but insidious crime of data evaporation. Almost no funders will fund data preservation. Even genomics struggles but is way ahead in biomedical research. Neuroscience is pathetic in this regard (and I chaired the Society for Neuroscience’s Neuroinformatics Committee).

I have a talk on this socio-political crime of data evaporation.

https://www.youtube.com/watch?v=4ZhnXU8gV44&embeds_referring...

If someone brings a plum to a market for lemons, they can distinguish the quality of their product by offering a warranty on its purchase, something that sellers of lemons would be unwilling to do, because they want to pass the cost burden of the lemon onto the purchaser.

The full paper is fairly accessible, and worth a read.

Not sure how this could be applied to academia, one of the problems is that there can be significant gaps between perpetrating fraud and having it discovered, so the violators might still have an incentive to cheat.

Is this really the case the though? Isn't the whole point of tenure (or a big selling point at least) insulating academics from capricious firings?

The big question I have is that there are names on these fraudulent papers, so why are these people still employed? If you generate fictitious data to get published, you should lose any research or teaching job you have, and have to work at McDonald's or a warehouse for the rest of your life. There are plenty of people who want to be professors that we can eliminate the ones who will lie while doing it without losing much (perhaps anything). If your job was funded by taxpayer funds there should be criminal charges associated with willfully and knowingly fabricating data, results, or methods. At that point you're literally lying in order to steal taxpayer funds, it's no different than a city manager embezzling or grabbing a stack of $20 bills out of the cash register.

I've met so many people who self-diagnose with those "conditions", because, I think, they want the world to feel sorry for them, or something.

That feels extreme. Zero is a cold, dark, lonely number. Maybe it’s correct—i dont know. Ive worked on only a couple of projects in this space, and while the incentives certainly involved publishing, i dont feel that it equated to abandoning the SciMethod. Instead, it was the cost to pay for the ability to continue doing science.

Can you really stand by ZERO? How about a 1%. Meet me somewhere above zero, or, if you’d be so kind, make a compelling case why were truly rock bottom.

[0] https://pretendradio.org/

forbetterscience seems like a good idea, but the writing style, the images, and even the about page gave me pause on if this is a reliable site for trustworthy science commentary

Translational research is probably part of it but I think it's part of a broader hype and fad machine tied to medicine, which has its own problems related to rent-seeking, regulatory capture, and monopolies, among other things. It's one giant behemoth of corruption fed by systemic malstructurings, like a biomedical-academic complex of problematic intertwined feedback loops.

I say this as someone whose entire career has very much been part of all of it at some level.

Pick your poison, to some extent. Better would be to not have to do it after-the-fact, but to vet better at every intermediate step, but it's hard. Just a very difficult people problem.

My concern is that with AI getting better and easier to use (e.g., in Photoshop) fraud will be extremely hard to detect.

We might need to re-think how research is done and results verified.

Similarly, the heads of famous science labs have lots of talented scientists who want to work with them. The involvement of a lab director varies wildly, but for the hyper productive, famous ones, it's largely the director curating great people, providing scientific advice, and setting a general research direction. The lab director gets named on all these papers that get generated from this process.

So 800 papers isn't necessarily a red flag if the director is great at fundraising and has lots of graduate students/post docs doing the heavy lifting.

> Founder, CEO, and chief engineer of SpaceX. CEO and product architect of Tesla, Inc. Owner, CTO and Executive Chairman of X (formerly Twitter). Founder of The Boring Company, X Corp., and xAI. Co-founder of Neuralink, OpenAI, Zip2, and X.com (part of PayPal)

It can only be a fraud.

Want to publish in a peer reviewed paper? Well then your institution or you should take out a bond or insurance policy that guarantees your work is accurate. The insurance amount would fluctuate based on how big of impact this study could have. Is it a drug that will be consumed by millions? Big insurance policy. Is it a behavioral study without much risk... small insurance policy.

Is a a person in an institution found caught committing fraud, well now then all papers from that institution now have higher premiums.

Did you sign off on a peer reviewed paper that was fraud? Well now your premiums are going up also.

Insurance costs too high to publish? Well then keep doing research until the underwriters are satisfied that your work isn't fraud and adjust the premiums down.

It adds a direct near-term economic incentive to publish honestly and punishes those that abuse the system.

1) revoke all of their academic accreditations and degrees

2) put them on a public “do not publish” list permanently banning them from being named on any paper in a journal

But if a health care professional does the same thing, and does something negligent, then they are usually liable. They are professionals and are held to a different standard. Similarly, that's why lawyers keep writing: this is not legal advice and you are not my client.

Perhaps a professional in science should have higher standards. Obviously they shouldn't be sued for being wrong - that would destroy science, disregard the scientific method's means to address inaccuracy, and go against science's nature as the means to develop new knowledge. But intentionally deceiving people perhaps should be illegal and/or create liability: When you publish something, people depend on its fundamental honesty and will act on it.

To put the scale of this problem in perspective, the ORI was set up in the 1970s after Congress became concerned at widespread reports of scientific fraud. It clearly didn't work, but hangs around regardless.

It's ultimately a culture problem. Until academics have the same level of respect as ordinary corporate employees, you're going to get judges and juries who let them off scott free.

I wrote a blog post on how to make this easier, including a new criminal statute specifically tailored for scientific fraud. https://news.ycombinator.com/item?id=41672599

It's like the old quote... "If you commit fraud as an RA that's your problem. If you commit fraud as the head of department that's the university's problem."

I suppose a silver lining to all the academic fraud exposés of the last few years is that more grad students and faculty now know that this is a thing, and one that many will try to cover up, so trust no one.

Another silver lining might be that fellow faculty are more likely to believe an accusation, and (if they are one of the awful people) less likely to think they can save funding/embarrassment/friend by neutralizing the witness.

(ProTip: If the success of your dishonesty-reporting approach is predicated on an internal administrator having humility and integrity, realize that those qualities are the opposite of what has advanced a lot of academic careers.)

The Alzheimer's and Parkinson's fields are too easy to fake, and too difficult to replicate. The new ideas are only ~20 years old. Big pharma companies are understandably wary of published papers.

When people say "trust the science", they often refer to things like masks, and antibiotics, and vaccines. That science is hundreds of years old and have been replicated thousands of times.

TL;DR: Some science should absolutely be trusted, some shouldn't. It's not surprising that you can't make blanket statements on a superfield ranging from germ theory to cold fusion.

Questioning novel science is one thing but questioning if the Earth is flat or Germ Theory is another thing all together. The problem with skeptics is that they sometimes hang around conspiracists.

It's hard to not discount these people when the person next to them thinks black people are biologically inferior. Then when those skeptics don't distance themselves or don't explicitly condemn those bad actors, it brings to question if their positions are born of skepticism or some strange prejudice, and that they merely constructed the cover of skepticism to hide their strange prejudices.

For example, during the Covid pandemic there was a lot of questioning around masks. In hindsight, the answer is obvious: it doesn't really matter if masks were or were not effective, because they're essentially free to wear. Even in the worst case, nobody is actually hurt.

But there were many, maybe millions, of mask deniers who would simply refuse to wear them. They were doing this because of institutional distrust and political motivations, not because they truly believed the masks were dangerous. And this is the trouble: these people are skeptics, but they're skeptics with an end-goal of political destabilization, i.e. they're dangerous.

When you mix it all together, which people often do to themselves, it's discredits the very thought process.

I honestly think if some serious attention was paid to this and various other real world paradoxes around us, we could actually make some forward progress on these problems for a change.

Obviously making it mindlessly laudatory.

I'm thinking about all the plagiarism issues uncovered with the publications of the former Harvard president Claudine Gay (and, similarly, of Neri Oxman, the wife of Bill Ackman, who basically was exposed due to Ackman's campaign against Gay). I looked over all the instances of plagiarism in detail, and, while not excusing them, they seemed like less of egregious theft of others' ideas and more like laziness/sloppiness. But I could easily imagine that laziness/sloppiness being fostered by an idea of "How could someone really check this word-for-word anyway?"

Well, now we have tech that makes it almost trivially easy to expose this type of misconduct.

70% of the studies they attempt are successfully replicated. 20% are inconclusive or equivocal. 10% are clearly debunked.

Now the world is richer, but Sweden? No return on investment for the Swedes, other than perhaps a little advanced notice on what hot new technologies their sovereign funds and investors ought not to invest in.

A bloc of nations, say NAFTA/CAFTA-DR, or the European Union, might be more practical.

That's the carrot. As for the stick, bad lawyers can get disbarred, bad doctors can get "unboarded". Some similar sort of international funding ban/blacklist for bad researchers would be useful.

Is there evidence of the fraud levels in those fields?

So we're systemically safeguarding the quality of astronomy research, by setting up a gradient (at MIT: restaurant catering for business talks, pizza for CS, stale cookies for astronomy) to draw off some flavors of participants and thus concentrate others?

You are talking about a part of the academy that relative to medicine, so few people do.

Show up to a bank looking like someone who knows math, and they'll cut you a huge check. Is that not fraud?

Sure money could be a factor, but the desire for prestige can motivate people just as easily.

https://web.archive.org/web/20240303093209/https://www.nia.n...

During COVID I talked to a guy who used to do computational epidemiology. He came to me because I'd written about the fraud that's endemic in that field and wanted to get stuff off his chest. He was a research programmer, assisting scientists. One of the stories he told involved checking the code for a model written in FORTRAN. He discovered it was mis-using an FFI and using pointer values in equations instead of the dereferenced values. Everything the program had ever calculated was garbage. He checked and it had been used in hundreds of papers. After emailing the authors with a bug report, he got a reply half an hour later saying the papers had been checked and the results didn't change so nothing needed to be done.

Little known fact: the COVID model that drove lockdowns in the UK and USA was nothing but bugs. None of the numbers it produced can be replicated. When people pointed out that this was a problem academics went on the attack, claimed none of the criticism was legitimate because it didn't come from experts, and of course simp journalists went along with all of it. They got away with it completely and will probably do it again in future. Even in this thread you can see people defending COVID science as the good stuff! It was all riddled with fraud.

Part of the issue is that scientists are mostly self taught coders who aren't interested in coding. They frequently use a standard of: if it looks right, it is right. The whole thing becomes a circular exercise in reinforcing their priors.

Wow that’s pretty crazy. I have to say many times in my career I’ve been writing a paper and realized “** there’s a bug”, and had to redo everything. But the overall conclusion never changed because the idea was grounded from several different angles(usually the pieces fit together even better). One bug might invalidate your result, but even if your code was correct the underlying assumptions behind the code could be wrong! I think the real issue was your boss wasn’t active enough in your work to make it robust to coding mistakes.

I met Leslie Lamport seven or eight years ago and asked him what a completely modern LaTeX might look like. He replied “well, we won’t be using PDFs in twenty years” and so it would need to be something completely different. Something interactive, with depth. Remembering, of course, to focus on quality content first and quality presentation second.

In a world with LLMs, this question becomes ever more interesting - why write a literature review if one can be generated?

We have done a great disservice to human intellect to have mistaken the gift that empiricism gives of predicting the world, with knowledge of the world itself of which we possess almost nil.

https://en.wikipedia.org/wiki/Francesca_Gino

That guy has a channel on fraud.

Also check out Data Colada: https://en.wikipedia.org/wiki/Data_Colada

This isn't theoretical, there's a lot of it going on.

How would this work? AI can't even detect AI generated content reliably.

It's fraud all the way down, where even the fraud detectors are fraudulent. Similar story to the anti-malware industry, where software bugs in security software like CrowdStrike, Sophos, or Norton cause more damage than the threats they prevent against.

[1] https://www.reddit.com/r/ChatGPT/comments/11ha4qo/gptzero_an...

Every one of these /discovered/ cheats.

Remember this particular cheat was one of your ethicals until a few moments ago.