The Academic Culture of Fraud
palladiummag.com
palladiummag.com
Minor point, but I don't think one can reasonably credit Yudkowsky with this. Yudkowsy was no booster of neural nets and was quite surprised by their recent success. He's certainly said a lot of important things about artificial intelligence (mostly regarding the potential dangers of such), but his ideas haven't really contributed to current methods of constructing them.
(And yes you could say that Yudkowsky's ideas about the dangers of AI were influential on Anthropic specifically, but this quote is obviously not referring to that.)
The article is excessively negative in tone, and very dramatic and aggressive. I have found many people adjacent to academia, drop outs, or even some inside, very disenchanted and angry at how it works. And it's true, the sets of incentives, structures and political organisation in academia don't relate at all to academic excellence, and are something we have to "suffer". I wish we could come up with a better set of incentives, but it's very hard to do in a mostly vocational and passion-based activity. So what people have come up is structure the incentives along the chores (eg teaching) and easily measurable results (eg publications). And whenever you come up with an incentive structure, some people will game it. And the current state of publication stress (publish or perish) is extreme and counterproductive. But please note that these measurement requirements and incentives are imposed from outside academia. Of course, I'm not saying leave us to our devices, academia is nepotistic and political enough. But the system sure could use some overhaul. Suggestions welcome.
On the other hand, this "fraud" is incentive fraud, but not "truth" fraud. The way science Truth works is by accumulation of imperfect, even erroneous results, leading to an ever more refined understanding of the world. Scientists don't just blindly trust others, even if they cite each other (nowadays, citations are a political and incentive-gaming tool more than actual references). So these massive scale frauds don't bother us so much because they don't make understanding necessarily go backwards. Of course the payer feels it's a waste of money, but in academia we see money as support for research, which is mostly failed anyways because you only make discoveries by failing and failing again.
And progress in knowledge is nowadays still going on, even in the medical fields. And academia still works, much as healthcare and compulsory education, becausemany people feel a calling to do these professions properly, even if it doesn't seem so from outside. So let's be optimistic, even while trying to come up with improvements to the current model.
PS: So sorry for the wall of text
That some can endure the suffering and keep working in this sort of environment does not reflect well on today's institutions and doesn't inspire much optimism for the future of academia.
I have two home communities in the academia. One is string algorithms, compressed data structures, and things like that. It's a small community where people enjoy what they are doing and have somehow found a way to make a living out of it. The groups tend to be small, the atmosphere is friendly and honest, and the results rarely appear in top conferences.
Then there is a subset of bioinformatics and genomics, which often uses results from the other community. This is supposed to be important, so there are a lot of big well-funded labs from top universities involved. Stuff sometimes gets published in prestige journals, administrators from funding agencies (particularly NIH) are trying to direct the research, and even the Big Tech is trying to grab their share of something. I haven't seen fraud, but everything is so serious and competitive and there is a constant pressure to get things done before someone else manages to publish something similar.
I don't know how things are further downstream, and I'm not interested in finding out.
I have mixed feelings about this article. I agree with the sense of ignored crisis it points to, but also think it doesn't understand the problems with the solutions it recommends, or maybe misunderstands the sources of the problems, like you're saying.
Academics is different from finance maybe in that outside of outright fraud, things are murky. What one person considers "unscientific" another might consider perfectly reasonable or even rigorous, and vice versa. I've seen debates like this, where the two sides both consider the other unscientific and theoretically and methodologically unrigorous. I don't see the outright fraud as the core of the problems either, it's an extreme version of something that exists because of rotten incentive structures. Getting rid of it is akin to replacing the roof on a house that has rotten foundations: important, but not solving the problem.
The problems in academics can't be easily reduced to one thing. There's lots of extremely competent people working under a broken model of reality, one that assumes that such competence is rare rather than common, that progress is due to single individuals rather than collaborative groups, administrations looking for money from research rather than money for research, fame rather than truth, and so forth and so on. Then there's the other side of the coin, employers using degrees to avoid competent training and hiring, reducing people to specific degrees etc.
Increasingly I see the problems with academics as pervasive to society (at least US society), something deeper, just incentivized and maybe manifest more clearly because it's so broken when applied to academics. I don't think it's a coincidence that health care and academics have both seen huge inflation in the last several decades, for example, and are both huge sources of controversy in US society. I think the average people working in those fields are doing so in good faith, but I also think there's systemic pressures that create huge problems and bad actors in both, and there's enormous reluctance in both to change things because of power structures and poor understandings from the rest of society about what's going on.
Anyway, quite a wake-up call to be told we should model ourselves on finance. Ow.
LaCour? Gino? The Ariely study mentioned in the article?
It is deeply embedded in system. Opposition is not possible, there is not even "correct" language to express dissident.
I don't doubt there are many, many researchers and physicians who genuinely want to help people. It's possible to me that virtually everyone does.
But if that's the case, the incentive structure has apparently gotten away from everyone and become some sort of monster working at cross-purposes toward them at all times, keeping them from succeeding.
I don't know the specific numbers, but what fraction of the U.S. workforce and economy is involved in medical/pharmaceutical or related insurance research, treatment, or administration? It has to be well into the double digits.
If people started actually being cured of things, actually being well, whole segments of this structure would collapse--and if that percentage figure is high enough, so would the economy.
I don't think we even need to posit that there are actually malevolent people involved who are trying to keep people sick. It could all just be a consequence of a whole system that depends on it to keep operating, and consequentially nudges just enough of every benevolent, neutral, or merely self-interested person's work in a direction that keeps it from making people well.
I don't really see what that has to do with my point, though. I'm saying we don't have to assume anyone has ill will to explain why the system is failing. I'm not saying it's impossible that some people do.
Also, curing people is hard in a lot cases.
Lifespans are going up, including the length of time spent being relatively healthy; infant mortality is down; cancers are getting healed; there’s even a miracle cure for obesity.
E.g., There was some noise a few years ago about the obstacles Louisiana faced in changing the medicine for hep C from the expensive lifetime subscription model to a cure.
https://www.nola.com/news/healthcare_hospitals/louisiana-off...
https://projects.propublica.org/nonprofits/organizations/823...
The replication crisis is clearly a big issue, but the conclusions I would draw from the examples presented are more difference between fields than a condemnation of academia. To just say "move it to private instituions" is shortsighted.
I think the biggest institutional mistake of the academic system is the over-production of academic bound PhDs.
People are strung along for years thinking they have a shot at some professorship, so they work for low pay over their prime societal contribution years before realizing there are no spots left.
Academics afraid of their future security will be more prone to fraud, and more of them need enough FU money to try things that go against the grain.
If we pre-filtered academics half way through their training based on the number of faculty positions that are available it could help curtail the fraud ridden rat race.
Meaning what? Kick out students despite that they're on track to graduate?
Plenty of people get a PhD with no intent of going into academia.
Yes, there should be effort to address academic fraud. But no, we should not buy into conservative propaganda and torch our institutions of higher education.
I think the logical pathway between “two 20 year old papers that ‘sparked’ the Amyloid hypothesis are responsible for the entire research agenda and billions spent on ineffective drugs” is strained. “Sparked” may be true, but the idea that a few twenty-year old fraudulent images tricked Big Pharma into spending two decades chasing this hypothesis and developing drugs, and all along the way the whole thing was based on fake data that nobody every bothered to test in a subsequent experiment? That seems deeply unlikely to me. If true I would like to see that case made with evidence.
TL;DR What I am not saying: fraud is acceptable. What I am saying is that there are many replication and error correction systems in science, and people with an axe to grind really don’t want to understand that.