Harvard professor Francesca Gino was accused of faking data

> all related citations are lost

What an extreme BS argument. It's really grasping at straws.

The accusations here are very different: Producing fake data is fraudulent, of course the papers need to be retracted, and people involved and in the know need to be punished. The quote from the article describes well something that seems to be a growing group of people within academia:

> It's possible "these people don't care and never cared about the science," Michael Sanders, a professor of public policy at King's College London, said. "They talk about science, and they talk about experiments, and they sort of wrap themselves in the fabric of the scientific method — just as a way of selling stuff."

The bar for being accused is so incredibly high. People are given so many possible outs and are allowed to produce the most implausible explanations. And then, even having one or two faked papers basically has no impact.

Here's an example.

Ken Rogoff and Carmen Reinhart. Professors at Harvard. Despite having faked the paper that introduced austerity to the world, had an immense impact on the economy, and caused massive and needless suffering for hundreds of millions of people. That paper still continues to haunt countries today. To make their paper work they had to lie 3 times: to exclude some countries, exclude data from others selectively, and reweight all of the data. None of this is reported in their paper! Each of these took deliberate actions on their part, they cannot be mistakes. Every single "mistake" they made is carefully calculated to produce the result they want. Every one of them is needed otherwise their paper would be junk.

What happened to them? Well, Reinhart has been promoted to the Chief Economist of the World Bank and is still a professor at Harvard. Rogoff is still a professor at Harvard too. Their faked paper literally resulted in deaths, toppled governments, and the rise of the alt right in places like Greece.

But they got a promotion for it.

The only people who are afraid are the people who are faking their data. No real scientist is getting accused of anything here.

What kinds of steroids and how much of them can I get away with using so I'm in the perfect spot where I am ahead my fellow doping athletes, but still under the bar of detection...

Coming soon to a San Francisco near you!

The first quote here is not a defense of Gino, it's a nuanced opinion of self-proclaimed vigilantes who are also human and might not always make perfect decisions. It is followed by this:

> Discussing the allegations that Gino falsified data is seen almost unanimously as fair game, especially in light of Harvard putting her on leave. People expressed more mixed feelings toward disgraced researchers such as Cuddy, who failed to replicate her findings but did not fake her data.

I have no special insight here, but this at least seems like a reasonable thing to discuss.

> It feels, to me, that they're making Data Colada (the data vigilante group) the bad guys here.

I strongly disagree. The light questioning of Data Colada was a very small part of the article, and even that section did not give me the impression that they are "bad guys".

This is what happens with sex abuse. When there's enough guilt to go around in the right places, the guilty form a kind of hellish camaraderie, a Mexican standoff of pointing fingers. Once such people assume positions of power, they also have a tendency to concentrate corruption in the institution by favoring the corrupt. Doing so entrenches the standoff and eliminates those who would oppose or expose them. The so-called "lavender mafia" is a well-known example, because the media hate nothing more than the RCC, but this occurs everywhere and at greater rates.

What's important is what we distinguish between institutions and the people that temporarily occupy them. Good institutions should generally be preserved. It's the corruption that should be dealt with. Of course, that's not always an option. Sometimes, corruption destroys the institution, or perhaps the institution is beyond repair. But generally, there is a presumption in favor of cleaning house rather than abolition.

"Trust the science" is a poorly-phrased attempt to counter the persistent conspiracy theories that various science disciplines, and the scientists that work therein, are part of this or that conspiracy to reduce your freedoms, mislead the public, etc. Or the related conspiracy theory that science broadly is bullshit and that the primary motivation of any scientific study is the continued justification of funding, rather than pursuit of knowledge.

The fact that scientists can be wrong, some studies are faked or their results are overstated, etc. has little or nothing to do with it.

> I refuse to be browbeaten by priests in lab coats or their acolytes. If you cite scientific papers in front of me trying to make a political case for a course of action, I will use my own judgment, informed by he paper as well as other factors like motivations and so forth.

And this is exactly the kind of attitude that I described above. It's the ACAB approach applied to scientists, where it makes no sense if you look at what scientists actually do and care about. A huge number of scientists are persistently and actively fighting against the "priest in lab coat" archetype.

That has all to do with authoritarianism and how in partisan politics and during the covid draconian policies push (in many different countries) any dissent or non agreement was treated as a terrible sin that HAD to be punished.

Even nowadays, in HN, you'll get people repeating the mantra, downvoting and acting super obfuscated that someone dare point out that just because govs/tech/pharma/media sayd so, doesn't mean it's true, specially when we there's money and politics involved and that security or hygiene theater is nothing new.

Many studies, even those relied on for medical products, don't get replicated by independent entities.

It's safe to assume, any non-replicated study is an outright fabrication at this point.

Now if you'd prefer to put your faith in religion, or some demagogue, that's fine. And you're free to go eat all the horse paste you want. But overall dismissal of science as "stupid" is just a flat out ridiculous take.

1. The optimal rate of fraud is not 0%, just like the optimal rate of shoplifting in a store is not 0%, and the optimal amount of crime in a society is not zero. Science, just like the grocery, and the street, has to operate on some degree of trust, and not non-stop big-brother gestapo police state surveillance. Just like a zero-crime society, a zero-scientific fraud society would have a lot of negative externalities that aren't accounted for by 'bring fraud down to zero' paperclip-maximizaiton. That's why it makes sense to talk about what can be done to reduce fraud, but makes no sense to throw yourself off a roof when an instance of it actually happens. It's a statistical inevitability. Look at the blast radius, pick up the pieces, have a conversation about what, if anything, could be done better next time.

2. If even one bit of dishonesty is enough to distrust a system, you shouldn't trust anything. But nobody actually thinks or operates in that kind of insane solipsism. You wouldn't get anything done.

3. What actually happens is that people from low-integrity fields (Politics, sponsored media, lobbying, organizational PR, internal corporate politics) are trying their fucking hardest to use that argument to bring public perception of everyone else down to their level. If they are blatantly lying half the time, and someone working in science blatantly lies, say, 0.5% of the time, and is blatantly wrong 5% of the time, and is mostly-right, but subtly wrong[0] the rest of it, that solipsist from #2 would look at it and conclude 'well, there's no difference'. He couldn't be more wrong, and I would like to gamble on loaded-dice games with anyone who disagrees with me on this point.

4. You shouldn't use a single paper, or two papers, or even fifteen papers as an arbiter of expert consensus. Papers disagree. Expert consensus is reached by looking at the whole body of a field. This requires extensive understanding of a field, which you're not going to get from clicking a few links over an afternoon.

5. You shouldn't use expert consensus as a definitive arbiter of truth.

6. But you sure as hell should defer to an expert, over your own judgement, about non-trivial technical matters. I don't care how smart you are and how many papers you read last week[1], I have serious doubts that you're going to be any good at performing, say, an open-heart surgery. There's a reason we leave it to the experts, even if what they know is wrong. [2] [3]

[0] Almost all of our understanding of the world is at least subtly wrong. That's fine, though, many bits of that understanding are, while subtly wrong, useful.

[1] It's a different story if you read ten years worth of papers. And not just the ones that support your biases. And also spent five to ten years working with, and for experts in the field, making a lot of mistakes along the way. You don't get mastery of a subject by reading, you do it by doing, making mistakes, and correcting them.

[2] What experts know is almost certainly, objectively wrong. But it's often useful, and better than blind stumbling.

[3] If we treated the low-integrity fields with an iota of the scrutiny you want to apply to the lab-coated tech-priests, half the world wouldn't have a job.

That said, there are many who would use the fact that "humans are flawed" to throw out scientific research and distrust it. Scientific research will still be more reliable and dependable than the spew of others because there is rigor to it.

but the fact that this research is public and available is why frauds like Gino can be caught. This is the scientific method working. Media tends to amplify the hell out of the fantastical, fraudulent or not. But this IS the system working.

We may step in the wrong direction here and there, but we will inevitably push towards truth.

The scientific method isn't broken but the institutions that surround it and the people riding the tops of it tend to be. It'd be easy to fix but you'd instantly make a million high power enemies with psychopathic tendencies. Oh well.

That's no secret. Any psych professor will tell you, it's rare to find a student (and consequently, psychologist) whose interest was not sparked by a personal history of mental illness.

Well given psychology is one of the two epicenters for the replication crisis, I didn't think this should be too unexpected.

Gino, on the other hand, is accused of outright fraud. Data Colada claims she took real data and changed it to support her theory, or possibly created entirely fake data.

I started my eductation with a bachelor of maths and an engineering degree before doing a lot of economics while in business school. The field is full of interesting ideas but methodologically it mostly is garbage even you stick to the more mathematically oriented part of it. And that's economy which is supposed to be the most serious of the soft science.

And you may be thinking that's all harmless, but it's really not. Folks running experiments are competing for jobs with other folks who might do more worthwhile research. Because psychologists are good at running the pipeline, and they have found a nearly inexhaustible source of publications (experiments run on undergrad students), the up the bar for everyone else.

I remember just a few days ago a discussion on this site about how a hiring committee skipped over the "better" applicant with more publications and citations than the successful candidate. The reality is that there are plenty of faculty on hiring committees out there who are looking for candidates with a good publication record, because that is how they assess if someone is a good researcher. Psychologists have already succeeded in pushing out most of the less experiment-prone folks in their discipline. The pressure extends to adjacent fields.

1) People definitely did get away with this, all the time. Historically, researchers had no obligation (at least that was practically enforced) to maintain and share their data and code. Peer review would check for specific methodological flaws and nothing deeper. If someone emailed you about your 1995 study in 2005, you'd say "I no longer have the code," or, more likely, simply never email them back.

2) In a highly competitive landscape where cheating is effective, the "winners" will be heavily selected for willingness to maximize the use of cheating. Even if only 1% of the population is willing to commit explicit fraud, that 1% is going to be heavily overrepresented in a world where explicit fraud gets you the top-tier publications that bring you to prominence.

3) Ariely and Gino both made millions of dollars from their fraud that they will not have to give back. It's worth emphasizing how poorly Ariely's fraud was executed -- he did the laziest possible fraud and easily converted it into money and prestige.

4) Related to the first point, it's hard to overstate how much the culture has shifted since the '08-'12 time period. The replication crisis was just kicking into gear, and only among people who were paying attention to that type of thing. Ariely didn't come up reading Andrew Gelman's blog. There's simply far more light on any paper today than there would've been 10 years ago -- statistical and methodological understanding have come a long way as cohorts of academics came up in the shadow of the replication crisis. Having established credible groups like Data Colada to centralize these analyses has also been a big deal; tenured profs can't bully Data Colada by threatening their career progression the same way they could if accused via email by a random grad student.

Academic research is sort of a "Tyranny of Structurelessness"[1] system where people can't risk alienating themselves from the underlying power structures or it'll destroy their career. On top of that, there isn't a good mechanism to report fraudulent research. So when someone finds a paper that can't be replicated, the sensible thing to do is ignore it and move on. Causing a ruckus will get them labeled a troublemaker, and there isn't any guarantee it'll make a difference.

And further, there are no material incentives for exposing a bad paper. It doesn't get the researcher another citation which is their currency. It just wastes time they could have spent working towards a new paper of their own.

https://www.jofreeman.com/joreen/tyranny.htm

Hint: not much happened.

Still on the faculty roster. Still "Distinguished Professor of Behavioral Economics".

So why not fake data and cheat? You might get discovered, but by then, you'd have given tons of TED talks, speaking engagements and probably made millions of dollars. You get to keep that and you get coast relax as a professor for the rest of your life.

Unfortunately I think this means we need to disregard these popsci papers, especially in the social sciences unless there's a clear, uninterested party providing the data as well.

See my post that mentions Benford's Law, it's damn good a sussing out fraudulent data. Moreover, it's quite old and has stood the test of time.

[1]https://en.wikipedia.org/wiki/In-Q-Tel

[2]https://gizmodo.com/larry-ellisons-oracle-started-as-a-cia-p...

Is corporate life any better? I've tried both. And in terms of autonomy, it is a huge relief not to be bound to quarterly evaluations and KPAs and trying to belittle others' work just so that you don't get axed and so on.