A Post Mortem on the Gino Case
statmodeling.stat.columbia.edu
statmodeling.stat.columbia.edu
Her first job when he hired her was to hammer into shape a paper one of his PhD students was writing. Normal stuff, right? Adding literature, polishing off rough edges etc. Except the PhD student seemed really uneasy about the paper, kept talking about issues with the data that needed fixing. At face value, all was great - nice small p-values, great effect sizes etc. Eventually it transpired the PhD student was doctoring the data (these were the issues), was deeply uncomfortable with it, but also understood this is what my friend will be helping her with.
At this point my friend goes to the professor to inform him, nicely but firmly, of academic fraud. The prof comes down on my friend like a ton of bricks: who do you think you are, what do you know, I am the hotshot superstar and you are a no-name. All is well with the research and if you press on with the ridiculous accusations, she will be ruined in academia.
Well, my friend is a feisty one. She complained of bullying to the university, as an employment thing. The university was desperate not to touch it and in fact encouraged her to lie low, but she wouldn't. The whole thing kept escalating in really nasty ways, but eventually wound its way to an employment tribunal. Notably, at this point this isn't even about the academic misconduct, it's about the professor bullying his employee.
At the eleventh hour, the prof quits his job and goes to another, also top tier university. Since he is no longer my friend's boss, the case is dropped - no boss, no case. Still, however, no academic misconduct case! My friend, against all advice, wrote a letter to the new university, informing them verbatim of her findings. No response, to this day. The guy is still a top honcho there.
Whereas my friend finished her postdoc, didn't get another academic position and got an industry job, where frankly, she seems happier.
If fraudsters cannot be found or punished, it is logical to assume that everyone in the field is a fraud.
Everything is underwritten - the writing, if you zoom in, says "our advertisers approve this segment"
So how then, can one know that they're getting actual facts or good reporting?
Unfortunately, I don't know how to go about this
You’ll have higher costs, and will you have higher returns?
We need to get a lot more cynicism in society to counteract the fraud.
Public review is better (but also worse in important ways)
A possible improvement would be for reviews to be semi-public but (author) anonymous, also where the reviewers would have some choice in selecting papers
https://scienceintegritydigest.com/
Alternatively, maybe the people behind the Data Colada blog can help:
The issues are different of course, but there are common patterns of prestigious abusers, victims risking ostracism, etc.
I wonder if we'll end up seeing a similar movement in academia were we spend a few months watching big names taken down from their pedestal every few days.
Attempts to address this are probably going to get swept away by the anti-DEI crackdown in the US. After all, not harassing your juniors is inclusivity.
Now with the winds changing directions under Trump administration 2.0, the same people are probably already denouncing their past and jumping onto the "DEI is bad" bandwagon.
Maybe hypocrites are pretty obvious to spot for you. It's not as easy for me. I was shocked to find many of my peers turn their noses at DEI policies and act as if they've always been against it.
Bandwagoners are going to bandwagon. I personally don't see much use in labelling them as anything else.
Why were you shocked? Are you new to the workforce? Did you never live through a time when criticising a company policy[1] was a serious CareerLimitingMove(tm)?
Throw in the fact that people were both routinely and publicly punished for not jumping on the DEI bandwagon, and you have a large number of people who would rather keep their head down and get their work done in the most agreeable manner rather than tell you what they think.
People aren't going to work to have their opinions validated, they're going to work to get money. They aren't going to jeopardise that money by going against the company vision (whatever that may be).
[1] DEI or otherwise. Don't worry though, when the pendulum swings back these people would then present as supportive of DEI.
I can’t help but wonder if these mentors are also guilty of fabricating data and don’t want to cause issues that might lead to people looking at their own work more closely.
If you’re wrong and the result was legitimate, you’ve developed a reputation as a pain in the ass.
If you’re right, the researcher will say it was an honest mistake (maybe a clerical error) to absolve themselves modulo retraction, which happens all the time. Nothing of significance comes of it, but an “influencer” may well have other, above board, programs of research on which you could have collaborated and made a real name for yourselves.
There are crazy people in the world. But if someone comes to me and says "here is a thing that could end, or at least seriously screw up your career" I'm going to take them very, very seriously. Even if it is incredibly horrifying to contemplate.
At best, they get some 15 minutes of praise. At worst, straight to jail.
I don’t work anywhere near academia. I still wouldn’t blow the whistle or advise others to do so.
Were they completely innocent? Nobody is completely innocent, Cardinal Richelieu knew that. But they were definitely whistleblowers who went to jail.
You deserve the Nobel prize, but you critisized the work of someone? You get nothing! Swedish chemist Arrhenius blocked the Nobel price for Mendeleev because the later make some valid scientific critics the former didn't like.
>Bik’s venture into scientific sleuthing began as a hobby when she detected duplicated western blot images in a PhD thesis.
Now full time.
The "at worst" is worse: Two recent Boeing whistleblowers died in rather suspicious circumstances.
I think people are underestimatig the psychological impact of failing to act on fraud in your research. Once you've learned that you're a fraud and what you're doing is bullshit, it would be soul crushing to go to work every day.
I have a feeling that non-academics on the Internet have an overly bleak view of the ethics in acamedia. I don't know why, but several online influencers really like to peddle the idea that we're all corrupt.
ake = silent
heka ... in borrowing normally means "one hundred", but in Greek it's actually "he" (one) + "katon" (hundred), so "heka" doesn't mean anything. (eka- in chemistry is Sanskrit and unrelated)
demos = district
medius = middle in Latin; the Greek is mesos, which seems to maintain more of the explicit "between/among" sense (for which Latin has "inter")
damn, you started off so hard-core and no-nonsense like seal team 6 had entered the research review arena, then your finishing blow to fraud was... it's soul-crushing to the fraudster?
thats rare. i also worked in a lab that had a high degree of internal discourse freedom and low tolerance of fraud. it still took a lot of fortitude for one of the grad students to force a retraction of 4-5 papers of the lab (but it did happen). and many of my colleagues lost years of their lives by chasing other labs' misconduct. so either its a widespread problem or even the ppl in the good labs are too dimwitted to notice when things are sheisty
Well, congratulations for you, but there's a real crisis of legitimacy in some fields. Especially once they've been filtered through the pop-sci press release filter, which only makes things less accurate and more extravagant claims.
Social science and psychology have big reproduction crises and seemingly no recent usable results at all. Nutritional science seems to be compromised by the food industry. Economics blatantly has different "schools" which give different answers depending on politics. "Business science" seems barely worthy of the name of science at all.
The risk is that mistrust from those spills over into scientifically settled but politically controversial areas.
It's far more likely that they know whistleblowing creates far more pain than many people expect.
People who build their reputations on top of lies tend to lash out when threatened. They know they can't afford to have their lies revealed, so they launch an all-out offensive to discredit, bully, and push out their accusers.
I've seen it play out in the workplace a couple times now. Some people get downright vicious when they feel their reputation is at risk. I watched someone spend years trying to get another person fired simply because that person accidentally uncovered one of his lies. The person who discovered it wasn't even trying to do anything with the information, but the person who lied about it was so anxious about the fact that somebody knew that he started doing increasingly irrational and desperate things to push the other person out.
Things end mostly well, with my friend publishing, then restarting his PhD with a different advisor, then leaving academia entirely.
Academia did lose one (more) brilliant researcher, though, so it doesn't end well for academia.
Journal papers are single-blind, i.e. the reviewers know the names of the authors and can make decisions based on reputation, so if you publish great (e.g. faked) results once, it becomes progressively easier to do it again.
Reviewing for conference papers is typically double-blind - we don't know the identity of the authors, and although we can try to guess, a couple of studies show we're wrong most of the time. Most papers (65-85% at good conferences) get rejected, so it's easy to flush something that looks fishy. (then again, it's hard to tell the difference between someone playing games with their numbers and someone who just has the typical crappy CS systems understanding of statistics) The addition of artifact evaluation to a lot of conferences provides another avenue to flush out faked results, too.
However conference reviewing is vulnerable to reviewing fraud, since reviewers and authors typically come from the same relatively small community - see https://news.ycombinator.com/item?id=23459978, or just google for the words "computer science reviewing cabal".
Reviewing fraud might not be quite as destructive in the long term, as the papers that get published through such fraud are probably more crappy than faked, and will hopefully not get cited. In addition, when reviewing fraud is uncovered it's much more of an open-and-shut case - a single person can fake data and make it look a lot like good data, but review fraud requires communication between people who are explicitly barred from communicating on that topic.
A lot of it depends on the community, and the size of the community. When you have 1000+ reviewers for a conference things will always get out of hand - I know of someone who got his cat registered as a reviewer for a particular SIG conference that he hated, just to make a point. I assume one could get a lot of fraudulent research published in that particular conference.
"NOTICE OF RETRACTION: While investigating potential publication-related misconduct in connection with the ICIMTech 2021 Conference Proceedings, serious concerns were raised that cast doubt on the integrity of the peer-review process and all papers published in the Proceedings of this Conference. The integrity of the entire Conference has been called into question. As a result, of its investigation, ACM has decided to retract the Entire Conference Proceedings and all related papers from the ACM Digital Library.
None of the papers from this Proceeding should be cited in the literature because of the questionable integrity of the peer review process for this Conference."
> Retracted on February 24, 2022: Innovation Research on College Student Affairs Management Based on Big Data Environment
> Retracted on February 24, 2022: University Students in the Greater Bay Area Conduct Research on the Construction of Big Data Teams for Cross-border E-commerce Startups in the Middle East
> Retracted on February 24, 2022: Discussion on International Trade Governance under the Background of Big Data
> Retracted on February 24, 2022: Application of Incoterms in International Engineering Based on Information Platform
> Retracted on February 24, 2022: The Fixed-point Tracking Model of Volunteer Service for College Students of Traditional Chinese Medicine Based on Information Technology
> Retracted on February 24, 2022: Sichuan Consumers' Mining of Agricultural Product Brand Value Based on Big Data
> Retracted on February 24, 2022: Sports Consumer Behavior Based on Integrated Data in the Context of Big Data
> Retracted on February 24, 2022: Talent Heights Construction from the Perspective of Talent Concentration
https://www.theguardian.com/science/2013/dec/06/peter-higgs-...
>Peter Higgs, the British physicist who gave his name to the Higgs boson, believes no university would employ him in today's academic system because he would not be considered "productive" enough.
>The emeritus professor at Edinburgh University, who says he has never sent an email, browsed the internet or even made a mobile phone call, published fewer than 10 papers after his groundbreaking work, which identified the mechanism by which subatomic material acquires mass, was published in 1964.
>He doubts a similar breakthrough could be achieved in today's academic culture, because of the expectations on academics to collaborate and keep churning out papers. He said: "It's difficult to imagine how I would ever have enough peace and quiet in the present sort of climate to do what I did in 1964."
>Speaking to the Guardian en route to Stockholm to receive the 2013 Nobel prize for science, Higgs, 84, said he would almost certainly have been sacked had he not been nominated for the Nobel in 1980.
Perhaps academia could benefit from a similar culture shift that actively invites scrutiny to improve science overall? The article implies a deference and bullying problem in research, where junior scholars hesitate to challenge authority, even when they notice flaws. Fraud wouldn't be the direct focus of such a shift, but in an environment where questioning is encouraged, it would naturally be harder to commit, with fewer places for bad actors to hide.
* Some researchers make good money
* You could start honest, then realize cheating is the only good way forward given the incentives
Woit does do a bit of grey area coverage. Likewise, his academic pubs are more interesting to me, personally! His reputation, however, is mixed at best. A real hero!
Gelman: maybe he's trying to subdue the neuroticism*. I look forward to more compassionate, less overtly "curiosity"-driven grey-area coverage (not to mention stats papers) from him!
Here is recent one, sadly also about psychologists (would be more interesting to do it to card carrying statisticians)
http://stat.columbia.edu/~gelman/research/published/healing3...
*Not many people describe themselves as an exponent of radical self-honesty since radicality isn't a palatable, much less enjoyable, thing. It takes a real saint (or logician) not to reflexively flinch from the imagined tsunami of incoming pity.
Grey areas-- you can safely explore
Particularly in research, where you may be working on the same idea for decades, it can be damning when you realize that the idea might just be fruitless. If all you had to do was remove one or two zeroes, and finally your colleagues would praise instead of mock you, wouldn't that be great?
Why not a) finish the dissertation as soon as humanly possible, and b) now that you're bona fide, publish a separate paper disputing the famous paper which your dissertation didn't rely on?
Instead, it sounds like the student and committee advisors were a bad fit and spent most the time in a pissing contest.
The last thing you want to do before pointing out that the emperor has no clothes is get a job at the palace.
Given that, choose your sword:
1. Grad Sword that must be approved by a dissertation committee
2. Master (well, PhD) Sword where you choose your own adventure
If you reply #1 then I'm logged in to a fake HN site.
Research funding and research institutions are existing things, that operate in specific ways. One can lobby to change this, but unless you want to go back to the "gentleman scientist" model of the 1700s and 1800s, and a scientific establishment about 1% the size of the one today, you can't just buck the system.
Null publishing bias still exists and obviously it's not remotely a perfect system. However, it's not some sort of completely broken system where every paper is either fraud or replication, and interventions that start from that sort of an assumption (people are very aware of the need to trust but verify) are in my opinion unlikely to be useful.
If the world is at a 60/100, you need to focus on things that get you from 60 to 80, not from 0 to 20
https://www.newyorker.com/magazine/2023/10/09/they-studied-d...
Technical details of the fraud can be found in four famous-ish blog posts of the Data Colada group, also from 2023, beginning with:
1. It’s impossible to compete with free so business that do add real value are at a disadvantage
2. A future where tech adoption slowly was accepted and people eased into being comfortable actually paying for things the value never happened and now feels impossible, as free is expected.
3. It’s created bad incentives. Like keeping people on their devices as much as possible and only show them things that enrage them or confirm their prejudices.
4. None of it is really free because someone pays for it all and those costs are born by society anyways, while having less choice.
5. I think over time this ethos has become mainstream and it’s essentially a fraud mindset. Nothing matters as long as the line goes up.
(This process and the end results are surprisingly similar to academic fraud. “Everything is free” is the same thing as inventing whatever results people want to hear. Both will ultimately lead to the same, unsustainable end result, as people will no longer trust research because everything is fake.)
Society has at large approved of fraud. I’m not being a lone holdout.
? I thought OP was referring to things like the presidential cryptocurrency, the ultimate endorsement of pump-and-dump fraud.
I’ve seen fudging analytics and subscriber numbers to lie to ad buyers. I’ve seen people intentionally hold items over from one quarter to the next for accounting purposes. I’ve seen events use dubious counting methods to inflate their attendance figures. I’ve even heard stories about hospitals moving patients from failed surgeries back to their floor before they die in order to fudge the surgery survival stats.
A lot of this is Goodhart’s law in action. But also, when these very tiny frauds go unchecked in a competitive marketplace, everyone becomes forced to do them. If law enforcement won’t punish them for being evil, the market will punish them for being good.
Modern academia has terrible incentives. The easy path to success comes from not rocking the boat, even the guys at Data colada (who did the leg work to prove the case) have made a lot of enemies in the field.
I’m willing to bet there’s a large amount of less significant fraud that gets better defended, it’s only the people who are so successful they let they guard down that get caught in this silliness.
For what it's worth, while I'm sure that her advisors are known within that research community, I believe they deserve to be named and shamed. Their behavior, IMO, is almost as bad as the original fraud to begin with. It was classic "circling the wagons" bullying, and their scientific reputations absolutely deserve the hit that their atrocious behavior warrants.
I want to emphasize that this isn't about "retribution" or getting back at these people, it's about publicizing that their behavior was wrong, the antithesis of what scientific inquiry is about, and was fundamentally rooted in a cowardice that put professional standing on a higher pedestal than scientific integrity. Actions have consequences, and these advisors should face consequences for their abhorrent actions.
> “I formed the opinion that I shouldn’t use this paper as a building block in my research. […] However, my advisor had a different view…”
> “…scientific criticism of a published paper had no place in a dissertation…”
Are these dissertation catechism?
Or clash of personalities?
Realying on potentially fraudulent originator that put up the thing to take it down is apparently too much.
Having a jury dig thought the submissions and scoring everything should be hilarious.
What's missing from the toolbox?
Certainly one major problem is: although we live after the scientific method was developed, things can still be done without actually following its principles.
Like any set of tools, it has to be actually applied. There is a long and storied history of scientific advances and discoveries being fraudulent, because there are many incentives for fraud - or just plain delusion. Delulu is the solulu has always been tempting.
There is nothing missing from the toolbox here. It just has to be applied.
Or maybe we are gonna be saved by more and more of science getting depersonalized by the virtue of being created by AI?
Fraud.
And anyone who's seen "honest" public procurement will see the benefit to a system where the governor can call his friend and have a railway built by the end of the year.
https://www.science.org/content/article/after-honesty-resear...
https://www.vox.com/future-perfect/24107889/francesca-gino-l... "In response, Gino — who has maintained her innocence — sued Harvard and the bloggers who first published the allegations, claiming she’d been defamed. "
Look at the 2008 crisis. Only one banker went to jail.
Look at Binance’s Zhao.
Look at - ahem - Trump.
https://www.bbc.co.uk/news/business-68595204
https://www.cityam.com/former-barclays-trader-jay-merchant-j...
Thankfully in some cases, people do actually go to jail.
I don't think this is incorrect but it's definitely a grey area to spell it this way. It would be ironic to be downvoted for pointing it out based on the intent of the article.
Alas, fraud was always a viable path. The phrase "fake it till you make it" wasn't coined without reason.