They studied dishonesty – Was their work a lie?
newyorker.com
newyorker.com
I don't know whether the advisor was referring to collegial decorum in how good faith research is discussed, or more like whisper networks about bad faith that are impolitic or scared to speak aloud.
But I did actually once get a snap response like that. I was chatting with another grad student, who'd mentioned a student who'd just arrive, who'd be working for Prof. X. I hadn't worked with Prof. X, but I happened to see them treat multiple students poorly over time, belittling the student, disciplining them in front of groups of others, giving them non-research chores like a personal assistant rather than a research assistant, not letting them pursue their research, etc., so I blurted out, concerned, "Oh no!", and that X was a bad advisor.
This other grad student surprised me by snapping back at me, sternly, "You shouldn't say that!", and something about reputations. That other grad student's parent was a prominent academic, so I figured they were admonishing me in some decorum that they were brought up in, and which they knew better than me.
It might've been weeks later, that same grad student came back to me, apologetically, and spoke with surprise, of how miserable the new student was, once they realized the career disaster that they'd stumbled into.
Epilogue: A long time later, that grad student, who'd admonished me and then apologized, contacted me about a different professor, because they knew a prospective new student of that professor, they had some suspicions, and they thought that I might know something. The truth was much worse than they suspected, and the student fled after hearing only a little, in vague terms.
BTW, there's apparently a lot of all kinds of poor behavior, but the people doing it are almost never cliched evil, IME. For example, a couple times I saw Prof. X do something kind, and I think probably they had something like a very stern taskmaster upbringing that caused their other side. There was also another one, who was kind to me, but I later learned that they were decidedly unkind to some others, and were actually nudged out. And one of the most body-count professors I saw was actually genuinely warm and charismatic and humble in some ways, and I don't think they realized that they seemed to have emotional/cognitive problems that they let kill other people's careers in an awful way. Off-the-record gossip with grad students and (later) professors will tell you of all sorts of other misbehavior, especially by people who aren't all bad, but very driven and pressured. (Raging narcissist/psycho, however, seems relatively rare, or their trail of bodies doesn't survive long enough to complain about it. Maybe the worse people tend to go for careers with more money and power?) And, back to this article, I once met with Ariely, and he came across as empathetic, down-to-earth, and of goodwill, so -- iff it turns out that he's found to have done something academically dishonest -- again, that would seem like a bit of human frailty, in a more wholesome larger picture.
(Note: I've been a grad student a few places, and have talked with people at countless other places, so am not calling out a particular school or person. A lot of people have seemed paranoid about saying anything at all, myself included, so please don't speculate, or I think that would have even more of a chilling effect than already exists.)
Power differentials experienced too long can create character flaws, even when originally used constructively and with all good intentions.
I saw the cliche of a professor taking a students original and significant work, done without their direction, and quickly presenting it as mostly theirs, after adding their critical professorial value by waving the result around for their colleagues to congratulate them on.
The nice short paper that resulted was notable for an author list starting with the professor, followed by at least a dozen of the professors colleagues, and ending with the student who had solved the problem whole by themselves.
I also watched a professor's highly evolved and useful ability to dodge and protect his team from endless administrative performative/mata/made-up work in a university department translate badly to a business partial-ownership situation where their slippery ability to shirk work onto others gave them strong leverage over a project. Leverage that had much worse indirect effects than the obvious direct one of not contributing a lot.
This was someone with many great qualities, but also an internalized entitlement and talent for control.
* Students who are mistaken, entitled, and/or lying... is a thing. Which do you trust, and is there a crying-wolf effect applied to students as a whole.
* At the faculty level, they're colleagues, often friends (sometimes more), are more likely to being seeing a better side of people with problems, and will tend to identify with other faculty. (Though there's also internal dramas, but they might unite against the common enemy that is students.)
* The administration can be made up of people with similar problems (especially overconfidence, and aggressive self-interest).
* In some facets of university structure, it can be strictly a business, no vestigial ideals of academia, only branding. Which is a problem both for how they behave in those facets, and for students haplessly trusting the university when a sociopathic facet is involved.
A big one is students have far more to lose. “Solutions” are generally worse than the problems they address.
Grad-student / professor-advisor relationships are extremely personalized as well as career impacting. And not hot swappable.
Losing an advisor and getting any kind of rep can be disastrous. Degrees can flounder and die in an awkward aftermath.
I have no idea how faculty & peers should help, after the fact. In both examples above, people knew & didn’t approve. But stepping in would have been war - with the probability of everyone regretting everything.
A culture that emphasized ethical student treatment, with specific examples about research and going into business with students, might be preventative.
But on the going into business side, a lot of universities want to parasitically insert themselves into that too.
To me that is a conflict of interest with their educational mandate, but administrations think they must feed to grow & grow to feed.
Basically, she liked sports where merit counted and disliked others.
For example, if you're playing a game with a score, or running against a clock, the sport tends to have a lot less BS compared to say gymnastic or diving competitions where you impress judges.
I think the trick in academia is optimizing for something similar.
https://www.gofundme.com/f/uhbka-support-data-coladas-legal-...
In early August 2023, Professor Gino filed a lawsuit for defamation against Harvard University, and against Leif, Joe, and Uri personally, claiming 25 million dollars in damages. Defending oneself in court is time-consuming and expensive regardless of the merits of the lawsuit – as First Amendment lawyer Ken White put it to Vox , “The process is the punishment.” Targets of scientific criticism can thus use the legal system to silence their critics.
At present, Leif, Joe, and Uri do not have pro bono representation. The lawyers they’ve spoken to currently estimate that their defense could cost anywhere between $50,000 and $600,000 (depending on how far the lawsuit progresses). Their employers have so far only agreed to pay part of the legal fees. Defending science requires defending legitimate scientific criticism against legal bullying.
Edit: I initially wrote that they met their GoFundMe goal of $350,000, which is true. However, I’m not sure why they set their goal to only $350k when they mention that legal costs could skyrocket to $600,000 which they have not met
Evidence of fraud in an influential field experiment about dishonesty - https://news.ycombinator.com/item?id=28210642 - Aug 2021 (51 comments)
(Lots more related links at https://news.ycombinator.com/item?id=37719476)
The insurance company confirmed the data Arielly represents he got is not the data they sent. Arielly is a fraud.
Dan Arielly is a curious figure to give this sort of benefit of doubt.
This data was so shoddily faked that I have a hard time believing someone did this with an intention to deceive. Uniform distribution with a hard cutoff at 50K??
There was a similar case a while ago in spider biology: https://www.nature.com/articles/d41586-022-02156-2
Pruitt had several influential spider biology papers out and when others failed to replicate and dug into the Supplementary Data, they also found lazy patterns.
>When Laskowski dug into data sets that Pruitt had provided for the study, she was shocked to find stretches of data that seemed to have been duplicated, to represent findings for multiple spiders. This questionable data helped to bolster a long-unproven theory that repeated social interactions in a group of spiders cause individuals to behave predictably.
So yeah, who knows how much scientific fraud there actually is; I haven't seen a case of fraud where the data was convincingly faked, which means that these cases are hard to detect or hard to prove.
Faking data realistically is almost as hard as getting it honestly. I know, I “expanded the data pool” for my eight grade science project.
Your suggestion only works in the simulated science, like school projects where you are retracing the footsteps of past successful scientists, to verify an already known result. There putting in more work will reveal that effect more and cleaner, because your teacher already knew how the thing works in the first place. This is totally unlike real science where we confront the frontier of the unknown.
Do an easy experiment once correctly, then again with falsified data. Present the results side-by-side. Ask people if they can tell you which is which. Present have a bit on what sort of statistics could catch your faked data set.
In my defense I would have run more tests had I started when I should have :)
My teachers warned against this and told us they would catch us if we tried.
It's just so laughably faked it's not even funny.
The insurance company then showed their data and it wasn’t falsified. That’s just amazing. They both falsified data, and when caught, pointed at each other.
Gino was fired but Ariely kept his job then sued Data Colada.
She had claimed it was misogyny and discrimination. She may be right in respect to how she was treated compared to Ariely.
Was it still worth cheating for both of them? Money-wise, absolutely! Add up all their book fees and speaking fees that they’ll never have to repay back. They can already retire comfortably.
EDIT: Gino wasn't fired. She was placed on administrative leave. Revocation of tenure was only implied as a possibility.
I remember thinking even then that there was something off about his arguments, and I'm not surprised that he has since been exposed as a likely fraud. For example, throughout his talk, he kept making the point that when someone made a self-serving claim or argument, he would "hold on to his wallet," making an analogy to pickpockets. He then concluded his talk with a transparently self-serving argument that the importance of studying irrationality was growing over time because (as just one example, I suppose), the share of deaths attributed to preventable causes (self-inflicted, etc.) was increasing over the decades, making it sound like society is becoming more irrational. This seemed very weak to me, because that's exactly what you would expect if civilization is making progress over time... if science and technology keep eliminating the exogenous causes of death, over time we should be left with just the endogenous ones. Anyway, I thought of raising my hand to ask if I should reach for my own wallet, but was too young and nervous.
Don't forget their faculty salaries. Duke is a private institution, so we don't have access to his salary, but absolute minimum Ariely makes 400K (and probably way more). You don't leave MIT for a chaired professorship at Duke unless it comes with a very, very hefty salary. 25 years of a salary like that will by itself provide you with a good retirement. You don't need to get the media attention of Ariely to benefit from fraud.
If your main incentive is financial you would be much better off to go into industry, a starting salary straight out of undergrad I a consulting or software development role is on the same level as a professor salary.
The freedom to fuck around and (probably) never find out is considerable. And if you're actually inclined to apply yourself that 100-150k a year plus your grad students basically means you're your own y combinator year after year every year. Tenure is HUGE. It's worth way more than the equivalent salary in industry.
[1] https://www.glassdoor.com/Salary/MIT-Professor-Salaries-E288...
If you want to see what I'm talking about, here's the data for the first full professor of economics at Michigan that popped into my head. His salary last year was $466K:
https://www.umsalary.info/index.php?FName=&LName=shapiro&Yea...
That's normal for the elite schools. There were rumors 10 years ago that Chicago made a 7-figure offer (denied by the department head at the time).
Edit: To be clear, this is what wealthy schools pay. Salaries fall quickly once you exit the top 15 or 20.
https://finance.yahoo.com/news/harvard-professor-raking-over....
Because nothing there reaches 300K.
Is the disparity between disciplines that high?
At the wealthiest schools, yes. 200K is a high salary in many disciplines. Here's the data for a chaired professor of philosophy:
https://www.umsalary.info/index.php?FName=elizabeth&LName=an...
and history:
https://www.umsalary.info/index.php?FName=juan&LName=cole&Ye...
Still good salaries, but not at the same level.
Or maybe he was kicked out of MIT? ;)
https://www.jpost.com/international/dan-ariely-scrutinized-f...
https://www.gofundme.com/f/uhbka-support-data-coladas-legal-...
Leif Nelson, Joe Simmons, and Uri Simonsohn are professors who together publish the Data Colada blog. In June 2023, they published a series of blog posts (linked below) raising concerns about the integrity of the data in four papers co-authored by Harvard Business School (HBS) Professor Francesca Gino. They waited to publish these blog posts until after the HBS’s investigation concluded, with HBS placing Professor Gino on leave and requesting retractions for the four papers. In early August 2023, Professor Gino filed a lawsuit for defamation against Harvard University, and against Leif, Joe, and Uri personally, claiming 25 million dollars in damages.
Everyone who isn't a straight white male is a member of at least one protected class. The most we can do is play the whistleblower-retribution card, even if it's a farce.
It appears to be worth attempting-- it costs you nothing and shakes the employer down for a quick and favorable settlement to shut you up. I've seen it done by two ex-employees in the last three years, and there was a recent news article about a woman at some insurer who was submitting timecards for hours worked while caught redhanded as AWOL...she immediately cried discrimination too.
Ariely has been well-known outside academia since his TED talk and wildly successful book 15 years ago. Surely that offers a good deal of insulation for him, without needing to play the discrimination card too.
> she would be placed on administrative leave, and that he was instituting perhaps unprecedented proceedings to revoke her tenure.
They are at different institutions. I meant that her claim at her own institution about misogyny seems like lashing out. She falsified data, the evidence is fairly convincing. But in a broader picture, in the media, she has gotten more scrutiny and flak than Ariely, that's what I meant.
Can you or anyone give more information on this? That data was definitely falsified at some point, so I don't know what you mean that it wasn't. Are you saying the insurance company gave the four researchers reasonable-looking data, and therefore, the four must have falsified it? Do you have a source?
Why is that amazing?
It's just the default go to. These people don't reach these positions by being honest and admitting their mistakes.
They do it by playing these kinds of games.
“It takes one to recognize one…” or however that saying goes. Or another one is how people study their own affliction: a common occurrence in psychology from what I hear.
> So what was the scientific fallout of Stapel's demise? What theories had to be rewritten? What revisions did we have to make to our understanding of the human mind?
> Basically none, as far as I can tell. The universities where Stapel worked released a long report cataloging all of his misdeeds, and the part called “Impact of the fraud” (section 3.7 if you're following along at home) details all sorts of reputational harm: students, schools, co-authors, journals, and even psychology itself all suffer from their association with Stapel. It says nothing about the scientific impact—the theories that have to be rolled back, the models that have to be retired, the subfields that are at square one again. And looking over Stapel's retracted work, it's because there are no theories, models, or subfields that changed much at all. The 10,000+ citations of his work now point nowhere, and it makes no difference.
> As a young psychologist, this chills me to my bones. Apparently is possible to reach the stratosphere of scientific achievement, to publish over and over again in “high impact” journals, to rack up tens of thousands of citations, and for none of it to matter. Every marker of success, the things that are supposed to tell you that you're on the right track, that you're making a real contribution to science—they might mean nothing at all. So, uh, what exactly am I doing?
It's kinda like finding out an athlete has been cheating. You probably want to throw away any of their inspirational talk, but it doesn't affect the rest of your life because it was just a sport to begin with. Somebody was gonna win, somebody was gonna lose, and it matters a whole lot to the players (and fans) but that's it. Except worthwhile science is supposed to actually matter whether it's true or false.
Does this mean psychology is useless? I don't think so, there have been many people helped by application of psychology, but the sooner we take it out of the same category as for example mathematics or medicine the sooner we can move on. I'd love to be convinced otherwise that we as humanity have a chance of understanding ourselves to the point where we can draw meaningful conclusions about human happiness or other feelings/states of mind. However, I think at this stage our chance of accomplishing this is similar to a medieval blacksmith doing a successful root canal treatment on of his dental patients.
I'm definitely looking at the advancements in AI with hope that perhaps replicating our own thought processes in machines will enable us to study them more systematically and to understand them, but these are still early days.
Something only becomes well accepted theory if it can be reproduced again and again, fraud will come out eventually. That is the beauty of the scientific process.
I'm not quite as optimistic about the implications, however. The way academia operates is very trust heavy. This makes sense, no lab can be experts in all fields of study, and perform every experiment themselves. If something doesn't work for you, the default assumption is that something in your system is incompatible with the previous work, or you're just doing something wrong. So sometimes it takes quite a concerted effort to identify bad science, because legitimate work can be nudged towards supporting + compatible with this bad science. You basically have no choice but to have basis in what is widely believed to be true if you want grant funding or papers published.
From a purely theoretical/knowledge standpoint, I think I agree - the truth will eventually be accepted. But in fields like biology, this can be at the cost of drugs not working, actually promising avenues not explored, wasted resources - tangible harms.
[^1]: E.g. the recent Alzheimer's controversy, Stanford's president scandal, 5-HTTLPR, etc. Ironically, Ronald Fisher argues that Gregor Mendel's work, the foundation of modern biology, to be essentially fraudulent despite being correct.
There's nothing weird about that. You'll see exactly the same phenomenon by going through a low-level mathematics textbooks and noticing how many of the problems have small integer solutions. Does that happen in real problems? No.
Does the fact that all the problems are fraudulent mean that the math they're meant to teach you is wrong? No. The math is right; that's why the problems were written up the way they were.
Psychology is worst than useless. It is fraudulent. It is religion except backed by non reproductive "science". None of the theories matter because they have no real world function except to replace religion, which would serve the same purpose anyway.
Your idea didn't work to help anyone. Did it hurt anyone? How will we know? You can be trusted again. Your bridge collapsed? Who will trust you again, ever to make a bridge?
In physics, we've seen an immense amount of recent controversy over LK-99; while the material might have some interesting properties, it is almost certainly not the room-temperature ambient-pressure superconductor that Lee and Kim claim it to be. How many totally bogus physics papers might be lurking in the literature, ignored and mostly un-cited? We don't know, because most of those papers don't make sufficiently interesting or exciting claims to inspire a raft of replication attempts. For all our aspirational beliefs about the scientific method and the power of peer review, most papers are subject to little more than a superficial sniff test.
There are plausible reasons to believe that psychology might have a worse misconduct problem than other fields, but the idea that psychology is uniquely broken is, I fear, simply wishful thinking.
It points to the discipline not being coherently built on shared theories but rather being a set of disjointed hypotheses.
My understanding is that this is one of the hallmarks of the replication crisis in psychology - studies that claim high effect sizes or otherwise high impact results without any plausible mechanism of action to explain them.
One has to wonder about the practice of citation in modern academia, when not one in 10,000+ is consequential.
I am also concerned by the willingness of so many physicists to cozy up to the oligarchs.
> Experiments however take much longer and if the numbers don't line up, they don't get published.
If you have an expensive experiment and need a lot of cash, it makes sense to go with people who have extra cash and an interest in funding science.
Aug 2021: https://www.dailymail.co.uk/news/article-9914493/Landmark-st...
July 2023: https://www.npr.org/2023/07/27/1190568472/dan-ariely-frances...
etc etc
> The published study data includes two different fonts. All data in Calibri font can be tied to our data while all data of Cambria font appears to have been synthesized or fabricated.
https://www.npr.org/2023/07/27/1190568472/dan-ariely-frances...
Crowdfunding a defense for scientific research - https://news.ycombinator.com/item?id=37393502 - Sept 2023 (47 comments)
Is it defamation to point out scientific research fraud? - https://news.ycombinator.com/item?id=37152030 - Aug 2023 (13 comments)
Harvard professor Francesca Gino was accused of faking data - https://news.ycombinator.com/item?id=36968670 - Aug 2023 (146 comments)
Fabricated data in research about honesty - https://news.ycombinator.com/item?id=36907829 - July 2023 (46 comments)
Fraudulent data raise questions about superstar honesty researcher (2021) - https://news.ycombinator.com/item?id=36726485 - July 2023 (33 comments)
UCLA professor refuses to cover for Dan Ariely in issue of data provenance - https://news.ycombinator.com/item?id=36684242 - July 2023 (131 comments)
Harvard ethics professor allegedly fabricated multiple studies - https://news.ycombinator.com/item?id=36665247 - July 2023 (215 comments)
Harvard dishonesty expert accused of dishonesty - https://news.ycombinator.com/item?id=36424090 - June 2023 (201 comments)
Data Falsificada (Part 1): “Clusterfake” – Data Colada - https://news.ycombinator.com/item?id=36374255 - June 2023 (7 comments)
Noted study in psychology fails to replicate, crumbles with evidence of fraud - https://news.ycombinator.com/item?id=28264097 - Aug 2021 (102 comments)
A Big Study About Honesty Turns Out to Be Based on Fake Data - https://news.ycombinator.com/item?id=28257860 - Aug 2021 (90 comments)
Evidence of fraud in an influential field experiment about dishonesty - https://news.ycombinator.com/item?id=28210642 - Aug 2021 (51 comments)
Neither science nor medicine should be judged based on these examples.
It is medicine, but don't take my word for it:
>allopathic medicine: a system of medical practice that emphasizes diagnosing and treating disease and the use of conventional, evidence-based therapeutic measures (such as drugs or surgery) [1]
>osteopathic medicine: a system of medical practice that emphasizes a holistic and comprehensive approach to patient care and utilizes the manipulation of musculoskeletal tissues along with other therapeutic measures (such as the use of drugs or surgery) to prevent and treat disease [2]
>homeopathic medicine: a system of alternative medicine that treats a disease especially by the administration of minute doses of a remedy that would in larger amounts produce symptoms in healthy persons similar to those of the disease [3]
>medicine: the science and art dealing with the maintenance of health and the prevention, alleviation, or cure of disease [4]
Homeopathy is still a form of medicine, though it is not allopathic medicine. That said, the lines are quite blurred. I know that at least one type of leukemia is treated with a cocktail of arsenic and vitamin A. That treatment is given in a standard hospital setting. By the definitions above, such a treatment is actually homeopathic medicine, not allopathic. Yet, the treatment does indeed work.
[1] https://www.merriam-webster.com/dictionary/allopathic%20medi...
[2] https://www.merriam-webster.com/dictionary/osteopathic%20med...
[3] https://www.merriam-webster.com/dictionary/homeopathic%20med...
Here the answer is yes. Dan Ariely is a complete fraud and so is Francesca Gino.
There are so many credible accusations of fraud against Ariely that about four of them get treated together very quickly at the end of the article, else the article would be several times longer. If you own his books, throw them away or shelve them with your fiction.
Also note that there are serious behavioral economics researchers who do hard work that isn’t remotely like this “nudging” or “priming” BS.
Deleted comment
Then argue against the accusations rather than lobbing ad hominems.
Seems like someone was rushing for a scoop (what scoop? This is old news.) or someone’s PR firm is getting results.
[0] https://www.nytimes.com/2023/09/30/business/the-harvard-prof...
This show is loosely based on Ariely's work.
As PR ramped on the show, so did dig-ups like these.
The storyteller wants the reader to think certain things about characters and events, and facts and impressions are cherry-picked and deployed just so, to support that.
I guess seeing the persuasion as that of a storyteller might be understood in this kind of journalism (I don't know, especially since the subject matter seems delicate), but I believe that scientific research needs higher standards.
I don't know of any such research.
To be fair, they’re not completely useless either. They just need to graduate from being an astrology into an astronomy, but I’m not sure how that’s possible without a near perfect world simulation.
There's a world of difference between trying to make sense of a topic when none of the available data is all that great, and outright fabricating data to make a name for yourself.
Just a reminder that there is no Nobel Prize in Economics.
There was nothing to "reveal". Neoclassical economics and the mental model of "rational actors" is no more science than phlogiston or alchemical conversion of lead to gold. It's worse than those, since it's an ideological construct used to buttress the social order.
Which is why we should understand how whole branches of economics are strange pursuits within the realms of fantasy devoid from reality; and when someone introduces a shred of it back in, this is hailed as some great achievement.
Reminds me of this segment in "Yes Minister":