UCLA professor refuses to cover for Dan Ariely in issue of data provenance
openmkt.org
openmkt.org
Dan Ariely is a fraud. To be specific: he has been credibly accused of fabricating data in at least four separate studies (Ten Commandments, delta dental, Hartford car insurance, and the study which he claimed Rossi ran for him which is discussed at the link). There are likely more.
More broadly: There was a “golden age” for this kind of “behavioral economics”/social psychology work in the early 2000’s up to probably 2012-2015.
Researchers would generally publish “results” which were either “delightfully counterintuitive” (ex: prime subjects to think about Florida and they walk slowly) OR featured extremely cheap interventions which showed large effects on outcomes (ex: signing an honest pledge at the top or the bottom of the page made a big difference in how honest people actually were on their tasks).
In retrospect, there are two problems (at least!) with this literature.
1. There was no theory backing up any of these predictions, so you could rationalize any statistically significant result with a compelling story. Publish it and you’re on your way to tenure.
2. Related to (1.), you didn’t even need to bother running actual experiments at all! Just fabricate the data. Make it fit your cute story and you’re done.
There are many tenured people in business schools (generally in “organizational behavior” or “social psychology” style departments rather than Econ) who were tenured on the basis of papers published under (1.) or (2.).
This is a big shame because there are serious researchers who do difficult work in behavioral and experimental economics which is _not at all_ like this BS. (Some are personal friends of mine, so this bothers me.).
There was no incentive for about 15 years to do the hard work though. And given that tenure is for life, it’ll be a long long time before these people are flushed out (barring more investigations, which should be encouraged!).
What should you do?
- be extremely, extremely skeptical of any results in the “priming” literature (eg the “think of Florida” example above).
- be moderately skeptical of results in the “nudges” literature. The retirement savings stuff makes sense and has replicated but the honesty nudge above is BS and has not.
- don’t get really excited about a researcher who says “economics bad - people dumb! - I behavioralist!” More than a few of these people have been revealed to be total frauds.
- don’t click or share this stuff when you see it on Reddit, HN or anywhere else. That kind of engagement “feeds the beast” - Deans love it and Deans assume you haven’t fabricated your data.
- maybe if you are specifically a Duke or Duke Fuqua alum send them a note asking why they aren’t taking the Ariely misconduct more seriously???
- (edit) throw away your copies of any of Ariely’s books. They are BS.
Specific example: In the car insurance paper, which was covered by Data Colada, there was agreement among all the four authors that Ariely himself was the only person to touch the data. That data (IIRC) was completely fabricated.
I’m happy to edit this comment and my original one if those details are wrong. I’ll give Ariely a presumption of innocence which he does not deserve and indeed which I suspect he abuses in research integrity investigations.
(Edit) data colada link: http://datacolada.org/98
At the end under “replies” Dan says himself that he was the person in touch with the insurer. It really beggars belief to imagine that there was someone at the insurer who fabricated a dataset which perfectly tells the BS story of this paper.
Also note that another of the four authors (Gino) will soon be fired from HBS for fabricating data in a series of studies.
It is just remarkable to see how people with enough knowledge of datascience to perpetrate this fraud for so long would also make errors as rudimentary as producing a faked 'random' data set in a uniform distribution. And with zero cars reporting over 50k miles driven. (edit: and so many others...this is worth reading just to see it all, thanks for linking).
These frauds really are not as smart as they think they are. But I guess selling best-selling books about stuff you made up could make you believe you're a genius.
You would imagine a con artist of his caliber would be more careful than that, but still ...
They've been doing it for a long time and it's always worked, either because they've always been this sloppy but everyone else reviewing was sloppier or gave the benefit of the doubt for reasons, or that they've put less and less effort into it over time because it hasn't been necessary.
Also the simpler the fraud is in execution the more likely you might be able to explain it away as a simple mistake or lack of familiarity with certain "tradesperson" tools adjacent to the skills you have framed in latin blackletter on sheepskin.
It's also clear that the data fabrication happened in Excel (based on styling differences that wouldn't pass through other formats) and the Excel metadata has Ariely as the creator of the file. If the data had been fabricated by the insurance company as an Excel file, they'd show up as the creators. If it had been fabricated by them as a e.g. a CSV and Ariely just imported it to Excel, it wouldn't show the inconsistent styling.
> If the data had been fabricated by the insurance company as an Excel file, they'd show up as the creators.
Digital forensics does not work this way. It is very rare when many digital evidence line up and imply that an actual person did or did not do something in the digital realm. A user account that shows up in an Excel file is absolutely insufficient to tie a person to the keyboard.
> it wouldn't show the inconsistent styling.
This is also inconsistent how digital forensics works. It can imply that something changed, but nothing about who, when, and why. It could have been an incorrectly propagated filter, or coloring, template, or functions.
> This is also inconsistent how digital forensics works. It can imply that something changed, but nothing about who, when, and why. It could have been an incorrectly propagated filter, or coloring, template, or functions.
No, sorry, I don't agree. It shows the "when" of the data fabrication, i.e. after the data was imported to Excel not before. The nature of the styling inconsistencies shows the "why", since it shows that the data was duplicated and one half of the duplicates was manipulated. Half the data was in one font, half in the other, and the two sets of data can be matched with each other. It also shows the "who" in combination with the metadata, due to the chain of reasoning in my previous message, since it shows that the data fabrication happened after the import to Excel and we know Ariely is the person who did that import.
> it is impossible to tell from the data who fabricated it. But because the fourth author has made it clear to us that he was the only author in touch with the insurance company, there are three logical possibilities: the fourth author himself, someone in the fourth author's lab, or someone at the insurance company.
So you seem to be much more confident about it only being one possibility, when the link explicity provides three possibilities.
One of those logical possibilities can be excluded entirely thanks to everyone involved agreeing that Ariely was the only person in contact with the company and handling the data ingestion. It could not have been anybody else in the lab.
The other theory is beyond absurd. Not only did the insurance company fabricate the data, but they happened to fabricate results that showed the effect Ariely needed for a high profile publication. Theories like your "slacking off data entry clerk" just don't make sense there.
Now, maybe if Ariely could provide even a shred of evidence for this being the data he received, I could change my mind. But he can't produce the file he received from the company. He can't tell who at the company sent it to him. The trail just runs cold at Ariely.
He'd already done lab experiments that showed the effect he thought existed, and it's easy to imagine an associate who actually fabricated the data already knew what he was looking to confirm.
Again, I'm not saying I think he's guilty or not. I'm just saying that from everything I've read so far, it seems plausible it was sloppiness and not checking data for signs of manipulation. That this sloppiness isn't "beyond absurd", but rather more like the classic "never attribute to malice that which is adequately explained by stupidity" (or by an overworked/overambitious academic).
Based on Bazerman's response, actually, he hadn't:
> [The fradulent paper] came together in a merger of two prior non-published empirical efforts. Mazar-Ariely independently provided the data for Study 3, while Shu-Gino-Bazerman had written a paper containing two laboratory experiments (Studies 1 and 2). The Shu-Gino-Bazerman group knew of the Mazar-Ariely data from multiple public presentations by Ariely.
Mazar's response is basically "oh god, I'm ashamed this happened. I played no role in the data collection, and I don't know who did. But thank you for all the hard work you've done in exposing this fraud, and I should have done better at questioning the data."
Gino's response (for context, she has been implicated in other fabricated data studies) is basically "I'm ashamed. I didn't collect the data, and I trusted the people who did collect it. But thank you for all the hard work you've done in exposing this fraud, I should have done better, and this is what I'm doing in the future to do better."
Ariely's response is "I'm the only one of the authors who had the data. Also, it wasn't me."
Bazerman's response is "Thank you for exposing this fraud. Let me explain how the study came to be, and the history of my own doubts about the quality of this study [which includes two episodes which seem to be red flags]. I should have pushed harder on those doubts [he was in the minority who wanted to retract the paper after nonreplication in the follow up]."
Ariely's response to me feels like raising a preemptive defense against being thrown under the bus. Neither Mazar nor Gino implicate anybody (only disclaim responsibility themselves), whereas Bazerman suggests that it would have had to have been Ariely or Mazar. (There's an interesting bit where Bazerman has an unspecified issue with the [now known to be faked] data prepublication, where one coauthor has unconvincing responses but a later coauthor was able to provide plausible explanations, complete with reference to the underlying data file--but both coauthors are unnamed). Given that, I just don't find the other possibilities all that plausible (and in the case of the middle possibility, it's not exactly exonerative anyways).
Clearly he had control over the file, but I wouldn't put too much weight on the metadata when there's plenty of other evidence.
It's easy to imagine reasons. A lazy assistant who's supposed to go through 13,488 filled-out forms decides to count 1,000 of them and then fabricate the rest so they can spend the week watching soccer online instead? Or an employee lost the data on which form results were associated with which condition, and they might get fired or not get promoted if caught, so they made it up? And so forth.
And metadata as you've described it doesn't sem like a smoking gun either. It's easy enough to imagine Ariely created an otherwise empty Excel sheet in the column format he wanted them to insert data into, e-mailed it to them, and got it back with their results. Or he just copied their Excel results into a spreadsheet of his own that already had other tabs, or similar.
I'm not saying Ariely didn't do it or did do it, I'm just wondering if there's an actual smoking gun here, or if it's accusations but there isn't enough evidence to actually know.
He also was the only one handling it, so if the data is bad then the responsibility falls to him regardless, both as the handler, as well as the head of the investigation. He either deliberately faked it, or got garbage data and didn't realize it when, now, plenty of others have.
Which points towards either him fabricating it himself, or asking somebody else to give him results without going through normal procedures for registering/conducting the experiment and then naming someone else as responsible for it...
I do think there are cases where academics will have unwittingly published falsified or partly false data and sometimes even garnered fame and citations for it x, and no doubt many more where the academics spot issues with the data they've received and bin it, but this doesn't look like one of them.
I've just gotten around to Kahneman's "Thinking Fast and Slow" (2011) and it seems a LOT of the discussion in the book relies on non-reproduced or even reversed results in your list. I hope the book changes significantly in the second half, as the first half is essentially barren after reading your FORRT link.
If you limit your understanding to science that is conclusive, you will paralyze yourself. This is the reality of scientific study; there are exceedingly few conclusions, only more questions, confusions, misunderstandings, etc.
When scientists say we don't "know" anything, this is what they're talking about. Nothing replicates, everything seems more confusing and less clear the deeper you dig, and the further you stray from the "hard" sciences that are easy to experiment, the more true this gets. Psychology, economics, etc. are insanely complex and effectively impossible to test the way we think of a physics or chemistry experiment.
If 'priming' only works every wet Tuesday in February, after someone has had a cheeseburger with no sauce and took their Grandma to the bingo, perhaps there's some _actual_ interesting artefact explaining it...or it could just be random noise.
But the presumption here is that the same experiment is even being run a second time, and my point is that many studies who have the stated goal of replication do so in ways that I find hard to justify.
So even when a study claims to "fail to replicate", I don't think we should just take that on face value.
That goes too far. Some science really does follow the scientific method and documents the methodology well-enough to be replicated.
That’s the basis for giving deference to scientists on some topics — they aren’t just experts, they have subjected their ideas to the cold hard scientific method and others have replicated the results.
Somewhere along the line we demoted science to just be another expertise, where you can just walk around in a lab coat and have social status and people aren’t supposed to question you. Throw in some jargon. No, that’s not science.
They can’t have it both ways. If scientists want deference on important topics, they need to follow the scientific method and not extrapolate too far from established results. No excuses like being “paralyzed” by requiring replicable results.
Looking at that site, though: if I search on "Ariely" I don't get what I would have hoped, which would be a list of his publications that have been debunked. Instead, there are some generalized articles not about him.
Minor point here, the article says Rossi ran the Ten Commandments study. Not sure if there's a different study you mean?
It seems no one can provide receipts (which is bad! due to reproducibility), and since Ariely's name is on the paper and Rossi's is not, the onus is on him to have kept that documentation. He asked that Rossi at least corroborate the details described in the paper, and she refused to do so saying that it didn't align with her memory or her standard procedure at that time.
Maybe the whole scandal is an elaborate memory priming experiment?
However (!) there are a bunch of other Ariely articles which are extremely suspicious and which are suspected of being fake. At this point I would say everything he has ever written should be assumed to be faked.
I expect a broader investigation will reveal more fraud. Apparently Duke doesn't care enough but hopefully some renewed attention to the case will force them to take action.
I think this point is under appreciated. I also want to add another point
- Don't take a (strong) position on something you don't understand or have equally strong evidence for.
We live in a world of specialization, which means a lot of us "are dumb" with respect to a lot of domains. There's nothing wrong with that. But there's a propensity for society to generalize this intelligence, which makes people think that because they can adequately reason about one domain that they can do so in another domain. It is kinda like the Murry Gell-Mann Amnesia, but to ourselves. Look at a domain you _are_ and expert in, and you'll probably realize that a lot of nuance matters, resulting in many things that would be non-intuitive without that high level of detail. Then ask yourself, do you have sufficient nuanced understanding for this other domain topic? I think we know the answer. Opinions are totally fine to have, but it is the strength of them which matters, and places like Reddit and HN are full of people who have been taught that their intelligence generalizes far better than it does, because what we're comparing ourselves to is other people's ability to reason and rationalize and not the distance our knowledge/reasoning is from the truth. i.e. you can out debate someone while being an absolute walnut.
...or else what?
We all live in a world of specialization certainly, but we also live in a world where we must make decisions with imperfect information. This idea of "only take positions on things you fully understand" is a luxury that isn't practical when you realize how frequently (and rapidly) those decisions must be made.
One thing that matters even more than nuance is execution, and a not-so-great idea executed extremely well is much better than a super well researched and understood idea that didn't get executed at all because the person with it took forever getting to that level of understanding.
There is, as in all things, a middle ground. Take positions, make decisions, but accept you'll be wrong decently often, and that's okay.
I'm not sure how to answer this without sounding rude. But you'll make a fool of yourself if you do, put people at risk, and probably lose money.
Considering the other context of your post, I'm not quite sure where your disagreement is. I specify _strong_ as a qualifier to indicate that the strength of the opinion/position (your confidence) should be proportional to your knowledge/understanding/evidence. I explicitly state that having opinions is perfectly fine and would go even further to say that it is impossible to be absolutely devoid of them. Human brains just make conclusions.
To clarify my point, I'm saying that being an armchair expert is not helping anyone. Having some armchair expertise is perfectly fine, but arguing from that position (especially passionately) is generally unhelpful and just adds noise. These are the common posts that you see where someone responds with a very simple explanation that is rather incomplete. You'll often see these highly upvoted as well and better answers buried simply due to their length. The laziness hurts us all! It is better to build habits and communal acceptability to just shut up a lot of times, as the problem with information retrieval is often about accessibility not availability; noise decreases accessibility due to the additional information we must sort through. If armchair comments are given (and they can be helpful, especially on vague topics or where no other level of expertise is present), then admission often helps the person receiving the information as well as creates the ability for better communication to determine higher quality information as well as better allow a more informed specialist to append or correct the basic answer.
The problem we see is not the existence of these armchair expert answers but rather that they are done so with high levels of confidence and the authors often feel the need to defend them, including against actual experts. Corrections or additions are not a critique to one's self, just the knowledge that was laid forth (which is often anonymous, so identity/self-value should play a lower role, but doesn't). Being wrong is not just okay, but should be welcomed as nerds we pride ourselves on being "smart" which means we pride ourselves on _gaining_ knowledge. Not everything needs to be a debate and honestly it is in general a terrible way to acquire or disseminate information.
Being wrong is completely okay, and trying to police what people discuss with one another is a dark road to go down. Besides, where do you draw the line? Do I need a specific number of course hours completed? Some number of books read? Three or more articles published on the topic?
Discussion, particularly in writing, is a form of thinking, and thinking with others is infinitely more productive than thinking alone, so if what you're asking is for folks to stop thinking about topics they're not experts on, I'm sorry but no. That's not going to happen.
Thank you. If you have any suggestions as to how to make my point clearer, I'd greatly appreciate it. I hope it is clear now, but obviously I can always improve. I do want to promote discussions, but that also means wanting to decrease arguments. Especially arguments that are uninformed and ones which has a clear answer (or lack of) from the domain experts.
Where to draw the line of domain expertise is not clear, you're right. Academia is one example of a way to pass that line but I wouldn't say the only means as it is just a method that correlates with sufficient knowledge gain. But I also wouldn't say that watching some YouTube videos does not constitute expertise (more explicitly, videos by science communicators rather than lectures). The boundary is fuzzy, but that should put pressure to reduce confidence (which is good for science and truth seeking) rather than increase it.
Discussion is not only fine, but actively encouraged. We're having one right now and at least on my side I don't feel like this is a argumentative. I tried to clarify this previously but I guess this is difficult. I'm not sure how to make myself more clear so all I can do is ask that you read more carefully with the updated knowledge that there's a substantial amount that I agree with you on (differences are in finer points) and request that you point out specifically where what I said may be confusing or resulted in misinterpretation. Hoping that this can facilitate a better discussion, if you still wish to have one.
She wasn't an author, so she had no reason to be involved w/ (for example) the referee process. She was made aware that this was their claim when someone contacted her.
She also works in a slightly different literature IMO. Even if she had read the article, it probably didn't say "the data gathering was done by Prof. Rossi at UCLA"
How could he have successfully executed this fabrication? Doesn't he work with entire teams of others, don't others review his work, his analysis, the data collection, etc.? The idea that one man, alone, could pull off such a large deception seems, to me, to be pretty farfetched.
Some additional context here is that in academia, there has been a lot of pushback on the burgeoning field of behavioral economics as a result of political viewpoints; nudge is inherently a liberal concept, and puts conservative viewpoints of personal responsibility in a negative light.
Combine that with the ever-present reproducibility problem in all of psychology, and the economists who were made to look like fools (in their view) in the earlier years of the 21st century are striking back with exactly these kinds of arguments (economics isn't strictly a "hard science" insofar as it can't really run experiments the way physics can, so it's easy for economists to point out experimental issues).
Ariely's books aren't "BS", and throwing that in there kind of shows your hand. Large portions of his books are explanations of the work of others, and even if everything this UCLA professor says is true, that doesn't invalidate a single thing he wrote about in his books, other than perhaps specific citations of his own work (of which we're talking about a tiny handful of hundreds of published and peer reviewed studies he worked on with thousands of other people).
Huge swaths of behavioral economics have replicated successfully. Care to cite peer-reviewed studies related to astrology that have successfully replicated?
I specifically said don't throw out nudges - some of them work and replicate e.g., retirement savings defaults, while others are BS like the honesty pledge. Real experimental/behavioral economics is not this media-attention-grabbing garbage anyway. As I said, I have friends in the field. They don't do this BS.
> Doesn't he work with entire teams of others, don't others review his work, his analysis, the data collection, etc.?
No, he doesn't. See the Data Colada responses from his three coauthors on the car insurance paper. All admit he was the only author to have data access. If you can find it, check the earlier, separate investigation into his other fabricated paper w/ Delta Dental where the insurer itself said "we didn't give him that data."
> Some additional context here is that in academia, there has been a lot of pushback on the burgeoning field of behavioral economics as a result of political viewpoints; nudge is inherently a liberal concept, and puts conservative viewpoints of personal responsibility in a negative light.
I disagree on many dimensions w/ your characterization of behavioral econ AND the entire point is irrelevant here.
> economics isn't strictly a "hard science" insofar as it can't really run experiments the way physics can, so it's easy for economists to point out experimental issues
I can introduce you to ten experimental economists who would disagree. Economists have also been pioneers in figuring out when non-experimental situations parallel experiments enough that we can make the same inferences.
> Ariely's books aren't "BS"
Please continue enjoying them if you like them, but I don't have any idea how you'd separate fact from fiction. Ariely is not the first researcher in this exact field who has been revealed to be a fraud.
I'll only add that compared to, say, traditional economics we don't have a sense of how unique the problem is to social psych/organizational behavior. In the past decade psychology has done a lot more replication attempts and data auditing than have other fields in the social sciences. So it could be that other fields are equally problematic, but we don't know.
FWIW... In plenty of economics papers you can't do a meaningful replication b/c there may be no analogous natural experiment. You can only do analysis of the same data. Such "replications" do not always lead to the same results.
The more experimental parts of the econ literature are now very big on pre-registration (esp. in development, e.g.). Not a panacea by any means, but an improvement for sure.
And there are generally higher standards around making code available now. If you work in this area and read the AER you may be aware of a recent retraction in PF due to coding malpractice. The system has improved but is far from perfect.
in terms of defamation, I don't think you can summarize "accused" as "is a fraud"
Noted study in psychology fails to replicate, crumbles with evidence of fraud - https://news.ycombinator.com/item?id=28264097 - Aug 2021 (102 comments)
A Big Study About Honesty Turns Out to Be Based on Fake Data - https://news.ycombinator.com/item?id=28257860 - Aug 2021 (90 comments)
Evidence of fraud in an influential field experiment about dishonesty - https://news.ycombinator.com/item?id=28210642 - Aug 2021 (51 comments)
Related but different (I think?):
Harvard ethics professor allegedly fabricated multiple studies - https://news.ycombinator.com/item?id=36665247 - July 2023 (214 comments)
Harvard dishonesty expert accused of dishonesty - https://news.ycombinator.com/item?id=36424090 - June 2023 (201 comments)
I'm very happy to see this type of scrutiny come to headline research but I really wish it begins at the beginning, with funding agencies and journals. And at the very least the moment a university sends out a breathless PR, the media all scrutinize the data provenance... A boy can wish
It’s a shame that the worst that can happen to these frauds is, typically, job loss. The incentive to cheat is way higher than the cost times the likelihood of getting caught. There should be criminal penalties at least as severe as other white collar crimes.
One of my undergrad psych research projects was focus on testing some of the Kahneman & Tversky anchoring/adjustment stuff. We were able to replicate some things related to the theory itself based around economic estimates, but also had some "control" questions in our surveys (one example: "estimate the average weight of a human brainL). Our experience was that some populations just sucked worse at the kinds of estimation tasks we were using. We had no good theory for gender-based differences in estimating the weight of a brain, and in retrospect it was probably just people confusing ounces with pounds as a half-remembered number for weight... but no one gets published by admitting that their experiments were probably just poorly designed.
I don't mean that in a LMGTFY way, just in a "here's a tool that can help".
It's ironic that you try to attribute unprofessionalism to me using unprofessional tools. The experiments conducted by Prof. Siniver are not reproductions of experiments I conducted, but rather experiments that are clearly different in many essential differences. A claim for refutation through a completely different experiment is a provocation without a professional basis.
Researchers are required to keep data on studies for 5 years. I do not have data from studies that were carried out 15 years ago or more, and I have no way to find them either. This is not to criticize my work, and any attempt to do so is manipulation of the viewers.
I don't do the research alone. Throughout 30 years of research I had 248 scientific writing partners, among them the most respected researchers in the field. I led 2 labs at MIT and another lab at Duke where hundreds of interns, students and research assistants worked alongside me. You understand that if the claim is that I fabricated studies, you accuse a huge group of academics from dozens of universities in the world, when the only "evidence" for this is that I failed to prove "that I don't have a sister", that is, to provide documentation that I no longer have.
The experiment in which Prof. Amy Droley assisted took place over 15 years ago. I don't know why she remembers the things as she described to you. At first she didn't remember taking part in it at all, and only later did she remember. I and the other two professors who signed off on the study remember the details as they are described in the published article.
In the shredder test we met all MIT conditions. There are many methods to maintain the anonymity of the participants and not just one way as you presented.
There is a difference between academic articles, in which every word and data is carefully chosen, and between a free conversation in front of an audience, in which I convey the main points and do not go into all the details, for the important purpose of making science accessible. Your attempt to "catch" me with one word or another and take it out of context only reflects a desire to slander and nothing else.
Regarding the dentists, these things were said to me and Dr. Janet Schwartz in a face-to-face meeting, by senior company officials, as Dr. Schwartz also confirmed to you. I repeated the things that were said to me in media interviews because they seem important to me. The blog of my institute at Duke University accidentally mixed in an incorrect quote and I asked for it to be corrected.
We recently launched together with Harbor Capital various indices that try to improve the way companies treat their employees and also create value for investors. I forwarded to you the thorough research conducted by one of the best research units in the world, of the investment bank JP Morgan, on the data on which we were based, a study that reached the same conclusions as us.
Prof. Omar Moab holds opinions about many people, but to my understanding he is not an expert in the behavioral sciences and it is not clear why he was specifically invited to give an opinion on my research.
Only once was an error discovered in one of my studies, out of over 150 academic articles. My research partners and I realized that something was wrong with the data, we noticed it and shared the data transparently. I took responsibility for the incident, but I still don't know how it happened.
In my more than 30 years as a scientist, I participated in hundreds of studies, wrote seven books and had the great privilege of helping many organizations improve people's lives. I am proud of my work and contribution.
I don't like to see trials by Twitter. I also don't like to see institutions of academia shirking their responsibilities.
(Full disclosure: I met with Ariely a long time ago, and he seemed decent, but I know almost nothing about that field.)
Much more effective than endless pontificating and theorizing by people who have no idea what the fuck they're doing but haven't publicly fucked up yet so we trust them in reverence of their credentials. The amount of institutionalized deception that has permeated the field of psychology since its inception should scare anybody away from ever taking it seriously.
The Latin phrases don't all appear to relate to what's going on. Don't know what's up with that, especially the one from Horace's Odes.
Specifically
> Unfortunately for Ariely, the Israeli press won’t let the issue die and has been digging into this issue of provenance rather deeply.
Haven't noticed any stories about this issue, this year (2023), in the Israeli press.
You can even do ethical virtual experiments with the data of 3 billion people. Just specify the situation, the person and what's to happen.. Voila. All things happen regularly at some point..
You really need to clarify this.
> The government is trying to control your mind through the internet
Group of people who both have the theoretical training and practical experience is probably pretty small so private/public crossover isn't unreasonable.
I would think instances of fabrication are quite low — it’s high-risk, medium reward. Even if it goes undiscovered, academia is a marketing game, and you can’t control what other people feel is impactful or not. It’s very easy to do bad science without falsifying data, which is probably the deadliest sin in academia.
Yes, which is why I said the frequency of discovery is low. Seeing a story in the paper about fraud every day doesn't really tell us anything about the rate of fraudulence, which I was hoping would become transparent with my point.
I'm still glad I didn't go into academia for many other reasons, but I don't think I dodged an ethics bullet.
Reputation probably matters more for academia (people go into it because they think the research is important, not for the money, and they think they're standing on the shoulder of giants and not amiable bullshitters taking the credit for findings actually created by their overworked research assistants kludging the data), but if you think academia is full of lies and half truths relative to other professions, you haven't encountered many startups!
In my experience, this is absolutely not true. I mostly only have first-hand observations in the economics, finance, and marketing disciplines. They left me EXTREMELY cynical about all academics.
I have seen:
1) A well-known professor at an Ivy-league B-school openly talking to his class about how the NAR funded his latest study on the real estate market, which (of course) found that the market was not overvalued. This was 2007. The co-author on the paper was the dean of the B-school. He is no longer the dean, but only because he seems to have moved on to more lucrative consulting roles.
2) An older professor, who was one of the first to be in the group labeled "behavioral economics," when asked why he wasn't writing a book focused on more mainstream readers like some other such big names were at the time, STRONGLY hinting if you filled in the blanks although not outright saying that their ethics were questionable, they were all forming hypotheses that were useful for for consulting dollars and dredging up data to support it, and he was too old school (and too old) to be interested in playing that game.
3) A full cohort of marketing grad students (6-7) talking openly about how the system is messed up, and to get a job, you absolutely have to have a good dissertation with a strong results, but to avoid p-hacking allegations, you have to shoot in the dark. So you're left in this situation where you commit financial and career suicide if you DON'T make up data to support your thesis. It is not rare. It is an open secret. No one likes it. They hate it. But they also feel like they are the stupid idiot if they try to fight against it alone. It is how the system works. Only a true saint would accept data that actually does NOT support their hypothesis because it makes it unlikely they will get a job in a very competitive market. Some people do report non-findings, but they are rare. IMO, academic departments should specifically seek out graduates who were willing to report that and stuck by it. But that is not the way the world works. As another poster pointed out, we all like stories, and no significance means no good story to tell.
1) Yeah, lots of people missed the crisis.
2) Yes, when funding doesn't come from the government it comes from industry. And then you need to find hypotheses that are interesting for industry.
3) That's how science works! You don't know what is true or false. You need to take your shot, see if the experiment you run works out. Sometimes you can be the smartest person in the world, put in the most work, but be in an area asking a question that doesn't pan out. It happens!
There are many ways to mitigate this. For example, you can often make experiments where it doesn't matter what the answer is. It's always interesting and publishable regardless of how the statistics work out. But, there is definitely luck involved, not just skill.
> Only a true saint would accept data that actually does NOT support their hypothesis because it makes it unlikely they will get a job in a very competitive market.
Only a scientist would! That's the game we play to discover new truths. And the vast majority of scientists do this.
Pretty much every idea that I have ever had is bad! Yet, somehow, I managed to get a PhD and now graduate many of my own PhD students. The trick is to learn to fail quickly, to ask the kinds of questions that are interesting no matter what, and to develop a sense for the types of questions that are more fertile.
Your take on the last point is not so different from mine. I think you misunderstood. The incentive to change data to avoid a finding of no significance is very great. The risks are small. Yes, good science would be to report it accurately. It almost NEVER happens. It is widespread. More so than people think. These two dishonesty researchers are just the tip. A lot of people in academia are staying silent because they don't want people looking into their own research. False data is not the exception. It is endemic.
That's objectively false. Batteries keep getting better. Cancer survival keeps going up. Computers keep getting faster. By almost any metric you pick, research in almost every single discipline is a massive life-changing success! Not just a success, a wild success.
> Your take on the last point is not so different from mine. I think you misunderstood. The incentive to change data to avoid a finding of no significance is very great. The risks are small. Yes, good science would be to report it accurately. It almost NEVER happens.
Nonsense. I've been a scientist for two decades at multiple universities. I've collaborated with hundreds of people. Never has anyone tweaked data on anything we've ever worked with. We usually do the opposite, find that something works and work as hard as possible to trash our own work so we can't publish it. We spend more time trying to disprove our own work before publishing it than getting any positive results.
> It is widespread. More so than people think. These two dishonesty researchers are just the tip. A lot of people in academia are staying silent because they don't want people looking into their own research. False data is not the exception. It is endemic.
That's a conspiracy theory just like aliens landing on the moon and vaccines causing autism. Somehow, hundreds of thousands of people work together to stay silent and not leak. People who frankly hate each other's guts and want nothing more to do than to disprove each other's theories and show they're superior. Please, this doesn't stand up to even the most basic scrutiny or thought.
Every scientist that I know wants to do science. Wants to be remembered for contributing to our understanding of the universe hundreds of years after they die. You don't do this by lying, you do it by doing the best work you possibly can.
Just wait for people to dive into the false equivalence of scales and measures used in political research. It's very much "make it up and see if it sticks." The goal appears to be Psychology Today, not scientific rigor. I wrote about this in graduate school, as part of my hefty statistics coursework, it bothered me so much.
https://en.m.wikipedia.org/wiki/Half-life_of_knowledge
P.S. Ironically, the title of the paper is "The dishonesty of honest people" (it's 15 years old by now).
Lower values mean the field is advancing quickly. Rapid scientific advancement is a good thing for knowledge generally. It's merely annoying for people in the field because they have to do more work to keep up-to-date. :)
It's not a measure of things being proven wrong, but more about newer better-predictive theories making older theories obsolete.
What are ppl even getting so excited about? I mean this is a field which teaches kids to recite - "half the money I spend is wasted; the trouble is I don't know which half".
B) Moreover, our brains are also hardwired to storytelling[2][3]. When presented with a nice narrative, we feel rewarded.
C) Finally, to quote Charlie Munger, "Show me the incentive and I will show you the outcome". People in academia focus on the # of papers published.
Put (A), (B), and (C) together and we can explain news such as these.
[1] https://www.psychologytoday.com/us/blog/the-athletes-way/201... (hopefully this study is legit :))
[2] https://www.tadigital.com/insights/perspectives/art-storytel...
How could neuroscience possibly prove (or even provide any evidence in favour of) this? Did they use identify the gene for trust and produce a knock-out human?