Although many of the results in Fast and Slow didn't hold up, Kahneman was always refreshingly open and honest about that, and keen to identify the limits of knowledge.
Which surely is one of the best things you can say about a scientist.
Which surely is one of the best things you can say about a scientist.
One half-joking comment he made about science in the real world vs some idealized notion of it has always stuck with me. In a discussion about whether the results of some paper conflicted with some model or theory of cognition, he mused that scientific progress in psychology (and other non-hard sciences) was really about embarrassing rivals with competing models. No high-level model was ever stated precisely enough to rule out some particular finding; you could always tweak your theory a little to accommodate it. It's just that at some point, you might be too embarrassed to do so.
"As soon as you present a problem to me, I have some ready-made answer. Those ready-made answers get in the way of clear thinking, and we can’t help but have them." – Daniel Kahneman
His work got torn to shreds with science, what did you expect him to do?
It really should be embarrassing rather than acceptable. It doesnt matter how honest you are AFTER you are caught.
Between the replication crisis and his name on some bad papers, the guy seriously didn't care about correctness as much as interesting-faux-science.
Further, the confidence he extolled about his now debunked ideas make him a charlatan. This person was a bad scientist. If we esteem people who don't check their data and influence millions of people with falsities, we are going to create a society with low trust.
Just look at this thread, the man lost respect among the people in the know. There are a few people clinging onto 'well just because its not true, doesnt mean I didn't find it interesting". I'm not sure what we get out of promoting anti-science scientists.
https://slate.com/technology/2016/12/kahneman-and-tversky-re...
https://replicationindex.com/category/thinking-fast-and-slow...
> "Table 1 shows the number of results that were available and the R-Index for chapters that mentioned empirical results. The chapters vary dramatically in terms of the number of studies that are presented (Table 1). The number of results ranges from 2 for chapters 14 and 16 to 55 for Chapter 5. For small sets of studies, the R-Index may not be very reliable, but it is all we have unless we do a careful analysis of each effect and replication studies.
> Chapter 4 is the priming chapter that we carefully analyzed (Schimmack, Heene, & Kesavan, 2017). Table 1 shows that Chapter 4 is the worst chapter with an R-Index of 19. An R-Index below 50 implies that there is a less than 50% chance that a result will replicate. Tversky and Kahneman (1971) themselves warned against studies that provide so little evidence for a hypothesis. A 50% probability of answering multiple choice questions correctly is also used to fail students. So, we decided to give chapters with an R-Index below 50 a failing grade. Other chapters with failing grades are Chapter 3, 6, 7, 11, 14, 16. Chapter 24 has the highest highest score (80, which is an A- in the Canadian grading scheme), but there are only 8 results.
Kahneman to me always struck me as the one eyed king of the replication crisis. Yes he fucked up but he fucked up notably less than his contemporaries and most of his work is still readable.
R-index guys said [0]: "Table 1 shows the number of results that were available and the R-Index for chapters that mentioned empirical results."
Chapters where estimated R-index < 50: Ch 3,4,6,7,11,14,16
Chapters where estimated R-index > 50: Ch 5,8,9,12,17,24
Chapters that don't cite empirical results (by Kahneman, or who?): 1,2,10,13,15,18-23, all of 25-38
As to the chapters that had empirical results, and had an estimated R-index > 50: scores of 55, 57, 60, 62 are really scraping by; saying that means they "probably replicate" is setting the bar really low, even quoting Tversky and Kahneman (1971) back at themselves. (The R-index guys say "Even some of the studies with a high R-Index seem questionable with the hindsight of 2020.")
As to whether he was the one-eyed king of the replication crisis, he certainly started speaking out in 2012 [3] after the social priming scandal broke; did insiders have suspicions about non-replicability before that and should people have pushed back more, earlier? The fallout from the Francesca Gino and Ariely scandals continues.
[0]: https://i0.wp.com/replicationindex.com/wp-content/uploads/20...
[1]: Chapter listing: https://catalog.lib.uchicago.edu/vufind/Record/8467452/TOC
[2]: Chapter summaries by Conor Dewey: https://www.conordewey.com/blog/every-chapter-of-thinking-fa...
[3]: Nature (2012) https://www.nature.com/articles/nature.2012.11535
This is extremely irresponsible.
I did see a lot of charlatans in this thread fail to appreciate the broader context of the replication crisis and failed to appreciate how unscathed Kahneman was by it because he was being careful when his peers were not and long before people started judging him with the wisdom of perfect hindsight. Of course if they wrote such a book they would only express their ideas with timidity and never make a mistake.
I read his book alongside a guide as to what in his book could be ignored. I knew every damning word people said about the man before I read a word he said and left impressed.
People even publish studies on un-replicated research! There might be a lot to be said about his research, but I disagree that publishing your research is the worst thing you can do. Maybe there wouldn't be any replication of his studies if it hadn't been for his books.
What do you think?
It's a pretty sad state of affairs if the best that can be said about a person with his reputation, and about the process overall, that the system managed to catch his low quality output decades down the line.
Most of the "underpowered studies" are in the priming-related chapter, called "The Assoviative Machine". The rest of the book is still worth a careful read.
(I had to make this same correction here several years ago. I didn't look up the comment to link it here.)