I disagree with Turing and Kahneman regarding statistical evidence (2014)
statmodeling.stat.columbia.edu
statmodeling.stat.columbia.edu
https://replicationindex.com/2017/02/02/reconstruction-of-a-...
Is it? That's the very basis for error that begat this article. In fact, Turing's and Kahneman's minds were too open. They didn't express sufficient reservation by demanding more rigorous tests of those claims.
Perhaps a better maxim would be, "It's good to have a mind that's open just enough to entertain the impossible."
This article mentions Walter M. Kotschnig as the author of the earliest known instance of the saying, but notes that even Kotschnig seemed to be quoting something that was already known at the time:
https://www.skeptic.com/insight/open-mind-brains-fall-out-ma...
"The problem with having an open mind is that people will insist on coming along and trying to put things in it."
What you are doing here is that you are betraying two huge hidden assumptions that you implicitly make. First is that you are most likely expecting to obtain “mechanism action” in some kind of mathematical terms, which implicitly assumes ability to use mathematics to explain the observed universe. Second is that you are looking for “mechanism of action” for ESP in terms of known physical forces and theories. Here, you implicitly assume that known physics is all there is.
Now, I do not believe in ESP either, my point is only that you are demanding rigor from ESP you don’t from electromagnetism.
Another example of a "hypothesis" which has no mechanism of action (and I therefore do not regard as a hypothesis) is the "Stoned Ape Hypothesis". It claims that the ingestion of psylocibin explains the sudden increase in size of the brain at one point in our evolutionary history. But it does not actually give a mechanism for how your or my genes today would be affected, it just says that hunter-gatherer culture would have changed and there would be more group sexual activity. If I were to be presented with this "explanation", I would not expect a dramatic effect on brain size, because there would be less sexual selection going on.
Homeopathy also strikes me as a claimed effect without plausible mechanism.
Physics has formulated models of how electromagnetism works and from that model, predicted other effects that could be replicated in experiments as well. That's what a 'mechanism of action' means here. The predictive power of a model, by whatever means it's formulated makes it useful.
There are infinitely many ways to model electromagnetism but only a few of them are useful because they allow to make theoretical predictions that then also match reality.
> People have erroneous intuitions about the laws of chance. In particular, they regard a sample randomly drawn from a population as highly representative, that is, similar to the population in all essential characteristics. The prevalence of the belief and its unfortunate consequences for psvchological research are illustrated by the responses of professional psychologists to a questionnaire concerning research decisions.
(emphasis mine)
Not only does the statement say that “people” make this mistake then go on to cite a questionnaire only of professional psychologists, but the questionnaire is the prototypical non-random random sample the very psychologists taking it are allegedly proving misinformation about. Questionnaires select for people with the time, inclination, and attention to start and then finish a questionnaire.
Infamously, experiments held at universities bias towards undergrads. Experiments that reward participation bias towards people motivated by the reward. Experiments that don’t reward participation bias towards people good-natured enough to contribute just for science.
Randomness is hard.
E.g. if I said "I don't believe in the Holy Book because in verse 7 it says that one cannot trust anything written in any books". Isn't that an analogous reasoning?
I think these examples/arguments are ultimately about exposing a liar-paradox statement, and when you can show such a statement you have proven that something isn't right.
Yes. And might I say everything has a statistical bias. Picking people for an experiment is not like throwing dice
Your sample will always be biased, and it would be crazy to try and get a random sample from "the world". (and even then you have temporal bias - you can't run your experiments on people from the XIX century)
> The idea you should focus on, however, is that disbelief is not an option. The results are not made up, nor are they statistical flukes. You have no choice but to accept that the major conclusions of these studies are true. More important, you must accept that they are true about you.
So so arrogant, and he ended up being wrong about it too. It's very hard to take him seriously after reading this.
Yes. It is. But as another poster has noted, he largely recanted in 2017 [1]. And this is to his credit. Few scholars -- and fewer prominent ones -- have been willing to reverse course in as public a fashion as this.
[1] https://replicationindex.com/2017/02/02/reconstruction-of-a-...
This is a great insight.
So he admits that the studies don't replicate but he still believes in their conclusions? Somehow that makes him look worse, not better, in my eyes but maybe I'm too critical.
Admittedly, the experiments he conducted and cited were found to be flawed. But I think his point, as I understand it, still stands.
Blind faith in the results of a study is not good science. If a study is counter intuitive, one should question it.
Restated: "I'm correct, about us all being prone to cognitive errors, where you like it or not."
> My reason for writing this letter is that I see a train wreck looming. I expect the first victims to be young people on the job market. Being associated with a controversial and suspicious field will put them at a severe disadvantage in the competition for positions. Because of the high visibility of the issue, you may already expect the coming crop of graduates to encounter problems. Another reason for writing is that I am old enough to remember two fields that went into a prolonged eclipse after similar outsider attacks on the replicability of findings: subliminal perception and dissonance reduction.
> I believe that you should collectively do something about this mess. To deal effectively with the doubts you should acknowledge their existence and confront them straight on, because a posture of defiant denial is selfdefeating. Specifically, I believe that you should have an association, with a board that might include prominent social psychologists from other field. The first mission of the board would be to organize an effort to examine the replicability of priming results, following a protocol that avoids the questions that have been raised and guarantees credibility among colleagues outside the field.
His more recent (2022) retrospective on the episode is also pretty interesting: https://www.edge.org/adversarial-collaboration-daniel-kahnem...
Yeah, your reaction is. The whole point was to be humble and accept that you might be wrong. In particular, if well-done research contradicts your prior belief, then you "have no choice" but to consider that you might have been wrong.
It turns out that the "well-done" was not the case here, but I find it hard to think your post was in good faith.
> And that’s interesting. When stupid people make a mistake, that’s no big deal. But when brilliant people make a mistake, it’s worth noting.
Maybe not. In the words of the author,
> maybe these are real effects being discovered, but you should at least consider the possibility that you’re chasing noise. When a striking result appears in the dataset, it’s possible that this result does not represent an enduring truth or even a pattern in the general population but rather is just an artifact of a particular small and noisy dataset.
I'm not comfortable with the idea the unmatched creative engineering powers of mother nature did not discover something as simple as radio.
We could power down the electrical grid and do the experiment? But I can just call everyone, what use is telepathy?
The cat is in front of the house when someone arrives (which is never at the same time sometimes skipping a day) but the rest of the day he is nowhere near on the camera footage. For him there is a point, it means food!
Sheldrake would love it.
--Amos Tversky
Maybe
I Disagree with Turing & Kahneman Regarding the Strength of Statistical Evidence
https://www.lesswrong.com/s/BQBqPowfxjvoee8jw/p/ythFNoiAotjv...
Tentatively, that person may be Joseph Banks Rhine.
I think that today we overestimate how much we know, and discount the existence of “supernatural” phenomena that we don’t fit into our model of the universe.
Karma (or something like it) is a thing that probably exists; but we do not have the necessary concepts or tools to qualitatively or quantitatively describe how it works.
Here's one definition of "supernatural order" [0].