Hacker News Reader

Comment by yters | Hacker News Reader

I am curious why it has never been claimed. At least ESP seems to be very well documented from a scientific perspective.

View on HN

This is exactly the problem the Randi Challenge tries to solve. There are reports and papers from time to time about ESP, but as far as we know they are just flukes, p-hacking, bad experiment design or plain lying. Some of them get press coverage, but the press just copy&paste the press release.

So the idea is to make some challenge that where there is a magician team to detect frauds, a statistics team to avoid p-hacking and minimize the chance of flukes, something equivalent to pre-registering and other security measures to ensure the result is legit.

What is the best scientifically documented EPS experiment?

Dean Radin has a fascinating book looking at the evidence for ESP. There is quite a bit since the topic has obvious military applications if it works. There are a number of metaanalyses he looks at which indicate it is not p hacking. The book has some interesting brain impulse graphs which show a clear anticipation of randomly chosen images, when the image is distressing.

As for Randi, who watches the watchers? Is he truly disinterested in his challenge, or are his examinations also biased? Additionally, the research I've seen shows a statistically significant, but faint result, so it may not be easy to reproduce in Randi's setting.

All that being said, I have not personally been able to reproduce the results, even though the claim is that almost everyone has a low level of latent ESP capability, which can be amplified through training. SRI even has an ESP training app if you want to try verifying the claims.

Are you familiar with the critiques of Radin's works? They're plentiful.

I found this nature article which claims he has inflated the p values, because Radin fails to include hypothetical negative tests in his sample analysis of the file drawer. Seems pretty clear, although I don't understand it yet, so I will need need to think about it further. That being said, I would still have trouble accounting for all the various pieces of evidence in his book. E.g. the anticipation graphs are pretty remarkable.

https://www.nature.com/articles/39784

I made a search in Google and picked one of the articles by Radin https://www.researchgate.net/publication/254203087_Evidence_...

The first thing I though was a bad synchronization, but the anticipation was 3 seconds, that is big enough.

The second possibility is the filtering process. It's weird and it's a common source of error. The interesting part is in page 21-22:

> Low consistency responders

> Given that there was better evidence for presentiment from the consistent responders, one wonders how the inconsistent responders performed?

> Examination of the raw data revealed that in most cases, the inconsistent responders were so labeled because one or two of their calm trials had exceptionally large within-trial variances (i.e., variance of the physiological measure from the moment of the stimulus to the end of the cool-down period). Because of this observation, as a post-hoc test we examined the mean presponse correlation for SCL for each inconsistent responder, after removing the one calm trial with the most extreme within-trial variance.

> Table 9 shows that the effect of removing this single trial from each of the total of 49 inconsistent responders (across both experiments), indicated as I* in the Table, dramatically changed the overall results.7

> All data from all subjects combined resulted in a nonsignificant zpre-r = 0.03, whereas removing the single highest-variance calm trial from each of the inconsistent subjects (leaving 98.5% of all data) resulted in a combined significant zpre-r = 2.99 for SCL.

So the problem is that they are filtering the people that made a random move just before seeing a calm image, but they are not filtering the people that did a random involuntary move just before seeing an emotional picture.

Very interesting. It is suspicious, but I also wonder if removing 1.5% of the data is enough to produce a statistically significant effect? In other words, filtering is definitely happening, but is it enough to account for the effect?

Also, he has a number of such studies. Are they all suspect due to filtering, or are some filter free?

If you remove the 1.5% of the data at random, it will almost sure no make a change that is statistically significant. If you cherrypick the 1.5% to remove, it's possible.

To simplify the examples, I'll assume that each person saw the same amount of clam and emotional photos, and also that half of the photos were in each class. (In the paper the number varied from person to person and the ratio was approximately 2 to 1, the conclusion is similar but it's more difficult to write.)

They use an analog sensor, but they are essentiality counting how many times a person reacted before seen an image. It can be a premonition or a sneeze or any other cause.

In the part I posted, they selected the people that reacted exactly once before a calm photo. With this selections it's not clear how many times each person would react before an emotional photo. They compare the data, and the difference was not statistically significant, so they reacted approximately once before an emotional photo.

This is somewhat a coincidence, there is no theoretical reason for this, but if you assume some sensible distribution of the chance to react randomly before a photo and use some hand waving, this is not very surprising because if people have no PSI abilities, they'd react approximately the same number of times before each set.

So now you have a bunch of people that reacted exactly once before a calm photo and approximately once before an emotional photo. We all agree that this is obviously not a proof that they have some premonition.

Then they remove the 1.5% of the reactions before a calm photo and keep all the reactions before the emotional photos.

And now you have a bunch of people that never reacted before a calm photo and reacted approximately once before an emotional photo. So there is a clear difference in the reactions before the photos and they misinterpret this as a proof of premonition.

They actually use an analog sensor, so there is more noise involved and makes everything more fuzzy. If the noise level were too high it could have overshadow the bad cut they made in the data, but the noise was not so high.

> Also, he has a number of such studies. Are they all suspect due to filtering, or are some filter free?

I don't have time to read every study he published, but if you link one (with full text) I'll try to see if I can find an error.

Thank you, I will dig through his studies and see if I can find one that does not use filtering.

Ah, well, there's your problem: ESP has not been "very well documented from a scientific perspective", unless you mean that in the negative sense.

Have you looked at the research in this area or are you just winging it?

Somewhere in between those two extremes. I'd love to see/hear what you think qualifies as "very well documented from a scientific perspective".

In lieu of just linking to studies, I'll link to a great Slatestar Codex post: https://slatestarcodex.com/2014/04/28/the-control-group-is-o...

This would naively seem to be a yes to psi. Bem got a strong signal and Scott doesn't believe in psi so he thinks science in general is deeply flawed as it currently works.

That is absolutely not my takeaway from that article.

What is your takeaway? Scott at the end says he doesn't believe is psi despite the results because Coyne says psi is crap. That doesn't sound like checking your priors at the door.

At any rate contra your original claim it appears a psi effect has been extensively scientifically documented.

Should we "check our priors at the door", though? When neutrinos were "observed" to travel faster than light, the most appropriate conclusion wasn't "well I guess we were fundamentally wrong about just about everything we've found in the last century", but was probably something like "okay, but can I take a closer look at your data". With priors left aside, the foundations of physics are questioned. I think the same holds for psi: if you abandon your priors and just make conclusions from which way some data swings, there is something very different about the universe than what has previously been attested to.

I think Scott's analysis of "my priors against this are so strong that evidence in support of it looks more like evidence against the validity of the methodology" broadly matches with mine.

Forgive me if I am misreading you, here.

Yes, that is my takeaway. Psi is heavily evidenced by the scientific literature, and is dismissed because it goes against the priors of the scientific establishment. Not dismissed because there is anything inherently faulty with the research, at least compared to the standard scientific milieu.

Section V is where he explains why he doesn't believe the results.

"I looked through his list of ninety studies for all the ones that were both exact replications and had been peer-reviewed (with one caveat to be mentioned later). I found only seven...Three find large positive effects, two find approximate zero effects, and two find large negative effects. Without doing any calculatin’, this seems pretty darned close to chance for me.

...

That is my best guess at what happened here – a bunch of poor-quality, peer-unreviewed studies that weren’t as exact replications as we would like to believe, all subject to mysterious experimenter effects."

Why would an even proportion of positive to negative be considered chance? If half the entrants won the powerball and half did not, I would not consider that chance.

Because "large" here is still "very tiny". Large with respect to "p-values" isn't the same as "actually large". Equating the "positive results" to "winning the powerball" is not a fair comparison. If there were Powerball-level results here we wouldn't need further studies to confirm the effect exists.

Radin claims in his meta analyses he gets powerball level results, much beyond that in fact.

Either these are some quite incompetent and/or deceitful researchers, or they have found something we currently cannot explain.

I would love to find out that ESP really is possible. Unfortunately, I haven't seen any proof so far.

I believe that the CIA used to experiment with it, but nothing useful came out of it.

I don't have direct proof, but groups like SRI have published results in IEEE.

Reply on news.ycombinator.com