The Counterclockwise Experiment

"...this widely-cited and influential work was never published in a peer-reviewed journal. The findings were instead reported in Langer’s 2009 book ‘Counterclockwise’. Despite the intriguing results, this study was never peer-reviewed or replicated. In 2019, a protocol for a replication was published, but at the time of writing the results are still pending."

Indeed, she appears to have a history of doing experiments that never get published in peer-reviewed places, but citing them in her books.

Send two more groups on a week long retreat, tell them some quack theory and see if they improve.

i'm working my way through [1] to try and internalize the ability to recognize the weasel. The part about a test that's 80% accurate for positive, and 9.6% false positive rate made intuitive sense to me, and yet something like "15% of doctors get it right."

The excuses abound: You need a "lab", there's no money in replication, "science marches on[2]", and so on. It's just a cover for an alternate economy, at this point. I'm sorry if you're a scientist doing actual science and your entire foundation is being chipped away at by these heathens.

  [1] https://www.norvig.com/experiment-design.html posted here earlier this week.
  [2] https://tvtropes.org/pmwiki/pmwiki.php/Main/ScienceMarchesOn - "New discoveries make old theories obsolete and stories retroactively inaccurate."

Also people vastly misunderstand the replication crisis. "Soft" sciences are harder to actually do science with. It's harder to measure things of humans vs an electron for example.

But even in chemistry, there's a very high chance that if you take a random paper, even from the 60s say, and try to do it yourself you do not get the result from the paper, but that doesn't mean the paper is wrong, it can just as easily mean that the paper did not include some important environmental condition or procedure or whatever.

NileRed and the guy from Explosions and Fire have both had difficulty reproducing random chemistry papers. Does that mean chemistry is this big lie and narrative propped up by lying academics?

No, it means science is freaking hard.

Every paper has a nonzero "failed replication" rate. Realms of science where it is harder to measure things and control the experiment have vastly higher expected rates of replication failure. People who do science every day understand this and work with it, and it is not a problem.

Consider this; China has a HUGE replication problem with their science, with significantly more actual fraud than we have in the US.

Has it hampered their advancement?

Science is usually self correcting in that, if you put out fraud, anyone trying to build off your work will fail. This is what happened with several past famous fraudsters in science.

Scientists do not even consider "peer review" to be a real quality signal of any sort. It basically wasn't a formal system until the 60s.

They just read the paper.

That supposes that there is an army of researchers dedicated to repeating expensive experiments. As far as I can tell this is doesn't exist. Especially in cases like this when it's obvious that the experimental protocol lacked controls and hence would cost more to repeat than the original study.

There are plenty of things that are not reproducible yet they make their way into text books and general practice.

I don't know if the claims in this article are correct but it is suggestive nonetheless:

"The analysis, which is published in the Journal of Clinical Epidemiology, included 154 Cochrane systematic reviews published between 2015 and 2019. Only 15 (9.9 percent) had high-quality evidence according to the gold-standard method for determining whether they provide high or low-quality evidence, called GRADE (grading of recommendations, assessment, development and evaluation).

Among these, only two had statistically significant results – meaning that the results were unlikely to have arisen due to random error – and were believed by the review authors to be useful in clinical practice.

Using the same system, 37 percent had moderate, 31 percent had low, and 22 percent had very low-quality evidence."

https://www.sciencealert.com/around-90-percent-of-your-medic...

> Not remember it. Not discuss it. Live it.

> No drugs. No surgeries. No medical interventions at all. [...] Just a shift in context. A different story to inhabit.

> These aren't just throwaway comments. They're programming.

> ...health isn't just physiological but narrative. [...] Not because we're hypochondriacs, but because expectation is itself an intervention that shapes our biology.

> Not to pretend, but to remember through their bodies what vitality felt like.

> The men at the retreat weren't cured of aging. They were temporarily freed from the cultural story that aging means inevitable, passive deterioration.

> She's not peddling the fantasy that positive thoughts can overcome all illness. She's not suggesting we can think our way to immortality or that serious medical conditions respond to attitude adjustments. [...] What she's demonstrating is more subtle and perhaps more powerful[...]

> This is about epistemic flexibility, not magical thinking. It's about loosening our grip on stories that may be limiting us unnecessarily, not abandoning medical care or scientific understanding.

> The most radical aspect of Langer's work isn't that belief affects biology, but that she treats this connection as a therapeutic tool rather than an inconvenient complication.

And that's just from the first two sections!

The song was published in 1959. Enjoy the article.