Hacker News Reader

Comment by jpmattia | Hacker News Reader

Psych study with n=114. Given the reproducibility crisis in psychology, I'll wait for the replication.

Sure, do that. But also:

> The power analysis used a power criterion of (1 − β) = 0.80, which indicated that a mean cell size of n = 54 was sufficient to replicate significant effects of acetaminophen.

The question is not whether n=114 is "large enough" because that question, by itself, does not make sense. The question is whether n=114 is "large enough for the observed effect." Given that the p-values for the "personal pleasure" and "empathetic feelings" measures are under 0.001, the answer is "yes," potential methodological issues notwithstanding.

Disagree from experience. I find it far more likely that they considered how to skew the p-values low enough to where they become significant. “Personal pleasure” & “Empathetic feelings” do NOT sound like terms which correlate to reproducibility to me.

> “Personal pleasure” & “Empathetic feelings” do NOT sound like terms which correlate to reproducibility to me.

All studies that use terms like that have a section that explains how they are measuring it, how other studies have used the same measure and how that measure itself is tested for reproducibility and accuracy.

doesn't necessarily mean they are ACTUALLY measuring anything real. just sayin'

So without training in the field, having read about the specific example or anything else, you can say the entire study is trash?

Pretty sure caution was what was advised with no opinion one way or the other. Being extra cautious about preliminary psych findings is only sensible. It might be considered batshit crazy to do anything else given the issues we've found out about in that field. Does it reproduce? Is the right question to ask.

I'd think of it more like such studies being potential stepping stones towards the establishment of scientific facts, rather than establishing those facts alone. But it will take more work and by separate groups before anyone can have faith that these findings are meaningful - especially given the background of the reproducibility crisis.

I don't think you should sneer at a triple-digit sample size that's justified with a power analysis based on their previous work.

I agree that you'd want to see replications before making any major decisions--these effects are often finnicky--but this seems to me like a reasonable-sized step forward.

They also had a prior justification for running this experiment based on known effects of Tylenol on the brain.

Seems like every time now, when I read a medical study, especially psychiatry-related, I'm shocked that it could even be published. I have no training in the field, so nobody would take me seriously and I even doubt myself, but the study designs just seem so poor

I tend to agree. While I think the sample size is fine, the way perceived positivity and perceived pleasure were measured seems pretty subjective in a way I find hard to generalize.

The paper does seem awfully suggestive, though. Worthy of more exploration for sure.

There's been a few over the years.

[1] 2016: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5015806/

[2] 2015: https://journals.sagepub.com/doi/abs/10.1177/095679761557036...

[3] 2012: http://www.psychologicalscience.org/news/releases/broken-hea...

All undergrads from a single university also. Just based on the social development of people as they go through undergrad, I'd guess that the variance in behavioural change before and after treatment is fairly high whether they're treated or not.

Just like a lot of pharma papers need a `... in mice` appended to their titles, a lot of psych papers need `... in undergrads` appended to theirs.

Scientist: Hey, that's odd ...

HN reader: I'll wait for the replication.

Reply on news.ycombinator.com