Hacker News Reader

Hacker News

Comment by SpaceManNabs | Hacker News Reader

Parent Full thread

SpaceManNabs

> I've never heard anyone promote frequentist statistics

I will do it:

1. Computationally easier

2. often analytical theory available for most use cases so interpretability is high

3. more literature available so you can get unstuck faster if you mess up

4. no accusations of subjective bias in your prior (the con is clear, no ability to leverage subjective expertise)

5. In the asymptotic regime, MLE and bayesian MAP often converge anyways

6. king of hypothesis testing

For most people, it doesn't matter. It matters when you are doing treatment for small sample sizes or other situations that would cause low power.

View on HN

> king of hypothesis testing

This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to show that some effect happens in some direction. For this (generally acceptable) purpose, frequentist stats are great.

A statistically significant result without something like a Hedges G (or Cohens D if comparing means) measure would be called out by even a moderately attentive reviewer as a matter of course (especially if small sample sizes are involved), and many reputable journals actually require effect sizes to be stated, so you run the risk of being desk rejected for leaving it out.

For leaving it out perhaps, but the result is publishable even if the effect is small... where for significant you usually must p-hack your way to a standard threshold.

It's not entirely unreasonable, as often extra variables suppress the effect size. E.g. the effect is substantial in people with some genetics, insignificant otherwise. The fact that there is an effect at all makes it interesting as a starting point for further exploration e.g. to determine the mechanism or find the extra factors needed to make the effect significant.

> This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to show that some effect happens in some direction

That's not an advantage, that's one of the primary reasons psychology is among the worst subjects hit by the replication crisis.

Sorry for this very slow response, but I actually disagree. I don’t think effect sizes are usually meaningful beyond the context of a specific experiment.

For example, suppose you are researching the effect of an emotional/negative picture on how a participant makes decision in an economic game. Here, you may think “a big effect size implies this study will have a large practical relevance.” However, following typical statistical designs, an effect size may be largely determined by just how many trials the experiment used for each participant. This is also not the only factor that’s relevant. For instance, it’s debatable how the emotion induced in the task compares to the intensity of real life emotional situations. With these factors in mind, the effect size really doesn’t tell you much that can be applied outside the laboratory.

Maybe in some non-experimental sub-fields (e.g. personality psychology) effect sizes are more meaningful…

A cheap shot that misunderstands the so called "crisis" in psychology, which almost comes entirely from the fact that brains are non randomly exchangeable, unique, samples differ, the brain is complex, multiple ways to solve problems, and sussing it all out is complex as hell.

If they are at least as knowledgeable as you, who correctly describes how messy you can expect the data to be, then you should be more concerned with effect size, because small effects are almost certainly just "noise". Instead, sensationalism about miniscule effects that vanish on replication attempts abound.

So not a cheap shot at all IMO.

Add some sampling bias, a significance threshold of 0.05. Baby, you’ve got a replication crisis going!

Reply on news.ycombinator.com