There is a total lack of understanding of how it works, but people think they know how to use it. There are numerous articles out there containing statements like "there were no differences in age between the groups (p > 0.05)". Consequently, it is the wrong thing to teach.
That's apart from the more philosophical question: what does it mean when I say that there's a 40% chance that it team A will beat team B in the match tomorrow?
Reducing frequentist statistics to testing and p-value is a huge mistake. I have always wondered if that’s how it is introduced to some and that’s why they don’t get the point of the frequentist approach.
Estimation theory makes a lot of sense - to me a lot more than pulling priors out of thin air. It’s also a lot of relatively advanced mathematics if you want to teach it well as defining random variables properly requires a fair bit of measure theory. I think the perceived gap comes from there. People have a somewhat hand wavy understanding of sampling and an overall poor grounding in theory and then think Bayes is better because it looks simpler at first.
You're "pulling priors out of thin air" whether you realize it or not; it's the only way that estimation makes sense mathematically. Frequentist statistics is broadly equivalent to Bayesian statistics with a flat prior distribution over the parameters, and what expectations correspond to a "flat" distribution ultimately depends on how the model is parameterized, which is in principle an arbitrary choice - something that's being "pulled out of thin air". Of course, Bayesian statistics also often involves assigning "uninformative" priors out of pure convenience, and frequentists can use "robust" statistical methods to exceptionally take prior information into account; so the difference is even lower than you might expect.
There's also a strong argument against NHST specifically that works from both a frequentist and a Bayesian perspective: NHST rejects the Likelihood principle https://en.wikipedia.org/wiki/Likelihood_principle hence one could even ask whether NHST is even "properly" frequentist.
No, you are not. That’s an argument I often seen put forward by people who want the Bayesian approach to be the one true approach. There are no prior whatsoever involved in a frequentist analysis.
People who say that generally refer to MLE being somewhat equivalent to MAP estimation with a uniform prior in the region. That’s true but that’s the usual mistake I’m complaining about of reducing estimators to MLE.
The assertion in itself doesn’t make sense.
> Of course, Bayesian statistics also often involves assigning "uninformative" priors out of pure convenience
That’s very hand wavy. The issue is that priors have a significant impact on posteriors, one which is often deeply misunderstood by casual statisticians.
Is that not equivalent to a prior that the coefficient on variables in Z but not in X is zero?
It may not be everywhere, but even in the simplest case of NHST, there certainly is. It assumes no difference between H0 and H1. And NHST is basically the topic of this entire thread: it's what we should have stopped teaching a long time ago.
NHST is not associated with any statistician, and you will find no author claiming to be its inventor. It is a misunderstanding of statistics apparently originating from psychology back in the 1960s, or at least that's as far back as I've found it.
I wonder if the generality of the Bayesian approach is what's prevented its wide adoption? Having a prescribed algorithm ready to plug in data is mighty convenient! Frequentism lowered the barrier and let anyone run stats, but more isn't necessarily a good thing.
1. They aren’t answering the original question. The question is about the probability of a property of Saturn. Not about the process of repeatedly forming thousands of alternative Saturns. This seems like a subtle difference but that’s only because Frequentism has been the default for so long. It doesn’t attempt to answer the questions people are actually asking.
2. The assumptions it makes to answer that alternative question are just as flawed. We can’t go back in time and change the conditions surrounding Saturn’s creation. We can’t run 1000s of repeated trials of the creation of Saturn. For a group of people so ideologically opposed to a statement as simple as “the probability of this flipped coin being heads is 50%”, it seems absurd that they are fine with their entire framework being built around a premise that doesn’t exist and cannot exist.
Yes, I think this is kind of standard practice in many fields.
If someone questions this just quote "all models are wrong, but some are useful" as if the quote is actually saying "all models are wrong, but all models are useful".
I fear you operate under the illusion that frequentist statistics are somehow limited to hypothesis testing. It is absolutely not the case.
That isn't much of an argument to the mathematicians. Nobody ever came up with a compelling explanation for what -1 sheeps look like and yet negative numbers turned out to be extremely practical. If it is absurd and provably works then the math community can roll with that.
Math people prefer to generalize. But with frequentism, it is not possible to generalize because "frequency" is baked into its very name. Indeed you can imagine Bayesian statistics as the generalization of frequentism.
Bayesian methods are more intuitive, and fit how most be reason when they reason probabilistically. Unfortunately Bayesian computational methods are often less practical to use in non-trivial settings (usually involves some MCMC).
I'm a Bayesian reasoner, but happily use frequentist computation methods (max likelihood estimation) because they're just more tractable.
Maximum likelihood also tends to be equivalent to MAP with uninformative priors.
I find a lot of Bayesian analysis is a bit of cargo culting and frequentist/ML formulations are dismissed with tribalism.