Hacker News Reader

Comment by cultus | Hacker News Reader

There's no definition of probability that doesn't involve philosophy or metaphysics [0]. Calling frequentist stats "objective" really bugs me. There is no such thing. Every inference procedure involves subjective choices.

Of course, frequentist and Bayesian stats are completely mathematically equivalent. The choice just affects our mental patterns.

[0] https://en.wikipedia.org/wiki/Probability_interpretations#Ph...

View on HN

No, frequentist and Bayesian statistics are not equivalent.

There are some special cases in which a frequentist 95% confidence interval and a Bayesian 95% credible interval based on some sort of default prior are numerically the same, but that doesn't happen in general.

Statisticians would hardly have been vigorously debating the issue for two centuries if it didn't really matter.

From the article:

>>>> Let’s now suppose that we’ve done a Bayesian analysis. We’ve specified a prior distribution for the parameter, based on prior evidence, our subjective beliefs about the value of the parameter, or perhaps we used a default ‘non-informative’ prior built into our software package.

At first blush the difference is that the Bayesian is using more information. Now don't get me wrong, if Bayes theorem and its progeny give us useful tools for incorporating that information in our analyses, so much the better.

It's been only one century since Ronald Fisher misunderstood statistics and thereby created what's now known as 'frequentist statistics'.

Every frequentist technique has a Bayesian interpretation and vice-versa. Confidence intervals are equivalent to credible intervals with certain priors.

The result of Bayesian inference is a whole posterior distribution, not just a confidence interval. Any attempt to produce a frequentist version of a posterior distribution is either going to end up just being Bayesian inference in disguise, or be inconsistent.

But even if we focus just on confidence intervals and credible intervals, there needn't be the equivalence you state. A comment elsewhere here discusses a ridiculous confidence interval that is either the whole parameter space or the empty set. That's never going to be what a Bayesian credible interval gives you.

I don't think you're engaging in the specific sense of objective (, subjective) meant here.

A frequentist interpretation of probability is objective in the sense that it grounds the probability value in objective features of the world.

A bayesian is subjective in that the probability valuation of an event is grounded in the belief-state of its observer.

My point is you that you cannot ever actually claim to ground a probability value in objective features of the world. Many events aren't repeated, for example.

Sure you can, frequentists do not ground probability in repetition. Repetition is just a measurement process.

Their grounding is some real physical parameter, eg., with the coin the geometry of it having two sides.

I think a further problem may be that probability could take on different meanings under different applications. I admit that I may be influenced by how I learned statistics -- in a math class that was primarily focused on proofs and not applications. But I've formed the view that math is math, we choose a math technique that works for the situation at hand, then we choose an interpretation that works for guiding and explaining what we're trying to do.

It's just a perspective, but Bayesian inference makes the prior explicit, wheareas it is implicit with frequentist inference. You can't not have a prior.

That's fair, it forces you to document your assumptions. But at some point as you drill down into assumptions, the trail will grow cold. The article even talks about inserting priors that are either subjective or some default function provided by the software.

A possible third approach is to create a hypothetical model and feed random data through it, to get an idea of the spread of the outcomes. Modeling doesn't stumble on conditional terms, and if you're unclear on an assumption, it won't run. The computer doesn't know whether it's a frequentist or a Bayesian, or something different from both.

I'm not a statistician. Whenever I need to do something with statistics, I always test my computation with random data.

In fact I wonder, if modeling had been possible since the birth of statistics, if we would even bother with things like elaborate formulas for statistical tests.

Bayesian stats actually let you integrate stochasticity into deterministic models fairly easily, in a way you can't really do with frequentist stats. Bayesian methods are common for geophysical inverse modelling, for example. Probabilistic programming does this, for example. Exact Bayesian inference is impossible in the general case, but approximate inference often works well.

Subjective priors are one of the main advantages of Bayesian stats. Regularization used in ML corresponds to using subjective priors, for example. L2 regularization finds the MAP with a normal prior, favoring parsimonious solutions.

Reply on news.ycombinator.com