Comment by lliiffee | Hacker News Reader

Comment by lliiffee | Hacker News Reader

Parent Full thread

Roughly (and inaccurately) speaking, frequentism is about confidence intervals.

Like, suppose someone hands me a biased coin. A frequentist can flip that coin 10 times, then make a bound on the true bias of the coin that holds with the promised degree of confidence. (E.g. they can make a bound that will hold on 95% of such experiments.)

Bayesians can't do this. There are lots of other advantages to the Bayesian approach, but to frequentists, nothing could be worth giving up "coverage" (confidence intervals obeying their guarantees).

P.S. When looking at those ten coin flips, and trying to estimate the true bias, the frequentist and bayesian will have the following argument:

Bayesian: The probability that the true bias is b is __some formula___

Frequentist: Are you insane? The true bias of the coin is what it is! It's a number! Just because you don't know what that number is doesn't mean anything probabilistic is going on.

Bayesian: Well it is extremely useful to be allowed to make such statements.

A Bayesian can compute that confidence bound easily enough - it's plain old probability, after all. The thing is - the very moment you try to get from that confidence bound, to saying anything about the real value of the coin in a particular case with a particular result - you are performing a Bayesian update whether you like it or not. A Bayesian is aware of this. A frequentist clutches their confidence bounds and goes on trying to say something about the coin without admitting that this is a Bayesian step... or something.

As the saying goes, there are a million lies but only one reality. I know what it means to be a Bayesian; it means you acknowledge the sovereignty of probability theory as the law governing uncertainty. I'm not sure that even frequentists know what the rules are for being a "frequentist".

> I'm not sure that even frequentists know what the rules are for being a "frequentist".

I take no particular side on this debate, but that seems very unfair. The point of frequentism is this: Suppose we want to determine, say, the constant G of gravity. Suppose that 1000 groups around the world perform experiments to measure G, and then use frequentist statistics to compute 95% confidence intervals for G. ~950 of the groups will compute intervals that contain the true constant.

There are many advantages to bayesian methods, but none of them give you coverage.

Your point about not being able to say anything about the true value is technically true. The reason is that frequentists believe that one should avoid making such statements.

this is interesting - couldn't a bayesian get many of the same guarantees by computing the posterior distribution and choosing a range that contains X% of the mass?

Frequentist as described in the grandparent post do all the analysis they can without having to assume priors.

In a word, no. The reason is that if the bayesian does a really crappy job of specifying their prior distribution, the posterior will be completely inaccurate. On the other hand, if the bayesian does a good job, getting intervals as you describe will work much better (e.g. be smaller) than confidence intervals.

Reply on news.ycombinator.com