Hacker News Reader

Comment by twanvl | Hacker News Reader

Bayesian and frequentist approaches ultimately have a different notion of probability.

In the frequentist approach, a probability of 10% means that if you repeat an experiment many times, roughly 1 out of 10 times you will observe an event.

In Baysian statistics, a probability of 10% means that you are that certain about the event happening. So you would be willing to bet at 10 to 1 odds on the event happening. There doesn't have to be any repetition of the experiment for the probability to make sense. And as you can hopefully see, there is always a prior, that is, your belief about the event before doing any experiments.

View on HN

Another way to look at it is this way:

P(H|D) = P(D|H) P(H) / P(D)

Bayesians are interested in the probablity of various hypotheses h in H given some data D.

Frequentists calculate the probability of some data given a hypothesis (p-value is not strictly a probability but it can be one - it is ALWAYS a measure of extremity of data coming from the assumed hypothesis, which can be considered a relative probability).

Most interesting to me is that the Bayesian formula includes P(D|H) which is basically what the frequentists are calculating. In this sense, the question Bayesians answer is far closer to what we want to ask and far more powerful. In practice, the frequentist approach is often more than enough, though. The tradeoff is computability and simplicity.

Interesting, I've never thought about the likelihood as a confidence, but it makes sense. But sometimes the confidence also seems to reflect the opposite of extremity (e.g. for the null hypothesis).

But it seems to me one can still define this in terms of frequencies using a more general definition of what is meant by repeating an experiment. For example, when you use a probability P as a degree of belief or certainty about whether a patient X with symptoms W has a particular disease Y, one would define the universe as containing all possible realities in which X has the same symptoms W but with different underlying causal factors that lead to the same symptoms. An observation is a uniform sample from this universe and the belief is that a fraction of P of these realities has the cause Y.

Reply on news.ycombinator.com