Hacker News Reader

Comment by njohnson41 | Hacker News Reader

Good article.

I'm only a bit disappointed that the author seems not to realize that Bayes' theorem is just a simple consequence of probability theory, and should be attractive not because "maybe the brain is Bayesian", but because it is based on sound set-theoretic and analytic principles. If Bayes' theorem is false, so is probability theory, and so is nearly everything we know about probability.

Edit: Here is a good explanation of the theorem that makes it visually clear how only set theory is involved in deriving it: https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/

View on HN

Just because a theorem is true doesn't mean you can't misuse it or that you don't need to do some work to map it to reality.

For example, the Banach-Tarski theorem is solid, but that doesn't mean you can start a business making golf balls by buying one and then endlessly replicating it.

Certainly. Just because you can name a theorem doesn't mean you can derive it either. The article had no actual computation or derivation of the theorem. Instead, it talked about beliefs and other things that don't really exist (in regards to the computation of a probability value).

I guess a hard-line frequentist (if such a person exists) would counter that you can't assign probabilities to hypotheses or fixed parameters. Then Bayes's theorem (and every other statement about probability) is true only when applied to statements about how often a certain event will occur.

But of course, most people do assign probabilities to hypotheses and fixed parameters, even if only informally. Bayesian probability theory is an attempt to formalize that kind of intuitive reasoning.

I think it's strange this sudden comeback of a theory that was dismissed more than 70 years ago by Fisher and many others, but no one, as far as I know, cares to explain why Fisher was wrong and why the theory is right. It makes me very suspicious, to be honest.

Actually, many Bayesian textbooks cover that. Try Jaynes - Probability Theory: The Logic of Science.

>I guess a hard-line frequentist (if such a person exists) would counter that you can't assign probabilities to hypotheses or fixed parameters. Then Bayes's theorem (and every other statement about probability) is true only when applied to statements about how often a certain event will occur.

If you model "thinking" and "believing" as sampling in probabilistic programs (which they do in some schools of cognitive science), then Bayes' Theorem becomes a theorem about how often certain execution traces occur when the sampling program is run with fresh randomness. You then need none of the weird metaphysics associated with "subjective Bayesianism".

I have heard of people genuinely saying such a thing. Fortunately, it is nothing but an empty redefinition of the word "probability". In fact, rational degrees of belief in hypotheses do follow the kolmogorov axioms (as shown by eg. Cox's theorem or the VNM theorem), and bayes theorem does therefore apply. Whether or not someone refuses to call that "probability" makes no difference.

All probabilistic tools are accurate in theory, otherwise we wouldn't use them. Bayes' theorem is no different from e.g. the t-test in that regard. The question of whether it's worth using Bayes explicitly rather than other tools is and should be a question of whether we find it aligns with our understanding and helps us think more clearly.

Given that

  p(A and B) = p(B and A)
  p(A and B ) = p(A) * p(B|A)

we have:

  p(A) * p(B|A) = p(B) * p(A|B)
  p(B|A) = p(B) * p(A|B) / p(A)

Reply on news.ycombinator.com