Hacker News Reader

Comment by wespiser_2018 | Hacker News Reader

wespiser_2018

The difficulty of teaching statistics is that the maths you need to prove things are right and gain an intuitive understanding of the methods are far more advanced than what is presented in a basic stats course. Gosset came up with the t-test and proved to the world it made sense, yet we teach students to apply it in a black box way without a fundamental understanding of why it's right. That's not great pedagogy.

IMO, this is where Bayesian Statistics is far superior. There's a Curry-Howard isomorphism to logic which runs extremely deep, and it's possible to introduce using conjugate distributions with nice closed form analytical solutions. Anything more complex, well, that's what computers are for, and there are great ways (STAN) to run complex distributions that are far more intricate than frequentist methods.

View on HN

> There's a Curry-Howard isomorphism [between] logic [and Bayesian statistical inference].

This is an odd way of putting it. I think it's better to say that, given some mostly uncontroversial assumptions, if one is willing to assign real number degrees of belief to uncertain claims, then Bayesian statistical inference is the only way of reasoning about those claims that's compatible with classical propositional logic.

The will to assign real numbers to degrees of belief is the controversial assumption. Converted bayesians tend to gloss over this fact. Many, as in a sibling comment, state that MLE is bayesian statistics with a uniform prior, but this isn't true of most if not all frequentist inference, based on frequentist NHT and CI, not MAP. Modeling uncertainty with uniform priors (or even more sophisticated non-informative priors a la Jaynes) is a recipe for paradoxes and there is no alternative practical proposal that I know of. I have no issue with bayesian modeling in a ML context of model selection and validation based on resampling methods, but IMO it's not up to the foundational claims its proponents often do.

Maximum likelihood (which underpins many frequentist methods) basically amounts to Bayesian statistics with a uniform prior on your parameters. And the "shape" of your prior actually depends on the chosen parametrization, so in principle you can account for non-flat priors as well.

IMHO, the discussion should not be so much whether to teach Bayesian or maximum likelihood. But instead, whether to teach generative models or to keep going with hypothesis tests, which are generally presented to students as a bag of tricks.

Generative models, (implemented in e.g. Stan, PyMC, Pyro, Turing, etc.) split models from inference. So one can switch from maximum likelihood to variational inference or MCMC quite easily.

Generative models, beginning from regression, make a lot more sense to students and yield much more robust inference. Most people I know who publish research articles on a frequent basis do not know p-values are not a measure of effect sizes. This demonstrates current education has failed.

Maximum Likelihood corresponds to Bayesian statistics with MAP estimation, which is not the typical way to use the posterior.

Reply on news.ycombinator.com