Hacker News Reader

Comment by Onavo | Hacker News Reader

Interesting, in my experience modern ML runs almost entirely on pragmatic Bayes. You find your ELBO, you choose the latest latent variable du jour that best models your problem domain (these days it's all transformers), and then you start running experiments.

View on HN

I think each category of Bayesian described in the article generally falls under Breiman's [0] "data modeling" culture, while ML practitioners, even when using Bayesian methods, almost invariably fall under the "algorithmic modeling" culture. In particular, the article's definition of pragmatic Bayes says that "the model should be consistent with knowledge about the underlying scientific problem and the data collection process," which I don't consider the norm in ML at all.

I do think ML practitioners in general align with the "iteration" category in my characterization, though you could joke that that miscategorizes people who just use (boosted trees|transformers) for everything.

[0] https://projecteuclid.org/journals/statistical-science/volum...

> the model should be consistent with knowledge about the problem [...] which I don't consider the norm in ML at all.

I don't think that is so niche. Murphy's vol II, a mainstream book, starts with this quote:

"Intelligence is not just about pattern recognition and function approximation. It’s about modeling the world." — Josh Tenenbaum, NeurIPS 2021.

Goodman & Tenenbaum have written e.g. https://probmods.org, which is very much about modeling data-generating processes.

The same can be said about large parts of Murphy's book, Lee & Wagenmakers or Lunn et al. (the BUGS book).

Archive for Goodman & Tenenbaum, since their site is flaky:

https://archive.ph/WKLyM

Reply on news.ycombinator.com