Comment by probdist | Hacker News Reader

Comment by probdist | Hacker News Reader

Stan is best viewed in my mind as a successor to BUGS (Bayesian Inference Using Gibbs Sampling) which more people may have heard of. In fact, there are lots of players in the probabilistic programming space now, personally I like the model of "Infer.NET" [1] from Microsoft Research, as I find variational and approximate variational inference a good solution to my problems and I like coding models in nearly C# more than in Stan format.

If you want to get started as fast as possible with computational Bayesian inference and don't need the performance and advanced features Stan, I'd recommend emcee [2] which is a lightweight python MCMC sampler.

Final recommendation to the HN crowd: Bayesian Data Analysis 3rd Edition [3] should be on your desk if you are thinking about using any of this software.

[1] http://research.microsoft.com/en-us/um/cambridge/projects/in...]

[2] http://dan.iel.fm/emcee/current/

[3] https://www.crcpress.com/Bayesian-Data-Analysis-Third-Editio...

Hi, stan dev here. I think viewing Stan as a better BUGS is helpful but limiting.

The syntax is similar, but the class of models Stan fits is far more general. The class of algorithms we have available also goes beyond MCMC, e.g., variational inference, optimization, and interfaces to Stan exist on all primary programming languages. It's more helpful to think of Stan as its own probabilistic programming language, and arguably the biggest entity with the largest user base.

>the class of models Stan fits is far more general.

This is mostly true, but last time I checked, Stan still couldn't sample discrete variables like BUGS can. Stan can only fit models with discrete parameters (e.g. finite mixtures) if the programmer is smart enough to integrate them out.

some useful books available as a PDF:

David Barber "Bayesian Reasoning and Machine Learning" http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=... -- Thorough and systematic, accompanied by a Matlab library

Marc Steyvers "Computational Statistics with Matlab" http://psiexp.ss.uci.edu/research/teachingP205C/205C.pdf -- Somewhat unfinished but very good hands-on guide for a total novice

David Draper "Bayesian Modeling, Inference and Prediction" https://users.soe.ucsc.edu/~draper/draper-BMIP-dec2005.pdf -- Quite good if you're already familiar with a field, might be slightly daunting for a beginner

and the classic, David MacKay's "Information Theory, Inference, and Learning Algorithms" http://www.inference.phy.cam.ac.uk/itprnn/book.html

Another endorsement for Gelman's books — they're some of the best instructional statistics work that I've read.

You really think they're instructional? I would say it's sort of complete, but no means introductory.

Stan is very much a successor to BUGS, although I'm not sure BUGS is better known outside certain social science research communities these days.

It's not.

+1 on all of this (except I'm a fan of full MCMC, have you used ADVI?)

Haven't tried ADVI yet, but I think it is (likely) a great idea. In general automatic differentiation is a wildly powerful technique and is underutilized in computational science. Even in pure optimization problems people rarely leverage it to get J or H information as often as they probably should. I'll admit to not being super familiar with all the machinery under the hood in the ADVI paper [1]. It wasn't out when I started working on my current project.

[1] http://arxiv.org/abs/1506.03431

do let me know how it goes if you do try ADVI.

we're also working on making it more robust to initialization and step-sizes. stay tuned.

AVDI is incredibly awesome. Thanks so much for this work. Really nice readable paper on it too.

As someone working with large datasets I think automating variational inference + SGD to work with a broad class of models is really the way forwards and a big force multiplier for machine learning.

Reply on news.ycombinator.com