Foundations of Probabilistic Programming

I am currently going through Statistical Rethinking [0], by R. McElreath. So far it's been great, all the code example and practice problems help comprehension.

I've also heard good things about Probabilistic Programming and Bayesian methods [1] by Cam Davidson Pilon.

[0] https://xcelab.net/rm/

[1] https://camdavidsonpilon.github.io/Probabilistic-Programming...

I started [0], but I got sidetracked. If anyone else has used this let me know as well.

[0] https://arxiv.org/abs/1809.10756

Gelman's (one of the authors behind Stan) Bayesian Data Analysis is considered to be one of the best resources on this subject, so I suggest giving that a read. It's not the easiest book to work through though.

Why do you say that, if I may? I looked at the index and preface and it did not give me that impression at all - the only thing that surprised me is that I was reading “foundations” as “introduction”, and at least part of the book is focused more on “foundational theory”/“underpinnings” (e.g. measure theory aspects, or how you can construct a lambda calculus that should demonstrate that general probabilistic programming ( as in languages such as anglican[1]) can be achieved.

[1] I only have hands-on experience with anglican and PyMC3. They differ in approach: in anglican you write anglican code (not clojure) that is parsed as a model; whereas in PyMC, you write python code that constructs the PyMC model, by calling PyMC library functions. To me, the former is closer to the idea of probabilistic programming, being Turing-complete and all, but perhaps this is not a productive segregation.