Anglican: A Probabilistic Programming System | Hacker News Reader

Anglican: A Probabilistic Programming System

probprog.github.io

78 pointsespeed26 comments

I am the author of the most of current Anglican code base. Anglican's name indeed comes from the Church of England, Church being the name of an influential probabilistic programming language https://cocolab.stanford.edu/papers/GoodmanEtAl2008-Uncertai... . Frank Wood, the inventor of Anglican, moved to Oxford from the US and the name reflects the move.

Clojure was chosen because it was a modern lisp which was apparently easy to deploy (leveraging the JVM). That choice was given to me (I learned Clojure to join, but new Common Lisp well before). Original Anglican syntax was Scheme-like, for Church compatibility, but we switched to Clojure syntax later.

Anglican has been a platform for a number of research projects (and some practical applications). Now Anglican is in dire need of new maintainers or it will stagnate (I am not working on Anglican code base anymore, doing other things: http://infergo.org among others). One low-hanging fruit which unfortunately refuses to fall is automatic differentiation.

Anyone interested in learning probabilistic programming, this is a great book that has integrated code written in WebPPL.

https://probmods.org/

Another book from the same source:

http://dippl.org/

why did they pick Clojure to do probabilistic programming?

btw, Python programmers have the option of pymc3 https://docs.pymc.io/

Or Pyro https://github.com/uber/pyro

Or ProbTorch https://github.com/probtorch/probtorch

They are not the only ones picking Clojure to do probabilistic programming: https://github.com/probcomp/metaprob

Why not? I think lisps could use a visibility boost in the various data sciences. Also probabilistic programming tends to be highly declarative, which works well in a language family where "code is data" is a core philosophy.

At first glance, maybe Common Lisp would have made more sense. As far as I'm aware, the CFFI is pretty good, which means you can interact with other machine learning code that is largely written in C under the hood.

That said, there is a mature machine learning ecosystem for the JVM, not to mention the prospect of running massive probabilistic models on Yarn or Spark.

By the way, there are many probabilistic programming languages in addition to PyMC3. There are Stan, Edward, Pyro (fairly new), and even the venerable BUGS/JAGS.

One advantage of Clojure compared to Common Lisp is very well implemented parallelism and corresponding functional data structures. Something which is hard to find for Common Lisp implementations.

>why did they pick Clojure to do probabilistic programming?

It probably came down to a coin flip.

It seems that you can run it directly in the browser with clojurescript, which might help a lot with educating about the project. If memory serves, church did this.

Also there is some tradition on using lisps for AI; church, for example, is implemented on top of scheme.

Other probabilistic programming languages use prolog as ascendent and Prolog and lisps such as clojure are quite related.

But I don’t know if these were the reasons :)

One reason might be macros. Since it's a Lisp, it's very easy to implement custom syntax for the abstractions they want to expose, like defquery in their example. Something like that could only be loosely approximated in Python. Probabilistic programming, being a whole different programming model benefits from this more than most libraries would.

All of Anglican is macro-compiled into CPS-ed Clojure: https://bitbucket.org/probprog/anglican/src/master/src/angli...

I am very happy to see this. Clojure is being used in more and more applications and seems to be quite popular in larger-scale data analytics applications, where many other solutions struggle.

It is particularly strong when you want to build a data processing/analysis pipeline for moderate amounts of data (say, below a TB). With the fantastic built-in concurrency support, small external tools (core.async parallel pipelines and perhaps aphyr/tesser for parallel reducers) and the performance of the JVM, you can crunch through these amounts of data on a laptop, without involving anything bigger like hadoop. Saves time and makes for good consulting margins (I speak from experience).

It also happens to be the language which I use to develop fairly large applications, so I guess you could call me biased :-)

Wouldn't a language with support for continuations (Scheme or Racket, speaking of the Lisp family) be a better choice for the task and allow this to be implemented as an embedded DSL rather than a separate (even if it is tightly integrated with Clojure) language?

"Embedded DSL" vs "Separate" is a matter of use of words. Anglican is macro-compiled inside Clojure into Clojure. If you like to call it a DSL, call it a DSL.

An important design feature is that it is 'anti-DSL' --- the syntax is exactly the Clojure syntax. But the program is transformed into an extended CPS form and run through inference executor. There is an academic paper written by Anglican team explaining the design and internals:

http://www.cs.ox.ac.uk/people/hongseok.yang/paper/ifl16.pdf

My mistake, I misread the docs. Anglican can actually be considered to be an embedded DSL.

> An important design feature is that it is 'anti-DSL' --- the syntax is exactly the Clojure syntax.

This is not "anti-DSL", this is what gives Anglican embedded DSL qualities.

And yet there are limitations: «The border between Clojure and Anglican is subtle and usually will pose no problem to most programmers, however, some confusion can arise from the fact that Anglican programs are macro compiled into CPS-style Clojure functions. This means that some wrapping of “native” Clojure functions needs to happen in order to use them in Anglican. Errors arising due to misunderstanding this boundary crop up in the form of “wrong number of argument” exceptions».

Would that language border, wrapping of functions and transformation to CPS be still necessary in a language with native support for continuations?

Anglican is a CPS state monad. Clojure functions are automatically lifted into the monad, but to maintain fast execution of Anglican code, Clojure functions must be declared as such to Anglican.

A language with native support of continuations is a nice toy but does not bring practical benefits to implementing CPS state monad.

> why did they pick Clojure to do probabilistic programming?

As most language decisions for new projects, it probably came down to "it's their favorite language".

> btw, Python programmers have the option of pymc3 https://docs.pymc.io/

And Edward (http://edwardlib.org/), "Pyro" (http://pyro.ai/) and "ProbTorch" (https://github.com/probtorch/probtorch).

The first built on TensorFlow, the latter two on PyTorch.

Stan (http://mc-stan.org/) also has python bindings: https://pystan.readthedocs.io/en/latest/

And Pomegranate

https://github.com/jmschrei/pomegranate

There are also WebPPL[1] and ProbLog[2].

[1] http://webppl.org/

[2] https://dtai.cs.kuleuven.be/problog/

It took me a second to figure out why they chose their project name. Clever.

(edit: maybe not? The good reverend was not CofE)

I don't know what CofE is but always assumed that they named it Anglican in reponse to Church.

CofE is an abbreviation for the Church of England (parent church of the Anglican Communion).

Don't look now, but even Church was (like Bayes) a Presbyterian.

I expected a programming language that has Book of Common Prayer syntax.

Reply on news.ycombinator.com