Hakaru: An embedded probabilistic programming language in Haskell | Hacker News Reader

Hakaru: An embedded probabilistic programming language in Haskell

indiana.edu

89 pointswcbeard1019 comments

The last example in the more detailed notebook [0] linked from that page is also pretty cool, especially with the graphs. If you're familiar with Haskell or other similar languages, the code is fairly easy to read.

[0] http://indiana.edu/~ppaml/HakaruTutorial.html

I'm not familiar with Haskell, but can someone provide some insight into how this compares to PyMC or OpenBUGS?

One of the authors here. In terms of expressiveness, Hakaru is much more in the mold of something like Church, Venture, or Anglican where we can more easily express nonparametric models. This means in the context of the model, once a random variable is generated from a prior distribution, it might be used as an ordinary Haskell value and be directly used to in the control-flow. When that happens you can write models where the number of parameters changes based on values of parameters generated earlier in the model. As a concrete example, imagine a Bayesian Neural Network where there was a prior on how many layers are in the neural network.

This is in contrast to something like PyMC where random variables are very much special objects that interacted with, and making nonparametric models is trickier.

Like PyMC, Figaro, Infer.NET and Factorie but unlike OpenBUGS this is a library so it is easier to use a larger prediction application. We don't support some of the optimizations that OpenBUGS can do when presented with a model.

As another note, this is a fairly new piece of code. Unlike PyMC, BLOG, Stan, and other probabilistic programming systems we only support Importance Sampling and a fairly naive version of Metropolis Hastings. This is in contrast to many of the existing solutions where you can do things like Particle MCMC and Hamiltonian Monte Carlo. We do intend to eventually support these inference techniques.

Do you have a non-monadic, Applicative subsystem, too?

I love how it's titled "Simple Example". I don't think I'm ready to delve into Haskell, much less an abstraction embeeded within it. Maybe Clojure for another 6 months and then we'll see.

As someone who can Haskell, I can reassure you that the example is indeed simple. But I understand your concern.

It is indeed simple, though the use of ``Data.Dynamic`` raises a few eyebrows.

Most of the complexity of needing Data.Dynamic has to do with the way we store the heterogeneous collection that is our observed data. There is a nicer way to do this which I hope lands in a future release.

A while ago I tried to port some of the Python code from the book Programming Collective Intelligence to Haskell, and I got stuck on the seemingly simple problem of reading and processing heterogenous collections (strings and numbers).

What is the nicer way to do it?

So I don't really consider myself a Haskell programmer, but the standard advice is to think about operation you are going to perform on those strings and numbers later, then think if there is a datatype you could create instead.

As an example, suppose I am trying to load a csv file with two fields being strings and one being Double. A list of lists representation isn't going to cut it. Instead I should make a Row datatype

  data Row = Row {field1 :: String, field2 :: String, field3 :: Double}

Then you can parse the csv file into a [Row] representation.

As an aside, if the task did involve parsing csv I suggest the cassava library which I found out about through the amazing What I Wish I Knew When Learning Haskell (http://dev.stephendiehl.com/hask/)

Use a sum type:

   data Datum = S String | D Double
   
   lst :: [Datum]
   lst = [S "a", D 3.14]

In Python we'd have to write the logic to handle both cases and fail with an exception or do an instanceof check. In Haskell we'd do this by pattern matching on the sum type.

   case (lst !! 42) of
      S n -> -- handle string
      D n -> -- handle number

I recommend "Learn You a Haskell". Free online.

It really is a very simple language. The ML languages (and Haskell in particular) are just rather different and initially hard to read.

Out of curiosity, which part of the example did you find difficult?

Haskell bears very little resemblance to traditional pseudo-code, compared to something like JavaScript.

True. Traditional pseudo-code tends to be imperative.

I think you'll face a similar problem with Clojure :)

Since the example used Monads, it was rather imperative.

Monads aren't really imperative, though the syntax sugar sometimes looks like it, but in any case that's too fine a point. You know what I meant. Pseudocode usually looks like this:

  x := 1
  while (x < 100)
    stuff(x)
    x := x + 1

etc, etc. Typical Haskell doesn't look like this. The example didn't look like this. Clojure won't look like this, either (in fact, it's likely to look even weirder than the Haskell example). That's neither good nor bad; I don't think programming languages should be judged by how much they resemble C/Pascal/Python/pseudocode.

this was written for getting an intuition about the syntax:

http://blog.ezyang.com/2011/11/how-to-read-haskell/

The things that make haskell and ocaml outwardly different are the pattern match syntax for function definitions, type signatures, the IO monad's slightly different syntax, all the category theory structures and their punctuation filled names, "<*>", and, you know, all that "theorems for free" and Curry-Howard stuff. But to get started, there's LYAH and (3)good tutorial books by Alejandro Mena, Simon Thompson and Graham Hutton, you can type expressions in to GHCi (the REPL) and see what types GHC infers, and you can, uh, encounter GHC error messages and warnings for the firs ttime.

Reading Haskell is not that hard, but working with Clojure won't help you with it, unfortunately. Try OCaml or F# (or maybe Scala) for a more beginner friendly introduction to most of Haskell concepts.

Reply on news.ycombinator.com