Probabilistic Programming
cs.cornell.edu
cs.cornell.edu
If you like functional programming, probabilistic functional programming might also be interesting for you. I'd recommend reading Practical Probabilistic Programming with Monads [1], and checking out monad-bayes [2] which builds on the ideas discussed in the paper.
[0]: https://github.com/hakaru-dev/hakaru
Where in this picture does formulating the problem in a Turing-complete language help anyone? If making the "compiler"/analyser is (for practical/interesting purposes) impossible anyway?
I have spent several years in my PhD trying to map out just one probability distribution and make it computationally tractable. Postulating the existence of a "magic" analyser that work is redundant, which would be nice, but I don't see it ever happening.
Granted it has a billion input variables, but so does any "real-world", non-toy computer program.
TL;DR if the problem fits stabdard MCMC or rejection sampling just do it. If not then a magic performant analyser is going to be hard to build. I hope I am wrong :)
There's some chance we'll be able to find clever new ways to derive better inference algorithms in specific situations using program analysis, but I wouldn't hold my breath. And it's impossible, of course, for anything like that to work for every possible program in a Turing-complete language.
Uh oh. That's dangerous talk.
The problem is that once you start writing the kind of program that Prolog is best at, in Prolog, then you really, _really_ don't want to go back and write the same thing in any other language. Because most of the time it's a huge pain, and you'll probably have to implement Prolog in that other language anyway (and it's going to be a slow and buggy Prolog, but without all the good bits).
This has actually harmed the adoption of Prolog in a roundabout way, I think. Programmers are (or, well, were ... in the distant past) worried they might be stuck with it and unable to write the everyday stuff they need (REST APIs and whatnot).
If PP languages are sold in the same way, as very niche products that are only useful for a specific kind of programming, then I don't think they are ever going to be widely adopted, not even from the people who could benefit from them. Which is to say: eventually, everyone.
Instead of risking being stuck with very valuable software written in a one-trick pony sort of language, much better to take the long road and write it all in a language like C# or Java, where it may look attrocious, but at least it's the same language as the rest of your application (and you can find plenty of warm bodies to throw at it once it starts going south, to boot).
As a for instance, grammars (in Prolog, a grammar can be expressed as a first order logic theory, to be proved by the compiler; therefore, a program).
you implement your own universal probabilistic language embedded in js.
Since it's on subject here - anybody know of any good clojure libraries or resources for probabilistic programming? Anglican definitely looks good, but their intro pages use some kinda strange workflows.
[1] http://mc-stan.org/ [2] https://www.youtube.com/watch?v=uSjsJg8fcwY
Do remember that Anglican is a partial language in clojure, so you can't use all clojure functions (and I think none of the macros) within anglican's defqueries.
We taught a summer school on Anglican last August. The materials are available online:
https://bitbucket.org/probprog/ppaml-summer-school-2016 http://www.ccs.neu.edu/home/jwvdm/talks/
(Disclaimer: I'm a sysadmin and hobbyist programmer, so my workflow might be a bit different from most.)
The strangeness I saw was mostly in from difficult to follow documentation and the use of gorilla, not necessarily Anglican itseld. As awesome as gorilla is, it's just another new thing to learn. Since I'm already trying to learn Anglican, learning Gorilla and fighting my muscle memory just adds noise to the learning process. It would be nice to have something similar straight within a REPL (similar to how incanter does it with `view`). It's not a problem exclusive to Anglican, though.
That said, the barrier for entry with a lot of Clojure - even the language itself (no, not just its parens) is like that, where frustration is by a thousand cuts.
(After I posted that comment I dug a bit deeper into Anglican and found that I really enjoy it :).)
The idea is that if you can write the program, you can sample from the generative model. Most libraries like Stan don't really provide a "Turing-complete" probabilistic language.
Daniel Roy, IIRC, studies precisely why or when you want this kind of "completeness", as well as other topics in PPL theory.
Outside of that sphere there is even more: you can make any program probabilistic / learning with something like PyMC that embed PP in a regular environment. Think bots, agents, UIs ...
Bangs head into desk repeatedly :-(
(Yes, I know that security wasn't the main point of the post, but still...)