Comment by dagss | Hacker News Reader

Comment by dagss | Hacker News Reader

Parent Full thread

My feeling is everything you could do this way is either a) trivial anyway, or b) is making use of all the active research on MCMC sampling (which generally gives you DSLs to express a subset of all possible programs, which they are optimized for).

Where in this picture does formulating the problem in a Turing-complete language help anyone? If making the "compiler"/analyser is (for practical/interesting purposes) impossible anyway?

I have spent several years in my PhD trying to map out just one probability distribution and make it computationally tractable. Postulating the existence of a "magic" analyser that work is redundant, which would be nice, but I don't see it ever happening.

Granted it has a billion input variables, but so does any "real-world", non-toy computer program.

TL;DR if the problem fits stabdard MCMC or rejection sampling just do it. If not then a magic performant analyser is going to be hard to build. I hope I am wrong :)

(I'm the author of the lecture notes.) I totally agree, and this is an underappreciated view in the PP community. The hype says "you just write your model; you'll get inference for free," but that obscures that current languages do nothing to help with cooking up an efficient inference algorithm for a particular situation. That's still a hard problem. When the only inference algorithm anyone actually uses is some MCMC variant, I think that's a broken promise.

There's some chance we'll be able to find clever new ways to derive better inference algorithms in specific situations using program analysis, but I wouldn't hold my breath. And it's impossible, of course, for anything like that to work for every possible program in a Turing-complete language.

Thanks for confirming my suspicion. So would it be fair to say that PP is an interface to statistics, so that those who know how to code but not advanced stats can do stats without learning it? (On simple problems)

As it stands, I'd agree with that---but the characterization definitely depends on who you talk to. I might be in the minority among "PP people," since the hype is still going strong and some smart people seem to see more potential than I (and you) do.

Reply on news.ycombinator.com