One thing that's not entirely clear from the article is that probabilistic programming is Bayesian modeling and inference. Probabilistic programming isn't a new machine learning technique it's a more flexible way to do Bayesian statistics.

The new term "probabilistic programming" stems from the recognition that stochastic programs can be automatically interpreted as defining a joint probability model and Bayesian inference on this model equates to sampling execution traces that are consistent with the observed data. This is an extremely powerful idea. It enables one to quickly build and compose both the standard Bayesian models you'd find in textbooks, and innovate easily to build new models tailored to particular application domains, often by composing more standard models with more exotic ones. Probabilistic programming provides a natural way to specify these complex models and has the potential to hide the computational difficulties that plague applied Bayesian modeling.

Making all this practical is the real challenge. Performant sampling of execution traces for a broad class of models is an unsolved problem, which is part of the motivation for DARPA's new project.

For those interested in this area, we're working on one effort alone these lines at Sense: https://www.senseplatform.com.