Say you have a model, y = mx + b, and you have some data. Normally you can solve this explicitly with matrices or whatever. But you'll need to implement the solution. With probabilistic language like BUGS, you can feed in the model and the data and it would return `m` and `b` as a probability distribution.
This is an trivial example. One example of where it really comes in handy is doing bayesian inference.
Consider BUGS as it's a very prototypical probabilistic programming example. If your goal is to specify that you have a model where you observe the "wetness of the grass outside" and infer whether it "rained previously" then you can use BUGS to express it easily.
model {
rained ~ dbinom(0.1) /* prior probability */
for (i in 1:N) {
grass[i] ~ rained
}
}
(That was a very sketchy example, don't read into it too closely)
The heart of this is that quantitative methods are getting so well understood that we can express the algebra of models as a formal language. When we do this we get many of the benefits that programming has done for formal expressions of flow-charts.
Given that probabilistic graphical models is in general NP-hard (or #P-hard) then approximate algorithms are often used. However, for many problems discrete valued networks can be effectively managed with exact algorithms.
Some tools for exact inference (that are free or commercial with free versions) are:
http://genie.sis.pitt.edu http://www.norsys.com/netica.html http://www.hugin.com
Netica in particular has a large library of example networks.