Probabilistic programming is not really "programming" as we might use the term without qualification. It's wholly focused on the specification of probabilistic models, their structure, data, and evaluation pattern.

Consider BUGS as it's a very prototypical probabilistic programming example. If your goal is to specify that you have a model where you observe the "wetness of the grass outside" and infer whether it "rained previously" then you can use BUGS to express it easily.

    model {
      rained ~ dbinom(0.1) /* prior probability */
      for (i in 1:N) {
        grass[i] ~ rained
      }
    }

for some data grass[i].

(That was a very sketchy example, don't read into it too closely)

The heart of this is that quantitative methods are getting so well understood that we can express the algebra of models as a formal language. When we do this we get many of the benefits that programming has done for formal expressions of flow-charts.