"Updating priors" is quite complex, and arguably describes every interesting task you'd want to do as a learner, reasoner, analyst, etc. One view of probabilistic programming languages is that they allow you to express your priors—i.e., uncertain beliefs about the world—as programs, and help you to answer all sorts of interesting questions in the contexts of those programs.

As a concrete example, you might write a program expressing your beliefs about how the scenes you see come to be: some random objects are placed at some random positions, lighting sources are placed, and then a rendering algorithm converts that scene description into the light intensities perceived by your eye. If you write this up in a probabilistic programming language, you can then ask questions like: supposing I see this image, what are some likely scenes (positions and shapes of objects) that might have produced it? How does that distribution over scenes change if I know that the light source was here, or that there was a small chair there, or that my vision isn't that great and I should trust the image's pixels a little less? Section 1.3 of the introduction linked above lists many more applications for which probabilistic programming turns out to be useful: breaking CAPTCHAs, constrained procedural graphics, automatic program induction/synthesis, recursive multi-agent reasoning... and, as the authors write at the end of this section, "the list goes on."

Of course, solving the inference task in full generality, for any program and query that the user might write, is very difficult; you can design pathological cases where it boils down to solving the halting problem, and is therefore uncomputable. Different languages take different approaches to addressing this difficulty. They might limit you to writing certain kinds of programs for which a limited class of inference algorithms can always be made efficient; they might try to plan a fancy inference algorithm based on the specifics of your model; they might just expect you to learn which programs will be tractable and which won't; or they might provide abstractions that make it possible for you to safely experiment with writing new inference algorithms (or combinations of existing ones) specialized for your use case. Probabilistic programming is still a young field and this is an area of active research.