As an aside, does this line bother anyone else?
> programming from a computation/understanding-first, mathematics-second point of view
Mathematics is understanding. What's implicit? The suggestion that mathematics isn't understanding, it's just funny symbols that no one understands.
> The latter path is much more useful, as it denies the necessity of mathematical intervention at each step, that is, we remove often-intractable mathematical analysis as a prerequisite to Bayesian inference.
In fact, I would say that by removing maths you're removing at least some amount of understanding. You're left with a workflow of "if x use model a else use model b." That will work until it doesn't, i.e. until you have to face a problem that isn't predicted by the workflow. Not saying that's a bad approach (it's certainly the most pragmatical) but it shows that the phrase "understanding first maths second" is misguided at best and disengenious at worst.
I think a more appropriate phrase for what the author describes in the prologue would be "practicalities first, theory second".