The benefit of the Bayesian treatment here that I want to drill down on is how natural it is to adjust the prior to capture your beliefs about how items should be perceived in the presence of incomplete information. The frequentist approach is fine, but it does not provide such a pleasant, intuitive knob to tune.
But I agree that the Bayesian approach is conceptually much cleaner. IMO the frequentist approach is just computational corner cutting for when the math in the Bayesian approach gets too involved, which is sometimes useful. What's nice about the Bayesian approach is that you state your assumptions and then it's just turning the math machinery. In contrast, in the frequentist approach the assumptions are interwoven and hidden in arbitrary choices in how the math is done (And then they claim that Bayesians are subjective! It's just that Bayesians admit that they are subjective. Frequentists try to hide the fact that they are more subjective in the math). The not so nice thing is that turning the math machinery is not always so easy and does not always produce fast algorithms. That's where maximum likelihood and friends come in, but I'd view them as an approximation to Bayesian methods.
I love Bayesian statistics from a conceptual point of view, but the ease with which one ventures into the land of analytic intractability kind of puts me off more complex models. MCMC is such a clumsy tool (in addition to taking forever); variational methods look interesting to me but I don't really feel they are quite there yet.