I've dabbled in Stan, and it's really good and state of the art for Bayesian inference. Starting using Stan is a bit difficult though, as it has a C like programming language that is difficult to master initially. Especially since statistics is usually done in languages like R, so the learning curve is a bit steep for beginners.

I've personally liked PyMC for simple models and relative ease of inference, as it's more integrated with the Python language. That being said, if you want the latest in inference methods and statistical alchemy, Stan is the place to go.