The most compelling use case I've seen where Passenger loses to Unicorn is rolling restarts. The biggest trust issue that I've seen is that of source editability - Unicorn and Mongrel are easier to debug and patch, while Passenger requires a commanding knowledge of Apache to fix.
Do you routinely edit the source code of your web server? Just curious since I've never needed to do that.