I think the behavior of the "C" collation is what most software developers expect. Plus, there is an advantage to the behavior being the same everywhere, rather than dependent on the system/environment settings.
But those preferences may be specific to my situation -- a software developer running an HA web application.
* If you're not a software developer, you might be surprised by the "C" collation, which puts "Z" before "a".
* If you're not running an HA web application, then it's easy to change the collation later -- just dump/restore the DB.