Dreamhost, as an example (seeing as I use them), just upgraded all of their servers to support PHP 5.4, which is much better than any of the other hosts I've found. They're cheap, give relatively good support, and have recent versions of PHP available. If there's a problem with PHP or apache, that isn't broken application code, they'll fix it.
With python or ruby, you're on your own.