The real problem is that their extremely high costs are an enormous barrier to entry for anyone trying to build a product that is more than just a low throughput CRUD app.
$35 vs. $500 means that when you are in the bootstrap phase you do not use Heroku. Meaning you do not get used to the very nice quality of life features they have for deployment, CI, etc. Then by the time you have income or an investment switching to Heroku makes even less sense because you've already invested into whatever stack you are using.
With the availability of high level tools on AWS and GCE, it does not take that much work to get to a functional system for CI and deployment. My small team of three at my last gig managed to tie Bamboo to a large scale AWS deployment with what amounted to basically a side project worth of effort.