I tried using Heroku for a (very) small project which I already had working in Ruby, and using Google App Engine (even though it meant rewriting the code in Python). The reasons were:

1. Heroku didn't support a certain Ruby gem I needed for the application.

2. While (1) in itself is understandable (and eventually I found a way around it) Heroku never replied to my email asking whether they would support it in the future. As far as I recall there was no other way to contact them but feedback@heroku email address - if they don't answer it, that's that.

3. Heroku is just not certain to be around for as long as Google (or Amazon etc) will.

4. I admit I didn't research this thoroughly, but Heroku seems more expensive for what you get than other options (eg GAE, perhaps even Slicehost & co)

All in all, I'd say support was by far the most important issue. The last thing you want is to trust your application to someone who won't respond quickly (or at all). I realize Heroku is a small team, but this is just a critical requirement for such a service.