dynos != virtual machines
Well, yeah, a small instance on EC2 is the equivalent of 20 dynos, maybe more.
What you do get with dynos is scaling out when you need it, however you can do the same thing by having a prepared AMI, a load-balancer and a bunch of scripts with which you can start new instances in seconds.
focus on rapid iteration
I really do think that sysadmin work and rapid iteration are orthogonal. When you're starting out, administrating you servers is something that hardly takes up any time ... but the flexibility is priceless.
... infrastructure pieces such as reverse
proxies, caches, etc.
Well, Cedar doesn't have Varnish anymore -- and IMHO, setting up Varnish is just a day's work, which includes configuring it for your own needs.
Yes, Heroku takes that away by (1) giving you a useless Varnish configuration OR (2) eliminating Varnish altogether, as they couldn't figure out a common denominator.
Also, I know that "reverse proxies" and "caches" sound bad ass, but it's really a solved problem.
Of course, infrastructure can get very hairy further down the road, but that's what I've been saying -- when starting out, you just need Passenger or mod_wsgi, as in one "sudo aptitude install" or "gem install" away. It took me a day's work to configure an EC2 instance, including a deployment workflow with Capistrano (for the first time ever). That server is still running just fine, with no further maintenance.
And if small deployments is not Heroku's strength, than what is? If you've got a successful app that needs special infrastructure care and you can't afford a good developer/sysadmin to take care of it, then you're doing it wrong.