I checked with Heroku regarding this before considering using it. It seems that you should not be seeing the above (atleast on their latest cedar stack).

When you make a new push to Heroku, they seem to make sure to setup the code on a different machine, get it up, and then finally redirect the incoming requests from the load balancer.