1) My scale is significant enough to require at least 2 servers (unless one of them is RDS).
2) I have some background processing.
3) I can commit to reserving an instance for at least a year.
A small ec2 instance running 24x7 will cost me around $40 / month (+ bandwidth) if I reserve up at least a year's worth of usage. That's 1.7GB of RAM and 1 EC2 compute unit. For $163.43 you can get a pretty honking large instance (way cheaper than any competing VPS offering). Add in ELB and RDS and you've got a very scalable setup for very little coin.
I'm not sure how the Heroku dyno or worker performance measures up to EC2 Compute units though, but I'd love to hear from someone who does know.