False. I'd suggest you take a look at their architecture overview. It's not like renting instances on EC2 at all because dynos != virtual machines. http://www.heroku.com/how
"Of course, people then start enumerating a whole bunch of stuff that they don't have to worry about when using Heroku. However, when starting out, configuring a server is just like configuring your localhost development environment. You just have to start with something that works, then gradually keep learning."
This may be fine for some, but for those that _don't_ want to worry about sysadmin work and want to focus on rapid iteration, Heroku makes more sense. Besides, it's not the initial configuration that is the most painful, it's the maintenance and cost associated with managing your own virtual instances, and infrastructure pieces such as reverse proxies, caches, etc.
"since GitHub's servers don't go sleeping when unused."
I'm not even sure what this means. Can you elaborate?