I went VPS because:
1. I'm more familiar with configuring my own web apps then trying to use someone else's admin console. (Low opinion of said consoles that I've seen in the past, but haven't tried Heroku or other new services.)
2. I need to run Node.js, PHP, and Ruby all on the same server. (Actually, just PHP and Node.js, which are in the same web app and so need access to a shared database. Ruby could be elsewhere at the moment.)
I'm still on VPS because:
1. Already paying for it, hasn't been a year yet
2. I still don't understand the new PaaS thing
3. PHP + Node.js PaaS host? Preferably with name recognition on par with Heroku, but at least lots of good reviews.
Complications of VPS are numerous, so I won't enumerate them. Lots of things to configure, update, and keep alive. Also it isn't the same OS as my dev machine plus I have to manage security for a public server but not for my desktop machine.
EDIT: just thought of one more reason I use VPS, I actually like ssh shell access. Ability to edit files and run arbitrary commands, many innumerable uses of this.