How about hosting though?
You get the pricing of shared hosting, but you get SSH access to install stuff in. Combined with the GUI for setting up common apps (Django, etc.), it's pretty nice.
If you grow larger I'd go with SoftLayer, not the cheapest option but I've found them to be very professional and reliable.