I still regularly run into problems with websites and applications related to geolocation, blacklisting, or routing, and being able to proxy stuff through my VPS enables me to keep getting things done.
In particular, if you're a Comcast user a VPS is pretty essential since their routing and peering are awful.
Being able to run sensitive traffic through an SSH tunnel when you're on public Wi-fi is also a nice perk.
EDIT: I'm talking about Chicagoland region. Didn't know it was geographic.
Caveat: These low-cost "companies" go out of business all the time, so you'd better read the reviews. But since I consider them disposable anyway, I don't really care if the uptime isn't 99.9% or the company goes bankrupt after 9 months. If you plan to do anything other than tunneling with your VPS, I'd suggest that you pay at least $10 per month.
They have a friendly management interface, which keeps getting more impressive over time. They scale very well based on what you need, and I've found their support quite helpful as well.
I also recommend their domain name registration; in addition to having much better policies than most registrars, they also provide email for their domains, which saves you the trouble of maintaining a mail server yourself.