I paid $9/yr. for my WordPress shared hosting. Unfortunately that plan is going away this year. VPSes seem to start at $20/mo.
I'll probably move my blog to wordpress.com hosting and then try a combination of Heroku and Amazon EC2 micros for my app prototyping.
thrustvps.com is great, I paid 50$ for a year with a 512Mb RAM, 15GB Disk Space + 1TB Transfer per month VPS.
yeah but the support isn't that great, that is where you suffer. They were really sluggish so I moved everything over to http://www.pacifichost.com and despite the cost they have been really great
You can get a vps at intovps.com for $10 a month, I'm not affiliated with them in anyway just a happy customer