The only thing that a more expensive provider will give you that linode does not is server management which can be useful but I don't really mind sysadmin at this point in time so I can't really review that.
I've had a few Linux VPSs under $3/month (posted specials on lowendbox.com). I agree with the article: don't expect the world for $2/month, back up anything of value frequently, and enjoy the too-good-to-be-true ride while it lasts. The performance is hit-or-miss, you'll find your VPS rebooted more often than you'd like, but for dinking around with an idea, they're perfect.
I'd even be comfortable using these cheap services for backup mail relays or DNS servers. Just don't forget the backups.
Most seem to support PayPal subscription payments, which makes them easy to set up and forget.
Shh, don't tell them, but given the'll host unlimited domain zone files I even use it to manage a few extra domains that have no zone entries pointing to a linode VPS. They don't seem to check/mind.
I'm sure their VPS support would be just as great.