Anything else that I should look out for?
My favorite has always been (and continues to be) http://www.linode.com/ for their value, simplicity, control panel and support.
I've always wanted to try http://www.webfaction.com/ too, might give that a shot next for a small project.
Setting up many modern webapp frameworks requires more than 'basic linux shell skills' and it requires sitting on several security mailing lists to watch for holes in the framework, etc...
Did a prelim computation for the EC2 fee equivalent of Linode 360 with 16GB storage and 200GB transfer. Assuming these limits are maxed out, it seems EC2 is more expensive by 10 bucks.
Look for one with an uptime guarantee and good customer service. Most have a 30-60 day money back guarantee as well, so you may need to use that if what you get isn't what you expected.
Slicehost has no money back guarantee. Linode refunds your 'lost time' (which is fairly standard in this industry.)
Were I you, I would completely ignore SLAs and uptime guarantees. the inconvenience downtime causes you will hurt you far more than any reasonable refund can make up for, and the reputation damage hurts us (the provider) far more than any reasonable refund would as well, so really, for a 'getting started' plan, the SLA should be about the last thing you look at.
Excellent support, intuitive management-console and lots of install scripts for various dev platforms.
the problem with the basic 'small' instances is that if there are system problems, you lose your storage. Sure, you can get around this by paying extra for EBS, but it's another problem you need to deal with (and pay for) that is largely taken care of on most VPSs. (which isn't to say you will never have data loss on a VPS, but all the major providers mirror disk, so it should be fairly rare.)