If you pay by the year, especially if you don't use much bandwidth (and you won't, if you are just figuring things out) ec2 is quite competitive on price, if you don't mind the problem.

the problem with the basic 'small' instances is that if there are system problems, you lose your storage. Sure, you can get around this by paying extra for EBS, but it's another problem you need to deal with (and pay for) that is largely taken care of on most VPSs. (which isn't to say you will never have data loss on a VPS, but all the major providers mirror disk, so it should be fairly rare.)