If you just use EC2, no EBS, and reserved pricing then AWS is just as cheap as other providers if not cheaper. It is all the nice features that make it expensive. S3 is pretty damn cheap too.
Of course, raw hosting costs are only part of the calculation. You have to include sysadmin time and salary.