Just buy a real VPS server somewhere and avoid all the issues with data persistence and EC2. Use EC2 for heavy processing and use S3 for backup/storage/static file server.
Going down the path you described will be full of pain.
Going down the path you described will be full of pain.
Yes, it's possible to design an application to either not use a database at all, by storing _all_ your data on S3, or to implement some workarounds that deal with DB master-slave replication, store commit logs on S3, etc etc etc. Question is: is it worth it? In my opinion, no.
S3 is great if you want to store images, movies, etc and aren't very popular worldwide. EC2 is great if you need extra processing power to deal with extra traffic coming from those Digg/Slashdot stories, or doing batch processing like converting movie formats, but going 24/7 on simple EC2 or EC2+S3 (using S3 as a database and not just for storing media files) is more of a hassle than a "good enough" solution. VPS is definitely the way to go if you cannot afford to lease a couple of servers or some rackspace for the cheap 1u boxes you can buy at ebay.
If Amazon sorts out the other issues EC2 could be really great.
Also, even though S3 doubles as a backup medium (and you _must_ backup your database), if you want a database running on EC2 you _must_ go for S3 to store your information. That too will increase your costs.
And on top of all that, you have to spend some time tweaking and testing your configuration (and testing again). If you're short on working hands, the time you spend on this activity may not be worth it.
I think that, unless you're doing it just for fun, you should let someone else help Amazon sort out the kinks of their system.
In the end, having your entire system on EC2 will not give you an edge over your competition, but the time you save by not doing it probably will.
I don't really need a monstrosity, just something small and reliable.
Slicehost and other similar services (VPS') are about as cheap as it can get but bear in mind that you're sharing hardware with other people so don't assume you'll be the only one using the server.
If you want real hardware, google for "managed hosting". Alternatively, if you already have a 1u server lying around or can get hold to one (or more) it may be cheaper to search for "colocation services" which should show you some companies that will provide rackspace for your servers.
Renting hardware may be cheaper but if you buy a couple of servers on ebay and go for colocation, in the event of your business going belly up you can sell the hardware and maybe get enough money to last you for a couple of months while you search for a job or make new plans for world domination.