I'm consistently surprised that more people don't use them for workloads that don't require super high reliability, but do use an enormous amount of compute/storage/bandwidth. I think a lot of services fall into this category.
AWS et al. charge insane markups because people fall for their marketing, which is cleverly disguised as spec sheets and technical documentation and other things engineers who think they are not susceptible to marketing fall easily for.
someone's paying for all those free AWS instances, and it ain't the low end customers.
edit: if you want to know the true price of the hardware, take the purchase price of your standard supermicro server, and multiply it by .025 - that's what big companies pay per month for hardware, usually on 3-year leases.
i.e. $10,000 box = costs $250/month. A decent cloud provider can usually get about $3000+ a month of revenue off of a box like that. even after you add power, networking, staff, it's still a large markup.
If you're designing for distributed data/services from the start, it's better still.
the entire $10k server costs amazon $250/month.
Never had a problem with their service, but their pricing is all over the place
I got so annoyed with it that I created a simple bash script that would attempt to connect, pipe the results to /dev/null and then connect again. Always failed on the first attempt, always worked the second. Their support weren't able to help me either.
The only upside was that it forced me to add decent connection error-handling to my scripts.
What happened when you have internet interruption during development/testing cycle?
To further separate dev/prod you can create separate VPC (virtual networks) and use IAM (ACL) to limit access. It's also possible to create an AWS account for each developer and keep an eye on the spending with consolidated billing but in practice devs often want to share infrastructure for example when testing out something together.
Downtime sucks, if it happens a lot try investing in a better Internet connection. It's possible to re-use ansible formulas to deploy something locally but usually you'll end up booting machines that themselves need internet to fetch packages / dependencies anyways.
I try to design my app with 12-factor in mind (http://12factor.net/) so all the configuration is passed to the app at run time (either ENV vars, config files, args or pulled from etcd). I'm trying to run the exact same code while in dev and while deployed in prod. For example I will absolutely try and avoid having "if Env == Test" or "if Env == Prod" in my code.
For example, my service connects to S3 to store/retrieve files. The access key, secret key and even the bucket name are all passed as ENV variables so I can work on a test bucket easily with a test api key without changing code. Some services I can run locally (memcache or redis are really easy to spin up locally) and some services I'll rely on Amazon to provide like S3 even in dev/test. Of course this requires a reliable internet connection but it's the best way to deal with this for me as a single developer/devops.
I'm sure bigger teams handling larger projects could give a lot of valuable information on the subject, I would love to hear about it too.
OVH will rent you servers for as little as a month at a time, for a _massive_ reduction in cost over AWS. We're talking >10x. Are there really startups that only have capital to last a week?