What they are really competing on is breadth and depth of service. The article goes into a lot of those services, but, as one example, if you launch an instance in EC2 you can allow it to access secured buckets in S3 without any need to store keys/passwords on the instance itself thanks to IAM roles.
Another example is services like AWS Lambda, which is a hosted way to run a function without any need to manage servers.
The list goes on and on...direct VPN connectivity, Hosted Active Directory, CloudHSM. While I'm biased, my perception is that AWS is pretty far ahead of the pack.
Look at Reddit, they have very few employees yet they can't make a profit because their hosting costs are massive.
Contrast them with StackExchange, which uses a small number of powerful, well-optimized servers and is very profitable.
Kudos to you for touching on containerization, I would just like to add that the many emerging platforms and tools (https://pbs.twimg.com/media/B33GFtNCUAE-vEX.png:large) are rapidly starting to provide an alternative to AWS that was not there before in their ability to realize more competitively priced hosting alternatives without sacrificing on administrative overhead.
Not valuing your time is easily among the biggest mistake you could make.
The real trick with AWS, IMO, is figuring out what you want to do yourself vs what you want to have hosted. For example, they offer a hosted cache such as memcached or Redis. For my app, I run Redis myself as it required roughly 15 minutes of work total to set up and maintain in 2014. OTOH, that's not nearly the case with Postgres. I spent a day and a half tuning the version I host, switching to RDS, then switching back because RDS was too slow (even though by the specs, it was using a more powerful VM than the EC2 box I was running my instance on).
Basically, once you grok AWS, things become more productive AND you don't need to spend time on doing things like setting up VLAN's, transferring data between boxes, etc. You can do cool things like detach a virtual disk from one VM and attach it to another, rather than pushing 100 GB's of data over a network. This, AFAIK, is not something that Linode or Digital Ocean can do.
Having said that, DO is great for lots of things. It's a good place to experiment and to run lots of really small things. For me, that's side projects or things that I am trying out before thinking about moving them to the "big iron".
> OTOH, that's not nearly the case with Postgres. I spent a day and a half tuning the version I host, switching to RDS, then switching back because RDS was too slow
These two statements are contradictory.
You would have been better off just renting a dedicated host for your DB and doing the tuning yourself (either in person or getting someone to do it) - because the AWS offering is slower than your own instance, and database is a service thats often ripe for easy optimisation by using a dedicated host/cluster of hosts vs virtualised host(s).
My point about physical was just another common optimisation for performance critical DB servers, you could probably get close to bare metal speed with the flexibility of virtualised by using a "single VM on hardware" model that some providers offer now.
I'm not suggesting that's ideal for all companies, but this idea that somehow it's impossible to run a load balancer or store files without using AWS/Azure/Google 'clouds' is a bit ridiculous.
Running an FTP server != an S3 equivalent.
For the majority of those people, a simple SFTP/rsync/NFS/whatever endpoint (potentially with replication to provide redundancy) would more than fit the bill, and would actually be simpler to use than the S3 API.
For larger setups you can do it at 1/3 or less of the cost if you're prepared to go the colo route (rather than managed servers) and leasing your own hardware.
If your average bandwidth usage per object is high you can trivially cut costs far beyond that, as the EC2/S3 bandwidth costs are atrocious.
The exception would be if you can't avoid huge amount of object accesses from within EC2 even if you move the storage out of S3.
The same resources that Sysadmins used to learn how to do it?
Or hire a sysadmin part-time.
Flexibility of networking, Regional coverage, Tooling , Performance,
On the cost side with reserved instances it's less expensive.
Having said that, nothing beats getting real hardware in terms of performance/$. I used to work for a company that exclusively worked on SoftLayer's hardware servers. These were $500+/month each, but they were fast. The point is that if you can devote a dual octocore machine to what you are doing, and are willing to pass up on the SAN, flexible networking, etc. then you get a very fast box. If what you are doing requires lots of very fast hardware, then yea AWS or anything like it is not really for you. But if you are like most people, your AWS bill is not going to break the bank and you can just spend one less billable hour figuring out why the server is slow, and just double its size.
I use AWS, and will continue to use AWS, because it means less screwing around with stupid things and more time making important things work.
Since you're writing to multiple EBS volumes for the RAID 1 portion of the RAID 10, you're going to require more EC2 to EBS bandwidth.
EBS volumes are replicated across multiple servers, so you don't necessarily need the reliability, especially if you're replicating the data elsewhere.
YMMV, of course, and everyone has different priorities and different levels of what constitutes acceptable risk, but RAID 0 with EBS isn't quite the data death wish it is with physical hardware.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/raid-conf...
No, it's not. It is if you want data-at-rest storage, but I don't care about that when I have multi-master MySQL and Postgres running on instance stores. Or when I have replicative NoSQL databases (Riak is more than happy running forever in instance stores and duping across AZs).
The point is you aren't locked to a single vendor. Heck, even mix and match - replicate your services across providers, across data centres.
EC2 seems to handle all that nicely by letting you just upgrade instance types as you need them, and providing burst performance for spikes in traffic. It seems like a pretty fire-and-forget solution and I've not paid them any money yet (10 days, 30,000 hits)... So far I'm happy but really I've no idea what I'm doing.
http://journal.dedasys.com/2015/01/22/when-to-use-aws-and-wh...
tl;dr: I would not use AWS unless you know that you need what it offers.
A 60GB r3.2xlarge spot instance on AWS will cost you about $50/month while on DO you'll be paying $640/month for a 64GB machine.
Granted, the DO machine will be way more performant, and you have to have a workload that suits spot instances.
Every time I look at things like DO to see if I'm missing out I price up my current AWS setup and they're always astronomical (for my use case).
Then there's all the other control around AWS; it's so much more advanced than all the other services. Nobody else really comes close to the flexibility they give you.
For a lot of (probably even most) web apps that probably doesn't matter. Have a look at your use case and decide if AWS is worth it.
In every other regard - disk space, CPU cores, data transfer, availability - the AWS spot instance option is MUCH worse than a regular VPS like from DO, or even a dedicated physical host.
Ignoring the CPU core difference, a regular (i.e. so you can use it all the time) instance with extra EBS storage and data transfer to match the DO instance you mentioned, would be $1500/month.
Places like Rimuhosting will rent you a pre-used dedicated server with 128GB RAM, 2TB HDD + 200GB SSD Dual Xeon (12 cores, 24 HT) physical server (which you can then run a single Xen VM on, for better portability) for $400 a month. Yes it's more than your $50 a month, but its completely yours, for as long as you want it.
To truly take advantage of a spot instance and NOT have it impact your business, your workload needs to be very tolerant of interruptions and variable processing time per day. For the vast majority of people using EC2 (web/app servers running 24x7) it's not a realistic option.
Aside from that though, spot instances aren't that volatile. I have some that have been running for months. Obviously they're not always at the low low price they can be, but if you average it out, they're not much higher than that.
In total honesty, I don't use those machines I mentioned. I use ones with 32gb ram and much better CPU performance. I need that ram for what I'm doing and any other provider I've found comes in a lot more expensive to satisfy that requirement. The CPU I get is good enough for what I'm doing.
But again, I don't know anyone other than Amazon that has an infrastructure that flexible at that price.
If there's anything incorrect with what I've said, please point it out. I'd love to know about other providers that would give me more bang for my buck.
Is it the assertion that spot instances are a viable option for some use-cases? Is it the fact that I compared something to DO?