Given how easy ES is to operate, I don't know why anybody would pay for a less flexible, more expensive hosted version.
Given how easy ES is to operate, I don't know why anybody would pay for a less flexible, more expensive hosted version.
Easy. Amazon's service is fully managed. Getting ES running on EC2 is easy enough. Making it fault tolerant, automated backups to S3 and multi-az capable, takes more work. Even more to make sure it works properly.
Add the time and complexity required to test your recovery/tolerance/redundancy plans, and you pretty much lose out on the savings.
Given the choice of spending 2-3 days getting everything configured right, vs letting Amazon deal with it, I'd pick Amazon.
Using platitudes: the time I save not dorking with infrastructure is time that I can spend solving my customers problems.
I was renting two ~$300/mo servers at Softlayer to run MySQL (Percona), with one as a backup + failover + to run my monthly browser/OS market share reports against without killing the website by overloading the main server. I regularly spiked high loads that killed latency anyway this spring. I was going to have to upgrade soon anyway, so I decided to try out Aurora during the free preview period, and migrated everything over there one weekend.
Turns out a single R3.Large instance (the smallest) was more than sufficient. It's sitting at under 25% CPU and 50% memory usage right now, which means there's plenty of room to grow too. I'd say performance is as good or better than native MySQL based on that. I'm paying less now than I did for the two servers, and get more for it, with free multi-AZ replication, nightly snapshots, and the ability to spin up a live replica in just a few minutes to run backups/large reports against, then shut down and only pay for the hour.
How long does it typically take for the live replica to come online?
It's never 2-3 days, especially when the next dude managing it needs to be on-boarded. He might not be familiar with how you do things.
This externalizes all that management overhead.
One less thing that you (and your co-workers, and the next dude after you leave) can break.
Also all the configuration settings are exposed via a neat UI rather than hidden in various files scattered out and about. Yes source control, Puppet, Ansible, Docker, etc makes this vastly simpler but then you need to hire and train people that are familiar with these things.
For people who don't have time or resources to actively manage an ES cluster, security is another reason to go with Amazon's service.
We're currently using Found to host our ES index and we had to come up with some pretty creative solutions to avoid embedding the auth credentials in our applications that need access to ES. With the new service, we'd simply setup the correct IAM policies for ES and assign the proper IAM roles to the instances running our application and be done.
you overestimate the average web dude's ability to run a service that actually works, and stays up.
you also underestimate the budget that lazy devops people have when their employer is used to (i.e. captive to) to a $100k amazon bill that really should be $25k somewhere else.