I will say I very much enjoy Elastic's suite of products, I can't say the same for MonogoDB.
MongoDB (the company), has been very strategic about defending themselves from Amazon. Likely due to the AGPL licensing on MongoDB (the database), Amazon doesn't offer a hosted MongoDB - you have to buy MongoDB Atlas which runs on multiple cloud providers, including AWS, GCP and Azure. Coincidentally, Atlas subscriptions are the fastest growing part of MongoDB Inc's revenue, which is why investors are bullish on the company.
Elastic (Apache licensed) has actually had their lunch eaten by Amazon to a much larger degree. Many businesses are purchasing hosted ES instances directly from Amazon via Amazon Elasticsearch service. The problem is bad enough that Elastic Inc had to write a marketing blog post about it [0].
Say what you will about MongoDB (the database), but MongoDB Inc. is the most successful open source software-based company since Red Hat. You can see it in the outperformance of $MDB compared to similarly timed open-source IPOs such as Hortonworks ($HDP) and Cloudera ($CLDR).
[0] - https://www.elastic.co/blog/hosted-elasticsearch-services-ro...
Now, I don't have a lot of data in the cluster, so we'll see how performance is when that happens, but would definitely recommend Elastic's offering.
Performance is miles and above better, there’s actual security options on the clusters and there’s way more features.
Elastic Cloud runs in either AWS or GCP. You get to choose but it's just between those. Unless you mean you were using AWS's ElasticSearch service... which is known to not be all that great.
You could build ES clusters on AWS directly on EC2 and it would be cheaper than Cloud, unless what you're really looking for is that support contract.
I spent a few months of my life learning ES inside and out and would choose this option every time. One of the best tech investments I've ever made.
Or you are willing to trade money for dev time. Setting up and running the ES cluster on EC2 will take time and energy (source, I did it ). Whereas setting up the ES cluster on either the Elastic Search hosted service or AWS ElasticSearch is really just clicking some buttons.
However, I'm sure the right solution also depends on the amount of data you are using and the problem you are trying to solve.
If you have strictly time-series data and are willing to make compromises around retention window, hosted elasticsearch is an easy conclusion. Invariably though, I find that where performance is a real concern, you have to carefully plan what data you're indexing and that requires operations knowledge around elasticsearch. At that point you almost might as well host it yourself anyway.
Also if X-Pack was something I _had_ to have, and tbh there are free alternatives for all of its good features, I would consider Elastic Cloud.
I'm even a customer of theirs on some clusters, but lackluster support and pushy upselling experiences have made me want to move out of it.
could you list a few?
Can't recommend Sentinl & SearchGuard enough.
Also the reason Redis Labs made the controversial decision to change their licensing (and they reference Elastic in it) [1].
[1] https://redislabs.com/blog/redis-license-bsd-will-remain-bsd...
> Cloud providers have been taking advantage of the open source community for years by selling (for hundreds of millions of dollars) cloud services based on open source code they didn’t develop (e.g. Docker, Spark, Hadoop, Redis, Elasticsearch and others). This discourages the community from investing in developing open source code, because any potential benefit goes to cloud providers rather than the code developer or their sponsor.
It's a big deal for a lot of companies to be able to pay the cloud bill in one place. Elastic used to provide the availability to pay for the Elastic cloud subscription through AWS' Marketplace. Elastic stopped this when they changed their pricing at the beginning of 2017 and we (my last employer) were negotiating a contract with them at the time. AWS billing was a hard requirement for us because it made it a lot easier on us from the finance side. We walked and I know a lot of others who did too.
The point is that AWS removes friction and Elastic chose to add friction. They're chasing big Enterprise hard and it doesn't seem to be working out.
Big Enterprise will not require billing through AWS (disclaimer: employed at a large enterprise, buying ~$900k software license currently). They’ll have an accounts payable department, a procurement process, and someone to cut a check or ACH to Elastic monthly/annually.
Would you want to be beholden to AWS' whims? You might as well be a Walmart supplier at that point, constantly dragged out to Bentonville having to make your case for every penny of your costs. Amazon has proven its model (all about scale), how it operates (your margin = our opportunity), and you partner with them at your own peril.
I'm suggesting the thought process to shift from, "Ahhh, those Elastic guys and their bad decisions!" to "Hmm, well, tough decision to make [forgoing revenue through AWS] but I understand why they made it".
Time is the ultimate non-renewable resource. Throw dollars at the problem when you have them and it’s cheaper than your time.
Recently I turned off some slow query and slow index advanced logging options, again AWS completely rebuilt the entire ES cluster when these options surely should be able to be applied without copying all data.
From what I can tell, it still does. I can’t comment on the length of time though. I did it before I had many documents and it was a very small Dev instance.
[1] https://aws.amazon.com/about-aws/whats-new/2018/03/amazon-el...
https://aphyr.com/posts/317-jepsen-elasticsearch
https://aphyr.com/posts/323-jepsen-elasticsearch-1-5-0
Though, all those issues have been fixed for both MongoDB and Elasticsearch so it's not necessarily fair to judge them based off past performance.
https://www.elastic.co/guide/en/elasticsearch/resiliency/cur...