Elastic and Amazon reach agreement on trademark infringement lawsuit
elastic.co
elastic.co
I’ve been tossing up moving our workloads to Elastic Cloud anyway, because AWS ES Service is a source of constant headaches for us. Feels like at least once a week a server ends up in a state where we can’t fix it, and AWS engineers have to manually fix their internal state.
Their standard response is “add more nodes”; well, we did that, and it is costing us an arm and a leg, and it didn’t fix the problems. (Plus, now we have new problems where networking blips appear to be causing quorum problems and sending the cluster into a death spiral.)
Purely off the customer experience, it feels like Elastic Cloud has to be better; the whole licensing debacle has definitely turned me off Elastic though.
Their move away from open-source has been unfortunate. For that and some other reasons we've ended up more impressed with Logz.io and Splunk SaaS.
We saved a bunch of money and gained performance by using our own cluster. That cluster hasn't gone down since... years later.
It's very difficult to debug these problems when you don't have direct access to elastic search's configuration... what would normally take minutes to verify can take hours to isolate.
Effectively a newer config file was lacking a jvm.options line that changed the behavior with an older machine setup. I would not be surprised if AWS deployed a new config to an old environment.
Unfortunately, not having direct machine access, I could not confirm whether this was the case in the failing cluster.
Sounds like it might be a "grass is always greener" scenario. ...maybe I'll spend more time looking into self-hosting...
We’ve rewritten most of the interactions with the older version of ES into a new service and use a self-managed OpenSearch cluster on Graviton instances and it’s the most stable Elasticsearch/Solr solution I’ve ever interacted with
This. I have had similar experiences and adding more nodes had just amplified the issues; nodes were all also barely loaded.
If you just need "Lucene but clustered" I highly suggest looking at Solr instead, it's design is much more straightforward and has pretty much all the most important knobs for actual indexing that ES has.
If however you are tightly coupled to ES API or use it with 3rd party systems you are sort of up shit creek without a paddle...
Ran ES at large scale for many years, eventually gave up and only use Solr or custom built search engines these days.
One very good example is Amazon redis. Amazon figured out that redis asynchronous replication didn't work at scale so instead of fixing issues upstream they chose to develop Amazon redis in house and monetized it.
https://aws.amazon.com/memorydb/
Enhanced version means patched made by AWS. https://aws.amazon.com/elasticache/redis-details/
Oracle didn't build their company in the style of AWS Redis, that cloning maneuver. Oracle's database was a pioneer. Oracle didn't get where they are by cloning open source and claiming it as their own. Despite the numerous bad things that can be said about Oracle's culture, that's not one of the key negatives about Oracle.
You're entitled to your opinion but your line of reasoning for how MemoryDB for Redis came to exist or the reasoning about why it isn't in upstream Redis is not factual. MemoryDB's architecture uses Amazon's home grown log replication services as pointed out by Werner Vogel in his blog post about MemoryDB[0]. This architecture is fundamentally incompatible with upstream Redis. The real reason why MemoryDB for Redis exists is far less juicy: MemoryDB came about by meeting customers where they were. Customer's love Redis, especially for its non-caching features, but replication is a headache with all the existing solutions today.
Also as far as I know one of the lead committers to Redis is from AWS.
0 - https://www.allthingsdistributed.com/2021/11/amazon-memorydb...
There were contributions sent upstream for Elasticsearch bugfixes and enhancements. Some of those PRs are still open, for example [1].
A sampling of additional PRs can be found in this blog post [2].
Further contributions upstream to Apache Lucene have been growing over the years. The new Approximate Nearest Neighbor support in Elasticsearch 8.0 comes from work that was sponsored by Amazon in upstream Apache Lucene [3].
[1] https://github.com/elastic/elasticsearch/pull/64513
[2] https://aws.amazon.com/blogs/opensource/stepping-up-for-a-tr...
I’m a total AWS fanboy, but even I cringe when I read that superficial, customer-centric sound bite. You know who meets people where they are? FOSS maintainers.
“Also as far as I know one of the lead committers to Redis is from AWS”
Conveniently cryptic, what does from AWS mean? Do they still work there? Did they specialize in log replication?
Madelyn Olson, at present an AWS employee, is one of the members of the Redis core team [1]. The invitation was extended because she had "been actively involved in Redis development for several years, contributing numerous changes throughout Redis, including bug fixes and features."
For whatever reason they chose BSD. And now Amazon made some improvements and is not contributing back.
Not sure why anyone is surprised.
But yes, this is the reason I won't work for free on my own or others' BSD or MIT licensed projects.
ianal, but imo, Apache License v2, Mozilla Public License v2, and xGPLs v3 are better at protecting the rights of the consumers (including contributors).
''' Amazon OpenSearch Service (successor to Amazon Elasticsearch Service)
Run and Scale OpenSearch and Elasticsearch Clusters (successor to Amazon Elasticsea... '''
This seems like a petty, small win from the Elasticsearch people. I understand AWS has a history of gobbling up OSS and productizing it, and that that's detrimental, but it's hard to see Elastic, Inc as anything but sore that they got their lunch eaten here. Maybe that's justified. But it comes off as incredibly petty.
(disclaimer: i used to work at aws, but not anywhere near the referenced offerings).
IMO, they chased the benefits of being open source and then changed course when the costs to them exceeded the benefits (which is fine for code going forward), but trademark concerns aside, I can’t see AWS as the bad actor here.
With Microsoft + GitHub intensifying their investments in F/OSS, AWS had to play ball. It is smart, not petty on anyone's part.
Judging from the tone of the article, I am glad Elastic is content in their current business relationship with AWS. Hopefully, the companies also find an agreement to have AWS' OpenSearch fork merged back in, as well.
disclaimer: ex-AWS, but zero insider information.
Step back from FOSS for a second.
I think most people would agree that there’s somewhat of a moral issue with just taking someone else’s open source software and just hosting it and making billions, with nothing for the creators, because you are a megacorp who is good at hosting.
Now, is there a way to solve that and have the benefits of FOSS?
Both Mongo and MariaDB have tried to address this with licensing - MariaDB seems to have done this much less clumsily than Mongo, but both still had FOSS advocates shrieking
---
EDIT to include response to below comments so everyone doesn't keep repeating the same things:
> Either your product is fully free and you accept that people can fork it, even Amazon, or you choose a restrictive license. You can't make "free" (as in libre) compatible with "but".
> I can totally see that you think this is unfair, but they did allow it and they were perfectly happy when being FOSS brought them market share. You have to take the good with the bad
> If you have a moral qualm with bigcorps using your work for free, you don’t license in such a way that they can. Make your own license, or slap AGPL3 on it - either way, no bigcorp touches it.
> But you cannot be mad when you say “I release this code under these terms” and AWS takes you up on your offer.
If everyone stomps their feet and says "there's no solution, otherwise it's not OSS!" then the end result is only going to be a lot less open-source software.
I'm not asking if someone who wrote code and chose an existing OSS license that allows any possible usage has any real recourse once they want to make money off of it - obviously not.
I am asking, can we find a way to ensure software creators are compensated while still having the generally-recognized benefits of FOSS?
> Don't you think there's a moral issue expecting free contributions to something which only you are allowed to monetize?
Nowhere in OSS licenses does it include the expectations of free contributions
> And how can you satisfy users to the greatest extent while also preventing them from using the provider that is best able to meet their needs?
Again, can we just find a way for the creators to be compensated?
I'd love for the OSS to be available as a hosted solution to be available on every cloud provider, including if the original creators choose to provide a cloud hosting.
I'd also love for the original creators to be able to get some sliver of the money AWS/Google/Azure are making off of it.
---
To me it seems like if you were to start a business around your open-source code right now, you would be smart to do one of these two things:
1. a MariaDB/CockroachDB BSL approach, where the newest versions are BSL, with a restriction that you can't provide a product where you just host our software without paying us, then all code automatically is fully open-source (GPL or whatever) after x amount of time, or x major version releases.
- this comes with the danger of all the bad PR and gnashing of teeth that comes with non-pure-OSS licenses, or people just not adopting it because they dislike or don't understand the license
- this seems even more dangerous, that you have to be distracted on such things that are not the core product in order to defend against ending up like Docker, with everyone using it and no one paying you a dime.The answer is simple - no. Either your product is fully free and you accept that people can fork it, even Amazon, or you choose a restrictive license. You can't make "free" (as in libre) compatible with "but".
I can totally see that you think this is unfair, but they did allow it and they were perfectly happy when being FOSS brought them market share. You have to take the good with the bad.
Don't get me wrong, I feel with the Elastic team and I can understand why they're unhappy (even if I don't agree with their pettiness). I also don't particularly like the move Amazon did here. But that's why we want free software: If the current owner of the software does something you don't like, you can still use the software. That's the whole point.
And you're still free to pay Elastic to manage a cluster on AWS for you; Amazon might have forked the project, but it's still the customer that needs to switch to OpenSearch.
Not technically open-source, but much closer to open-source than what we would generally think of as "proprietary" software
Imagine just as an example, a license for personal software that is exactly the same as GPLv2 except billionaires can't use it. Functionally that has the same benefits as FOSS, even though it's not.
Honestly it seems like the only downside there would be all the OSS purist/billionaire-defender HN commenters would have a field day, then you could move on.
I’d like to choose AWS to do this hosting in a lot of cases, assuming they’re willing. Barring them from hosting it is impinging on my freedom as a user and therefore with respect to whether I think that software is Free and Open.
That’s the exact freedom that Elastic is trying to remove here, for their own (totally valid) business reasons (they don’t want to compete head-to-head with AWS hosting), but in so doing, they’re removing an essential freedom from users. This is not about defending billionaires; it’s about having choice in the use of the software purporting to be free and open.
So it's a reduction in freedom, but it's not a reduction in user freedom.
“You have the freedom to contract for help with our software with this company but not that company” or “you have the freedom to contract for hosting in inefficient arrangements that we can beat in competition but not in efficient ways that we’d rather not compete against”.
And it is in the best interests of FOSS to make it clear, time and time again, that the answer is no, and to work to ensure the answer is no.
"Specifically and explicitly" is much more of a legal argument.
If I license any of my code under that license, I hope someone builds a billion dollar business on it. If someone else does the same, how should a licensor understand the difference and act accordingly?
This feels like an oxymoron: if it's open source licensed there is explicitly no 'owner.'
* requiring code to be assigned by the author to a single person/company, consolidating ownership
* building your entire base on friendly licensed software (3BSD) and ducking any GPL
So it’s very easy to say, for example, that the Mozilla Foundation owns all the code that makes up Firefox - they won’t accept your pull request otherwise.
EDIT: regarding your inline response,
> Nowhere in OSS licenses does it include the expectations of free contributions
The expectation is not encoded in the license, it is the reason for choosing the license. Why would anyone want to afford you the benefits of FOSS so that you can personally monetize it and nobody else?
There is plenty of code that is only open-sourced after it has been fully created by a single organization/entity - no free contributions needed!
If we're talking about expectations not encoded in the license it's very clear that developers would just like to be able to release software with all those OSS benefits, but also to have a way to be compensated when AWS takes it, charges people to have it run it on an AWS server, and makes a ton of money.
I'm not denying AWS is adding value, but it sure is odd that they're the only ones making a dime there.
> I'm not denying AWS is adding value, but it sure is odd that they're the only ones making a dime there.
I don't understand what's odd about it. If ES were never open source and was a proprietary product from the beginning, there would be nothing stopping Amazon from paying for it and providing it as a service under contract from Elastic, if they thought it added enough value to make that worthwhile. Then they would both be profiting in a monetary sense from the relationship. But it's not a proprietary product (or at least, it wasn't previously)
This license case isn’t about “you can personally monetize it and nobody else” but rather “you and anyone else who wants to can try to monetize it”.
If you have a moral qualm with bigcorps using your work for free, you don’t license in such a way that they can. Make your own license, or slap AGPL3 on it - either way, no bigcorp touches it.
But you cannot be mad when you say “I release this code under these terms” and AWS takes you up on your offer.
Google won’t: https://opensource.google/documentation/reference/using/agpl...
Unless things have changed recently I know Amazon won’t, and I’m fairly sure Facebook still won’t.
For most corporations it’s simply not worth the risk
Also the Google policy seems to stem from:
1. It's never really been defined very well what "interaction over a network" means for AGPL and how truly viral it really is - although I'm sure ever there were ever landmark cases in the US establishing it, things would magically end up in Google's favor
2. Google purposely spreading FUD around more restrictive OSS licenses to discourage their existence
Absolutely not.
The AGPL terrified AWS so much that they did a clean room implementation and built an entirely new database that was wire compatible with MongoDB.
Hence, Mongo relicensed under a new license, the SSPL, that says if you implement an SSPL-covered API, the SSPL extends.
So AGPL is open source, but in practice it seems like it's so much more restrictive than something like BSL that is not open source.
This just makes OSS vs not-OSS seem like even more of an theoretical line in the sand.
As for
> Hence, Mongo relicensed under a new license, the SSPL, that says if you implement an SSPL-covered API, the SSPL extends.
That definitely looks ugly on Mongo, implementing an API is such a fundamental part of software compatibility. Seems like the exact wrong way to go about it.
I've released code under permissive licenses (Apache, BSD, and MPL that I remember off-hand), and at least one feature I know made its way into a commercial product. In those cases I simply didn't care because I didn't consider the code particularly novel, important, or special. It scratched my itch. Occasionally it scratched my employers itch, so I was already paid to write it, which was nice.
If I ever write code that I consider novel, important, or special, I plan to release it under AGPL3 - while you may think Google is overreacting on accident or to create FUD around it, the plain text of the license makes the intent exceptionally clear:
> public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version.
This alone is enough to make the AGPL3 blacklisted anywhere that wants to keep their source closed: you may believe the courts would decide for the big corporation if push comes to shove, and you may even be right, but practical evidence shows that almost nobody wants to risk it.
Part of the problem here is that a generation of programmers seems to think that open source is all the same, so a license is something that you just whack <enter> to when your scaffolding tool suggests one. But in reality, talking about "open source" is useless at best and dangerously imprecise at worst, talking about licenses is the only way to get people on the same page.
> That definitely looks ugly on Mongo, implementing an API is such a fundamental part of software compatibility. Seems like the exact wrong way to go about it.
It did the trick for them, at least against AWS: as expected, they successfully forced the wire format to diverge after AWS launched DocumentDB. It also got them pulled from Debian, Fedora, and I don't know how many other distributions for being non-free. But hey, at that point, they were already a known quantity, so it probably won't hurt in the long run.
[0]: https://json.org/license.html
AGPLv3 licensed software is also permitted for use within the company in some circumstances.
Personally, I am a fan of AGPLv3. But I am uncomfortable with advice to apply APLv3 code in an expectation that BigCorps will not touch it. There are many very large companies, including multinational companies based in China, that have no fear of AGPLv3, and are also prepared to meet the obligations of the license.
Many times I have read (including in comments down thread) that "big companies fear AGPLv3." That may be true in some cases, but not all. And the state of adoption or rejection of community licenses changes over time. 25 years ago, many companies "feared" GPLv2.
That changed, and now GPLv2 is widely accepted. The same could happen for AGPLv3, and personally I think that the movements that advocate for Software Freedom will be closer to achieving their goals if that happened.
There's going to be less people trading on the idea of open source software while relying on the exclusivity of control of proprietary software—which in fact does not solve the problems which lead users to prefer open source software—to enable monetization, so we’ll be back to the status quo before the last handful of years where open source software was peripheral projects and supporting infrastructure funded by its users either as internal projects or via external foundations, rather than the central products of startups that have no business plan consistent with the product remaining open. But that’s okay.
1) there's a lot of software that's mostly-open-source-except-Amazon-isnt-allowed-to-run-a-hosted-version-of-it-without-paying-the-authors
2) all the software from 1) is completely proprietary
a lot more people benefit from 1 than 2. But somehow it seems that OSS purists prefer #2? It seems like they would rather less Open-Source Software in the world, all to maintain the purity of the meaning?
That's...not mostly open source. Being able to pay whomever to run a hosted version for you and not have that be exclusive to the original creators isn't, especially in the age when hosted services are in high demand, a peripheral feature of open source. It's integral to the value proposition.
If AWS/Google/Azure had to pay some fee to the original creators to provide a hosted version, they would definitely do it for a popular enough product.
Everyone would still get all the benefits, except AWS/Google/Azure get a little less money (down from some hundreds of millions) and the creators get a little more than $0.
I fail to see the negative effects on any entity there except what would amount to rounding error in AWS costs. Doesn't actually seem so integral to the value proposition of open source. Definitely seems "mostly open source".
For example, if ElasticSearch would have been closed source? People would go to Solr. If Solr closes? Postgres and a lot of swearing.
You have to remember that while AWS ElasticSearch was a bad thing for Elastic Co, it was absolutely great for all sorts of much smaller companies who did not have enough resources to run their own clusters, as Elastic's offering is 2x price of AWS one [0].
[0] https://medium.com/gigasearch/which-elasticsearch-provider-i...
(Of course, for the Linux kernel, many companies don't comply with with the license either).
They sell a hosted service that uses the software. They're not selling the software.
Pretty difficult to see how a company building a SaaS business around some software is not "for their own use".
Coupling that with the fact that, in the past, F/OSS project control or domination by a single party hasn't turned out well.
It does not prevent someone from making billions off of doing nothing but hosting your software because they are better than you at hosting.
I'm like OP, trying to take a step back. What do we want here? As users? As developers? That no one does too much (or any) money hosting our software for other people willing to pay? Profit-sharing? On what basis?
Really, naively, apart from the use of the Elastic brand, that I might conceive could cause problems, who's hurting whom?
Sure, but maybe there's a way for the original creators to get more than $0 (while still satisfying OSS people)?
Is there a way to get the benefits of OSS AND for the creators to get more then zero dollars when someone makes a shitload of money off of just putting their software on an enterprise cloud?
> Is there a way to get the benefits of OSS AND for the creators to get more then zero dollars when someone makes a shitload of money off of just putting their software on an enterprise cloud?
You're basically asking if something can be simultaneously open and not open.
The benefits come precisely because someone can come along and create a successful business using your software without owing you anything. That is the reason for the benefits.
You also have to take into account that companies use OSS to gain adoption and pull off once they're established. This is why GPL and AGPL are essential. It ensure everyone has to contribute back.
I figured if that worked, Elastic, Redis, MariaDB, Mongo, Timescale, and Cockroach Labs would have gone that route
The reason for everyone not selling the software in question to prefer F/OSS is exactly that it requires surrendering the kind of exclusive control that is also what allows you to build a company around the software as a product.
Yes, this means that F/OSS as such is unlikely to be the key product of a successful company. That's always the way it has been with F/OSS.
“But if I can’t center my business on F/OSS-as-product and I can't center my business on commercial software for the same use because F/OSS products, even if somewhat inferior themselves, develop more robust ecosystems, then how am I supposed to compete with free?” one might ask. And the answer is this: “Maybe you’re not: you are entitled to a business model.”
Yes, it is. Why the scare quotes?
Lots of things are legally allowed that would be very detrimental to me. Or you.
> Those companies can’t simultaneously claim to be competent and to have made that choice without knowing what they were doing.
I'm glad you can tell the future.
> IMO, they chased the benefits of being open source and then changed course when the costs to them exceeded the benefits (which is fine for code going forward), but trademark concerns aside, I can’t see AWS as the bad actor here.
You can at least see how the system has big problems, even if you don't want to blame Amazon for them, right?
But there are situations where it's not that things changed, with companies that would love to be open source but it would hurt them at all phases.
If you have to be super proprietary to make any money off software, that's bad for everyone.
When a piece of code is responsible for enormous amounts of money and none of that goes to the author, that's a bad way to get more code and a bad way to support valuable authors.
Those same properties of AGPL mean it’s generally viewed as not as effective at attracting a community of early users, but in a case where the alternative is proprietary-only and the company “wants” to be open-source, AGPL seems a better option for them.
I am guessing you didn't like how Amazon was profiting from ElasticSearch's work.. so would you retrospectively change ElasticSearch's license to SSPL? Or maybe help all companies: prohibit all permissive licenses (Apache, MIT, BSD etc..) entirely and force people to either AGPL, SSPL or fully commercial? Or make a law that no free software license may apply to companies with >$1 billion in revenue?
While we are at it, how about prohibiting commercial flat-rate unlimited site licenses? The huge companies can take advantage of those too.
The thing is, I bet Elastic's original decision to use Apache was to get more users, at the expense of also getting more competition. Now they have their users but they don't like the price they paid for it. This may have been a bad business decision on their part.. but I don't see "big problems" there, companies make bad business decisions all the time.
IANAL, and certainly not an expert on trademark law. Hopefully someone with more legal knowledge can provide some resources.
Even if it is just petty, it's petty against amazon, and I for one don't really feel the need to have sympathy for them.
Source: I have a best-selling book author friend who has been defending her trademark on a somewhat popular pop-culture term this way for a few decades.
Of course, Amazon has every legal and ethical right to continue providing their fork under a new name as they’re now doing. That’s the whole point of open source.
When OpenSearch was announced, I shared some insights into how both Elasticsearch and OpenSearch were evolving, and I'll share some more up to date insights here.
Looking at recent pull request activity, OpenSearch had 52 contributors
https://oss.gitsense.com/insights/github?q=pull-age%3A%3C%3D...
while Elasticsearch had 181
https://oss.gitsense.com/insights/github?q=pull-age%3A%3C%3D...
The metric that I'm most interested in, is knowing how many people committed within the last 14 days compared to those that committed more than 14 days ago. For Elasticsearch, they had 87 contributors which accounts for 68% of all contributors. OpenSearch had 20, which accounts for 67%. With these numbers, I can ball park how many people are working on Elasticsearch and OpenSearch full time and I would say Elasticsearch at the present moment has probably 5 times more people working on it fulltime vs OpenSearch.
An important thing to note is, Amazon has other projects that are related to OpenSearch so these numbers don't necessary give the full picture, but it is pretty obvious that Elasticsearch is evolving at a much faster pace and time will tell if they (OpenSearch) can keep up.
lots of software ist stable and just works. everyone uses bash. let the elastic guys make their new software more shiny, good for them. i am now glad i can use a real opensource version at home and for small to midsized projects. if opensearch won't cut it? no problem, the customer can pay elastic :)
I think the next wave of search tech will be
1. Leveraging automated ingestion optimization.
2. Capable of searching across mixed media: text, audio, images to start out with. This is only possible because of optimized ingestion. The ingestion layer will perform vectorization of the inputs. The vectors will be the entires stored in inverted indices.
3. A great jump from two to an optimized data store for searching mixed media.
People obsessing over elastic/splunk features are lost. Nobody will care in 5 years. Both will be legacy enterprise tech.
https://github.com/elastic/elasticsearch/graphs/contributors
https://github.com/opensearch-project/OpenSearch/graphs/cont...
Over an order of magnitude more development on Elasticsearch than OpenSearch since that fork
repo | commits | authors
------------+---------+---------
elastic | 2719 | 179
opensearch | 265 | 54
repo | commits | authors
------------+---------+---------
elastic | 1681 | 122
opensearch | 153 | 41
But I still think having a bunch of folks contribute isn't trivial and worth highlighting (57 is a ton for many projects) and anytime I see a truly OSS project at this scale I think it's a good thing.
For my tool, I can analyze any combination of repositories so getting the bigger picture isn't challenging. Out of curiosity, I decided to index all the opensearch repos which will take a few hours to index and I'll update this post if I can or just reply to it when the other repos have been indexed.
https://oss.gitsense.com/insights/github?q=pull-age%3A%3C%3D...
In practical terms, I now have to worry about supporting both Elastic and Opensearch as a consultant (always looking for new customers to help, if you need some advice) and I can't really recommend people stick with Elastic with a straight face either because Opensearch really is a decent alternative and increasingly the default for new users; as I have noticed with my own customers.
I maintain a Kotlin client for Elasticsearch and I have started work on making that work for both Elasticsearch 7, 8 and Opensearch; something that is complicated by the fact that Elastic intentionally made sure their Java client (which I depend on) no longer talks to opensearch. They also deprecated it and introduced a completely new one recently. And of course the recently released Elasticsearch 8 complicates things with a few new features, compatibility breaking changes, etc.
On top of that, Opensearch is starting to get its own features or alternatives to Elastic-only features. I agree that the momentum is still with Elastic but it is not the case that Opensearch is a dead fork. I know of several people that quit their jobs at Elastic because of the license change and that are now working on Opensearch or on Opensearch plugins. It will be interesting to see how the two forks evolve.
IMHO, the license change is long term a mistake for Elastic. They've cut themselves off from open source contributors who can still contribute to making Opensearch, Lucene, or Solr better. So, that's long term not going to be helpful. Elastic has always leaned on opensource heavily for contributions and hiring. A lot of their early hires came from the Lucene community and the many researchers contributing to that and later from people contributing to Elasticsearch. Additionally, a lot of value gets added to the ecosystem by OSS plugins. The authors of those now have to choose between Opensearch and Elastic. That's long term not good for Elastic as a lot of cutting edge stuff usually starts in the plugin community.
I don't necessarily like what Amazon did but I do think they did it the right way from the point of view of creating a credible alternative. And I think that the way Elastic responded to that is throwing out the baby out with the bathwater.
Perhaps the most striking statistic is that the total number of PRs for both combined declined: everybody loses. IMHO, they should just roll back the license change and grow their community rather than fragmenting and shrinking it. Amazon is going to get some of their customers either way. I actually think the fork is working out better for Amazon than it is for Elastic currently. It was a mistake. Amazon is not the most likely steward of open source. But stranger things have happened. Take MS in recent years for example. It's a sound business strategy to not piss off OSS developers. Elastic would do well to take note of that.
repo | month | commits | authors
------------+---------+---------+---------
elastic | 2022-02 | 195 | 66
elastic | 2022-01 | 269 | 68
elastic | 2021-12 | 194 | 64
elastic | 2021-11 | 233 | 59
elastic | 2021-10 | 435 | 74
elastic | 2021-09 | 318 | 64
elastic | 2021-08 | 49 | 30
opensearch | 2022-02 | 18 | 11
opensearch | 2022-01 | 34 | 17
opensearch | 2021-12 | 32 | 10
opensearch | 2021-11 | 19 | 13
opensearch | 2021-10 | 28 | 12
opensearch | 2021-09 | 20 | 10
opensearch | 2021-08 | 3 | 3
It's written in asynchronous Rust with native application speed albeit using much lower memory usage than ElasticSearch, and comes with even more features than ES, so it's feature-rich, blazing fast and can still benefit on multithreaded CPUs. Downside is that it does not have distributed indexing mode yet, but it is scheduled on this year (presumably Q4 2022 I guess)
The low end of the market is well covered with solutions that don't really offer a lot of features, are a bit challenged on the scalability front, lack usability, etc. Some of those have more merit than others of course than others. Things are not automatically better when they are half implemented in Rust. Lucene is an amazing piece of technology that over the years has resisted multiple attempts for people to do better in other languages. Contrary to the popular belief, it's actually pretty good with memory. Most of it is off heap memory or operating system file caches these days: it relies on memory mapped files for a lot of things. It's also pretty good with doing things concurrently. E.g. updating index files with 32 CPU cores while also serving queries is not an easy problem to solve. I've indexed documents at a rate of 500M/second on a 30 node cluster once. That's pretty amazing to see happening. I was basically saturating IO and CPUs. Indexed over a billion documents in about 1 hour.
Lucene has many issues; scaling isn't one of them.
I recently used Elastic Cloud, and as much as I hate the company, their product is actually really good. I’d always recommend Elastic.
Yes, Elastic’s press release is carefully crafted to give that impression, but, AFAICT, all AWS is doing is using the name of the open fork (“OpenSearch” [0]) for their service, which is now labelled “Amazon OpenSearch Service” and subheaded “(successor to Amazon Elasticsearch Service)” [1], and not using the ElasticSearch name (except as a historical reference.)
It's pretty disingenuous to say they don't open source.
Based off experience with AWS and GCP, AWS has significantly more widely available. On the other hand, Google the company has contributed a lot to OSS (like cgroups in the Linux kernel)
Edit: I see it’s called OpenSearch now.
It’s such a shame that this happened