This is not a good outcome for people who want an independent OSS ecosystem.
If you don't believe me, this is the line in the code where they 'fork' in their build script: https://github.com/opendistro-for-elasticsearch/opendistro-b...
It will be interesting to see if opendistro actually starts contributing to Lucene or adding value to the elastic code base or whether this 'fork' is effectively frozen in time and doomed to go nowhere due to a lack of actual development on the core product.
I seriously doubt that they are going to put any effort whatsoever in that. Because in two years of pretending they have a fork, there has been no serious code contributions from their side at all that I'm aware off. Correct me if I'm wrong.
If you want to back an OSS version of related technology, maybe use something like Apache Solr, which continues to provide many of the same features and is part of the same ecosystem of users and developers that work on Lucene, which also powers Elasticsearch and which of course continues to contribute a lot to Lucene; unlike Amazon.
Some information about Amazon's investments in upstream Apache Lucene can be found here: https://aws.amazon.com/blogs/opensource/amazon-giving-back-a...
Covered in the 2019 blog post are:
* Concurrent updates and deletes
* Indexing custom term frequencies
* Lazy loading of Finite State TransducersPersonally, I don’t think there’s anything immoral or shameful about selling proprietary software or shared source or whatever. However, SSPL and TSL and friends are just not open source licenses. They’re more restrictive than AGPL. They are designed to protect a business model, not your rights as an end user.
So it irks me that the only perspective I see reliably is that Amazon is screwing everyone and not how startups are willing to take advantage of the good will and marketing bonus and then switch it up when they realize it can’t be reconciled with their business model. Continuing to call it “open source” is just insult to injury, imo.
I'd love to have a license that grants FOSS rights to people but not to corporations. There are end users and then there are corporations monetizing FOSS and giving nothing back. On top of that the monetization is via SaaS which is a more closed computing model than commercial software. With SaaS you don't even have the data and everything you do is subject to total surveillance.
And then it got torpedoed...
Amazon dominates conversations because they extract the most value from open source projects. They make more money selling PostgreSQL, MySQL, Elasticsearch, _Docker_, Linux, etc than the creators of those projects. It's not the only perspective, but it dominates for a reason.
Startups OSS their work because it helps them get users and figure out what to build. Amazon has the benefit of watching which projects become most viable (frequently because the startup threatens their revenue somehow) and packaging them up as their own services.
I want to live in a world where smaller startups _can_ OSS their work and also monetize it. And I'd like them to be able to do it without raising $100mm in VC.
this goes to the question of history, are things structurally so determined that if you remove someone or something from history another would rise to take their place. If Amazon had not existed to take advantage of open source like this would someone arise to do it? I have to think yes, it would be bound to happen eventually, it's a weakness in the model and would be exploited by someone, and when it was exploited people would respond by doing sort of like what is happening now.
If Amazon was not here, someone would be making the history of open source exploitation rhyme.
They make more money selling EC2, with this software installed on top of it. If Amazon did not sell RDS, their customers would buy EC2 instances and install it on there.
Which of these realities would you prefer:
1. Mongo, ES, PG, etc have a cloud offering, which runs on metal they maintain. Naturally, the reliability and performance is worse. Amazon loses some money. Customers pay more for a worse product.
2. "Open source has failed, there's no way to monetize it", so we get DynamoDB and Aurora and S3. Amazon wins, Customers (mostly) win (unless Amazon takes the gun they've now loaded and decides to point it at them)
3. "Its open source, but you still have to buy it" lol ok, all the negatives of closed source with few of the positives of open source. No one wants the source code so they can read it and hack on it, they want it only as a signal for their ability to move hosting providers should they need to, or self-host. Source code usually lasts longer than the people who wrote it.
4. "We'll sell support packages" so now your revenue stream isn't aligned with customer success, because you're financially incentivized to build a product which needs support. Nope.
5. AWS does not have RDS, but does have EC2. Mongo, ES, PG, etc have their own "cloud" offerings, which are really just re-vendored EC2. They tack on a 30% upcharge so they can survive and develop the product. Well, that's what both ES and Mongo do, today. AWS loves this. They make the same amount of money from selling cheaper metal without all the expensive upkeep of RDS. Its the customers who hurt, not Amazon. You miss out on integration points in the cloud platform. You miss out on centralized IAM. You pay more for a worse product.
People keep acting like AWS is a big bully who got big for no reason, it doesn't make sense, the cloud is a monopoly and AWS is strong-arming other companies to get bigger. Maybe to some small degree, but by far the reason why AWS is big is because They Build What Customers Want. End of story. There's no philosophical debate between AWS Product Development and their Customers concerning whether they should build an open source ES. There's just "Jeeze, Elastic has been really screwing us over lately, is there anything you can d--" "Say no more, we've got you, by the way your two hundred thousand dollar bill is due." "Wow, thanks Amazon, no problem we'll pay that right away."
The real thing that's killing these database companies is that databases are a dime-a-dozen. You pick one, who cares which one it is. There's some minutia here and there, do you want SQL, NoSQL, time series, ok you get past that and the rest falls into place. I've sat in the decision making room on this for two or three companies. No one analyzes what the database can do. The discussions surround "where are we hosting it". What options does Azure have, AWS, oh that company has their own hosting platform, etc etc. HackerNews makes a big fuss about how MongoDB has no transactions; I've literally NEVER heard this brought up once as a negative against the product (though, sure, in some very specific industries or use-cases it matters). Version 4.whatever adds a new query operator; who gives a fuck, 99% of database usage is "write thing, read thing back, query for things in a way that SQL could do in 1992, update thing, delete thing." Databases are boring, and the success of the companies who tried to build billion dollar businesses on top of one (Oracle as well) reflects how boring the software actually is.
What these new quasi-open source (ie source available) companies want to do leverage their authorship of the software to be sole support service provider (of meaningful size)... which is exactly the sole value the enterprises extract from opensource: the lack of vendor leverage!
The market does not tolerate a contradiction.
I really don't know why they get so much defense in the community; I view their actions as a straight betrayal. The one thing you can say about AWS is, they're the enemy you know. They don't have a history of turning their back on OSS, but they also don't have a history of abusing OSS to gain traction for their product (though, the MySQL/Aurora relationship could reasonably fall under this. I think its different; not a betrayal because they haven't turned their backs on MySQL, but are just trying to make it better for their customers in ways MySQL alone can't do). With AWS, you know what you get; they care about the tech and the money; no politics (well, almost), no weird philosophies guiding decision making, just solving problems and getting paid to solve those problems. That's infrastructure, baby.
Its a lot hazier with, say, MongoDB. You wouldn't believe the pressure they've put on customers on 3.6; we're talking weekly emails, even a few cold calls, from sales reps. "Hey, I know its the holidays, but have you taken a look at 4.0 yet? Let me send you a link to check it out, lots of cool stuff." Yeah; the real reason they want you to upgrade is because 4.0 has a bunch of new features that competitors like AWS can't replicate due to its new licensing. I'd love to hear from someone running MySQL on RDS, and if they've had a similar experience with AWS sales concerning Aurora; I suspect they haven't. Frankly, AWS never talks to us and we never talk to them, unless there's a problem or we want a discount, and they're always accommodating for both. The perfect relationship.
Therein lies the problem: Startups esp the ones that need to grow really fast and at all costs choose to permissively give away their proprietary advantage but still want to "capture most value" from the market they help create are making it unnecessarily hard for themselves by going down the F/OSS route just to "get users and figure out what to build". It is a losing strategy (you'd agree?) because... if the startup's F/OSS product is any popular and lots of users do use it, it is only inviting the likes of Amazon (and anyone with the chops to build a cloud offering, really) to package it as SaaS / PaaS around it.
I'd like to see startups experiment SSPL / CCPL / Commons Clause licensing from day zero (like materialize.io) to see if they can still attract users or build mind share. I'd hate for them to instead release it under F/OSS licenses (and CLA all contributions) to only conveniently turn hostile with super-strong copylefts like SSPL or non-Libre licenses like Commons Clause and BSL.
I would rather give the money to the people who make the stuff than to people who (all but) steal the stuff and sell it to others.
And it usually comes out cheaper, too!
Compare to Postgres. Postgres will never die. Aurora is gaining serious traction; do you think the Postgres developers are concerned? Why would they be? Their livelihood, and their ability to develop Postgres, does not depend on it being the #1 database in the world.
Naturally, ODfES has a tether to Amazon. Amazon could decide to lock more features behind AWS ESS. Its not a perfect situation, but it is better than the one ES has been in for the past few years.
Postgres had a huge head start here, and was built in a different world WITHOUT an Amazon equivalent.
Are there a lot of major contributions to (not just on top of) ODfES outside of Amazon? If not, I'd be fairly worried about using it in a non-AWS-provided way in the long term, now.
But there is one more factor you failed to acknowledge that is driving them to make more money: Elastic.co took VC funding.
Once you accept VC funding, it narrows the definition of success down to what your investors think success should look, like rather than what the founders desire.
I used to manage a licensed ELK cluster. The licensing was expensive (order of six figures) and was really only necessary for LDAP integration. Features like timelion were cool, but definitely not worth the price tag.
I migrated to an AWS-managed solution to for two reasons: 1) to alleviate the burden of managing a cluster, 2) get away from the onerous costs associated with licensing. Even thought it was more expensive to run a managed cluster, since, at the time, AWS limited the disk space per node. The cost savings was significant enough to warrant migration.
I realize that Elastic.co offers their own managed solution. But the version on AWS was an easier sell to management due to known costs and not having to deal with sales people.
I think a lot of OSS is having difficulty transitioning to a cloud-first model. The people making choices on tech stacks want to spend less time dealing with infrastructure and more time developing stuff. And this model of charging high licensing fees for "enterprise" features feels antiquated, and provides a large attack service for companies like AWS. What enterprise engineers really want are AMIs, or better yet, fully managed cloud interfaces, to bring up and manage OSS deployments. Something that lets us focus on the products we sell; not the ones we buy.
I believe that we need to rethink how OSS is delivered and how to make money from it. "Enterprise" support looks a lot different not than it did when Redhat was founded, yet the OSS business model remains eerily similar.
I mean, I still buy stuff from Amazon, use Gmail, etc.
We definitely need to rethink how OSS is delivered, but part of rethinking that is understanding how dominant AWS over OSS revenue.
This aspect does not get nearly the attention it deserves.
Unless your product is an exceptionally well focused developer tool, it will take months, and sometime multiple quarters, to see whether it adds any real value. Exceptions exist but are rare. In practice the generous[TM] 30-day trial period is almost always worthless, and often a gateway to sustained net negative.
Add to that the constant barrage of dark patterny attempts to infest your calendar and the upsell pressure tactics. Every third-party integration is an additional liability for maintenance, and far too frequently an increase in attack surface. Time is your most precious asset and the sales people are trying to rob you of it.
To every sales person on the planet: if you want to sell me a service or product, then sell me NOTHING BUT the service or product. Make useful documentation accessible, and make additional features discoverable, with unambiguous pricing clearly in place where necessary. Then get out of my way.
With OSS product I can install and trial it as long as I need, without needing to give off my contact details or be added to your invasive CRM. If I need additional features that are only available under your more restrictive license offering, I will buy them when I have the need. Your sales quota or bonus thresholds do not factor in, and if you try make them my problem, I will look at migration options.
[1] https://www.elastic.co/blog/announcing-support-for-elastic-c...
Edit: added source
I understand why this is not ideal for Elastic-the-company, but is it a bad outcome in the longer term?
As someone who is happy open source exists but uses mostly proprietary software, this seems like a natural and expected response to an open source project adopting a more restrictive license.
A fork is a natural response to an OSS project license change. But when the company that _forces_ the license change also creates the fork, it's gross.