Hacker News Reader

Comment by sailfast | Hacker News Reader

As an active developer and maintainer of a cluster, I strongly disapprove of this move by AWS.

By forking on an earlier instance of the codebase for ES and Kibana they're not only creating an open source version of the code but also attempting a fork of the community - plugin developers, user groups, etc. It's extremely frustrating. EDIT: If folks actively building plugins have felt pain here because of decisions by Elastic, then my view on this point changes considerably, obviously.

I find it very hard to believe there was no compromise to be had here between the current Elastic roadmap for their product and their previous work.

How is this good for users of the current tools? How does this help create better capabilities on a common platform? Will this encourage users to build businesses on elasticsearch open source elements?

I'm not sure where the community ends up after this, so I'm not sure I can support by using AWS' elasticsearch tools.

View on HN

As someone who is involved in the ecosystem and maintains some open-source projects based on ElasticSearch, I am similarly curious about the move to not build upon existing projects (there could be a good reason, but haven't heard it yet).

Why wasn't ElastAlert used / built upon for alerting? https://github.com/yelp/elastalert

Or why wasn't ElasticSearch SQL used / built upon for SQL? https://github.com/NLPchina/elasticsearch-sql

Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard

All of them have Apache 2.0 license.

For SearchGuard, only the basic auth is Apache 2.0. Anything practical you need to pay to use it.

Why isn't AWS paying to use it, then? It's not like they don't have the money?

there's no "they", there's only us. AWS paying to use it means we are paying to use it. Between that and a truly Apache 2.0 project, I think it's a no-brainer for everyone.

Why shouldn't you pay to use it, then?

More generally: why should AWS get to leverage its near-monopolistic position in the SaaS market to perform essentially hostile takeovers of the software projects it packages and runs? Simply because they have the resources to do it?

I'm very glad that they've done this.

The code was open, and the company decided to close it for the new versions, so it forks. Forks are common when people don't like the corporate direction of an OSS project - It's a strength, not a weakness. Openoffice/Libreoffice, Hudson/Jenkins etc.

Imagine if Linus said that the next version of Linux would be paid-only.

Within 15 minutes there'd be a new librelinux repo that people could contribute to instead.

Someone was going to make a new fork anyway - This is Amazon putting their money where their mouth is to fund and support that new fork.

To be clear, no Elasticsearch code or features that were OSS were closed. As new Elasticsearch features are developed, some of them are now released as OSS and others are released with a commercial license.

Okay, are you glad that Amazon now owns Elasticsearch?

If ES didn't do the weird license change thing, we wouldn't be here. Either Amazon wouldn't have used it, or they'd be paying. Including non-com features in the default install was not a good way to play things.

Would you feel the same if AWS built a drop in/api compatible system from scratch?

It's not just about AWS, other people need to use this as well. Why are security features a paid for addition? Why not make core features a paid for addition instead? I've always hated Elastic (the company) for this decision, please stop trying to give away free things that are not secure, and then charge for security.

I'd recommend browsing the code to find out more about the security module:

https://github.com/opendistro-for-elasticsearch/security/blo...

From a look at the source code, Amazon has Apached 2.0 the entire Floragunn Searchguard codebase (including the commercial part)

> why wasn't ElasticSearch SQL used / built upon for SQL?

According to the release blogpost [0], "SQL Support [...]is an improved version of the elasticsearch-sql plugin."

Sounds like they are building on it, as you suggested.

[0] https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic....

> Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard

Looks like it was. https://github.com/opendistro-for-elasticsearch/security-ssl...

Beyond the thoughtful ideas below, an official blog post would be cool for that. Specifically, Elastalert and ODfE Alerting are completely different.

From the pom.xml of Open Distro for ElasticSearch: the fork is from elasticsearch-6.5.4, which is the latest stable release. Only alternative would have been v6.6.1. EDIT: also, nothing is there to believe they would not bump to higher versions in later versions.

6.6.1 is the latest stable release. 6.5.4 was released 19 december 2018 [1], so they’re not too far behind, though.

[1] https://www.elastic.co/blog/elastic-stack-6-5-4-released

Thanks for this clarification!

From the related AWS blog (so at least a few grains of salt):

>>This means that the majority of new Elasticsearch users are now, in fact, running proprietary software. We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch. They have made it clear that they intend to continue on their current path.

... maybe I'm misreading your comment, but it sounds like AWS tried to find a compromise, including contributing to upstream, and were shut down (and all their content about future contributing seems to say "we'll send it upstream, but it depends on whether Elastic accepts it," so seems to corroborate). It honestly seems like you get more certainty that everything you're using from AWS is open and will remain open, and Elastic is going to keep creating uncertainty / drive to proprietary source-available.

There is certainty around the open-source bits and the license - this is true. That said elastic the company has also seems to have brought the product a long way after becoming a company. Having a dedicated staff of engineers and developers behind a thing is helpful for that, and they should be able to make some money as part of this if we want to see future open source products advance similarly. (There is a difference too between consulting / services and money “at scale” - AWS is removing an at-scale component here)

That said, I also totally understand disagreements with Elastic’s decisions on charging for security, etc and why that causes concern. As a user I wish I wasn’t staring a fork in the face and I’m hopeful things get back to equilibrium here soon as a single community.

Reply on news.ycombinator.com