Doubling down on permissive licensing and the Elasticsearch lockdown
crate.io
crate.io
The only thing I haven't seen said succinctly, although it's been hinted at by many, is this:
There is no one on the planet yet who has agreed that they will run a project under the SS Public License and be themselves bound by the SS Public License.
That really says it all. Whatever your view about copyleft licenses generally: legitimate and well-intentioned copyleft licenses (e.g., CC-BY-SA, GPL, and Affero GPL) have many projects that use the license in an inbound=outbound contributor licensing fashion. All (two of the) SS Public License uses have a strict CLA that give the publishing entity non-SS-Public-Licensed rights to the contributions. We shouldn't take anyone seriously who promulgates a license but won't use it themselves in an inbound=outbound way. Period. I suggest we all try to not give further interest to this clearly risible licensing proposals from MongoDB and Elastic.
If there's energy to focus here, I'd say it's toward figuring out how to handle good governance of forks of these two projects. I hope folks can maintain the discipline to discern and bifricate these two very different issues.
I would just treat them as proprietary. With access to code and opened gate to provide also code, if you are paying someone and they still do not care about your bug.
What may be better than fully proprietary and unavailable source.
I would not dismiss for example Windows just because they are not open source.
GPL and AGPL are just as much used to "poison the well" for potential competitors to a product also released as open source, and dual licensing strategies are a well-known part of open source businesses. This hasn't changed with SSPL at all, there's just a lot of pearl-clutching about the fact that the high and mighty OSI has not signed off on it.
But it is also legitimately outside of the purpose as well... unlike the SSPL. The Linux kernel for example. No one has ever looked at the SSPL and said “I want to work on that project” unlike the the GPL, which I believe is the GP’s point.
> There is no one on the planet yet who has agreed that they will run a project under the SS Public License and be themselves bound by the SS Public License.
RMS would not be upset that the GPL poisons the well because companies don't want to open source their code, RMS would say they are denying user freedoms by failing to open source their code. The SSPL merely builds on this.
The only real sticking point here seems to be the OSI, and probably some manner of lobbying by tech firms to keep the SSPL from being accepted more widely as an open source license.
Comments of this sort was comme il faut in the 90s. But now, more than two decades later, GPL has proven its commercial value so many times over it's hard to take such arguments seriously.
Tit-for-tat business models grow faster than those staking out islands. Linux won. We need to move on from this.
Amazon could use SSPL-licensed Elasticsearch on AWS, if they wanted to open source AWS, and they'd probably still make bank on it. But they don't want to, so hence going to SSPL, like GPL, discourages their use.
Doesn't particularly specify what type of business you have to be running, though it's obviously partially related to whatever type of software you licensed under the SSPL.
The GPL affects projects which embed GPL code into them, and the SSPL effectively does the same, it just draws the borders wider, to anything used to provide it as a service to others. In a copyleft sense, that promotes providers of it to also contribute to the ecosystem of tools around it.
I will happily contribute to a permissively-licensed code base for a project, although I'm aware that my contributions might be used in some closed-source platform later on. Exchanging my contributions for the ability to do whatever I want with the software is a reasonable trade.
I'm also willing to contribute to projects using copyleft licenses, on the basis that a project that requires all contributors to leave all of their contributions and derivatives of the product in the open offers a fair playing field.
I am way less interested in contributing to this new breed of semi-open projects, which require me to sign over my contributions in exchange for access under a license which stops me from using the project in certain ways, but grants one entity special privileges to use it in that way.
I think the evidence suggests that both GPL and permissively-licensed projects can be wildly successful. There are good reasons for both approaches, with different benefits and trade-offs. It's also legitimate to be irritated at "bad neighbours" who take without contributing back. I'm not convinced that this semi-open licensing approach is a benefit to anyone.
It may be better than fully closed source? Just that it should be self-described as proprietary, with source available.
If your business model cannot survive when a critical upstream piece of your infrastructure moves to GPL, you probably have a bad business model to begin with.
They say:
> We would never have chosen Elasticsearch in the first place, had it been licensed under the GPL as some of our customers (and many large enterprises do by default) banned GPL licensed software from their application stacks for legal risks.
Which is completely new to me. There are numerous examples of software that has a successfull business model, and is GPL or GPL compatible. From WordPress to MySQL, and from ProtonMail to PostgreSQL (GPL compatible). Used in enterprise, and other large organisations, just fine.
It sounds like they are making up excuses for not wanting to fully Open Source their code; which is fine. But don't blame upstream for your reluctance or inability to adhere to this.
To be clear CrateDB started out as OSS and we decide to stay OSS. Elasticsearch used the Apache License and so did CrateDB. All in the spirit of OSS. Elastic are however now the ones how decided, that their business model isn't viable anymore.
> It sounds like they are making up excuses for not wanting to fully Open Source their code
We do want to make it fully open source! Everything that was under a more restrictive License is going to be offered under Apache License.
> We do want to make it fully open source!
This begs the question: isn't "a restrictive OSS licence" not less "fully open source" than a more permissive licence like GPL, MIT or BSD?
If you are fully committed to OSS, why not go full-oss, instead of retaining control through a restrictive OSS licence?
Is that really only because of some enterprises not liking GPL?
We gonna change CrateDB fully to Apache License v2 ;) I would say that counts as a "more permissive" license.
> Is that really only because of some enterprises not liking GPL?
There are various reasons for the change. A big part is definitely also the spirit of many our contributors. We built CrateDB on open source software and also want to make the software available as open source. It also was planned for quite some time to be more open.
The main point of copyleft is to pass down freedoms to use/modify/distribute all the way to the end user.
Instead we got locked-down privacy-breaching smartphones, IoT devices, SaaS, where only the manufacturer benefits from OSS.
• Unix-ish black-box "primitive" tools, that were so focused on accomplishing one fundamental job that they were essentially "final" in their interfaces, with there being no point to extending them any further; where you reused them by executing them, not by integrating with them.
• A library for such Unix-ish black-box tools to use. Most tools that used any libraries at all, would use one main library to accomplish their one primary purpose, effectively making the tool a "driver program" for the library.
• Academic data-science/statistics code.
• Cathedral-style highly-integrated software, e.g. Windows.
Copyleft was mostly devised for the purpose of licensing the codebases of the first two types.
As Unix tools are self-contained, they only "infect" direct forks. The GPL originally intentionally avoided infecting the things that called (i.e. interacted with) those tools — because, back then, a downstream project that "uses" a tool wasn't vendoring in its own version, but rather relying on the system installation of that tool, through that tool's known API; and it wouldn't make sense for a license to be infectious through a standardized API.
It was intentional that libraries would infect their downstream clients with copyleft; but downstream clients, back then, were mostly just those single-purpose tools. It wouldn't make sense for e.g. libgit to be GPL-licensed, but for git(1) to be proprietary.
Of course, there was also an awareness that the Cathedral-style codebases would have their whole monolith infected if they used the GPLed library. The idea there, though, wasn't to actually cause that infection — it was to inhibit Cathedral-style codebases from using GPLed software at all.
(With the parallel awareness that such entities could always reach out to the project maintainers, and buy a separate license, just like you can purchase a license from any IP-holder. There were few-enough contributors per project, back then, that "copyright assignment" and such wasn't a concern; you could just get a proprietary license from the one dude who built the whole thing.)
And that same consideration was implicit with academic use of GPLed software. FOSS programmers, back then, considered academic (or non-profit) integration/derivation of their software to be something they'd grant a free license for if asked; and academics knew this, and so didn't bother to ask for such licenses, because they knew they'd almost assuredly get one, for free, if-and-when it ever became important to do so.
---
The GPL was well-suited to this early-90s software IP ecosystem. It doesn't fit nearly as well in the modern software IP ecosystem.
There's a whole fifth category of software — semi-Cathedral, semi-Bazaar mega-tools, like youtube-dl or Calibre; or mega-libraries, like Qt, or LLVM, or WebKit; which both are components while also consuming many components themselves. The GPL never "expected" this type of software. This kind of software just didn't exist back then; only its entirely-Cathedal final-product equivalent did.
Which is a problem, because it's impossible to build something like WebKit or LLVM in a self-contained, "you call it over a standard interface" sort of way, where it's non-infectious. These days, lot more projects are infectious, even when "integrated" at a much higher level of abstraction, than the GPL was ever intended to require.
LGPL existed since 1991. Also large libraries and frameworks.
Yes, you can create different implementations of that concept, that are variously infectious. But the reason I laid out the whole state of the ecosystem as RMS would have seen it when he was still just conceptualizing copyleft, is that in that ecosystem, "infectiousness" was something that's almost trivial, toothless.
In the ecosystem of the early 90s, licensing a codebase was just a consideration of who you trusted to freely use and modify your thing ("us", hackers); vs. who you wanted to not use your thing, unless they paid you ("them", corporate.) Copyleft neatly prevented "them" from swiping and profiting off of the software created by "us", while not really inhibiting anything that "us hackers" wanted to do with that same software.
Contrast to the ecosystem of today: there's an entire category of people — individuals who start projects as hackers or academics, but then build huge software businesses around them — that didn't even exist back in the 90s.
Google is the epitome of this: at its inception, BackRub (Google Search) was exactly the type of project that copyleft was designed to avoid restricting. But it evolved, through commercialization, patenting, SaaS-ification, and scale, into exactly the type of project that copyleft wants to "shun out of" the FOSS ecosystem. (Not that Google Search integrated any FOSS libraries; just that it could have.)
Google's story, is the story of most software projects today. Every developer is considering their project as a potential "open core" for a SaaS, or considering having an "enterprise version" of their tool, or considering licensing their algorithm as a plugin to some big studio to redistribute. Which is exactly why many programmers avoid integrating copyleft software into even their hobby projects. Why build on GCC when you can build on LLVM, and ensure that there'll be no legal problem
Modern copyleft licenses are ever-more-strained legal contortions to make a design that's no longer very applicable to the modern software IP ecosystem, work for it anyway. They're licenses with epicycles.
Sure, I can in fact link AGPLed and LGPLed system libraries in my language runtime; and in a pinch — if there's no equally-good alternative — I'll take the time, work out the precise legal implications, and go ahead with it.
But if there's a BSD or MIT-licensed (or even Apache-licensed) alternative to those libraries? I'll choose that one. Because, in the modern landscape, by doing so, I'm saving myself, my future self, and my future hypothetical SaaS's future hypothetical lawyers, a lot of time and effort.
I'm surprised at the notion that RMS ever had the intention of inhibiting GPL use in Cathedral-style codebases. And I agree with others that big frameworks existed back then also.
1. literally "frameworks" in the inversion-of-control sense — where your code is a script that runs "inside" the framework — and you don't ship the framework to your customers as part of your product, but rather walk them through getting it from its own vendor as a prerequisite step to installing your product (e.g. TeX); or
2. proprietary, not copyleft-licensed (e.g. game-console SDKs.)
If anyone has a good counter-example, I'm all ears :)
You forget the user.
How does the user benefit from having a phone built on top of open source software, that they cannot update a well known security vulnerability because the manufacturer can't bother to run a build with the last upstream version?
Well, depending on the incentives and restrictions involved, an ecosystem of 3rd-party builds is a potentially viable escape hatch for the user from the manufacturer's grip.
Of course the sticking point is the degree to which the hardware requires proprietary and opaque binary blobs in order to enable important user-facing features. But then, that isn't anything really new, as open source PC operating systems have been dealing with this issue since forever, with the caveat that PC hardware is mostly modular, so having or swapping in well-supported components is an option, whereas smartphones are an integrated slab of metal, plastic, and glass, with "no user serviceable parts inside" as the status quo.
But even that caveat has precedents, in non-PC devices such as consumer networking gear that only became well supported through aggressive GPL license enforcement actions that freed some of the necessary code.
Someone said companies use GPL software, add their business logic or drivers, and never contribute back: e.g. android phones.
Then someone else said the insane thing that the user benefits.
I pointed out that if there is a security flaw, you CANNOT build/path because you do not have all the source (e.g. alternative android OS cannot use the camera or radio for lack of kernel drivers)
Also the SSPL is so vague, that we probably would not only have to release CrateDB itself - which we already do, but also everything we use for the services we provide. Also we could never make any kind of deals with OEMs, etc.
See section and links for "Why not the GNU General Public License?"
It's not GPL for some of the same reasons described here and enterprise does like it.
In short though - was developed at berkeley so released under a BSD license. The license has served them well (there are a lot of commercial entities supporting postgresql development, and lots of others with various forks). And finally, it would be hard to change at this point and they don't see a gain.
A note - in contrast to Elastic, there is no one copyright holder with postgresql. So this makes it much hard to get a relicense to for example a situation where one commercial entity has a special right to the terms / relicensing power.
(please note that GPL doesn't in actuality force anyone to back-contribute but in effect it tends to work that way - it actually puts a lot of responsibility on the original authors to re-acquire improved versions of the library but it does ensure that those improved versions are accessible in some manner)
They are changing the license on ALL of their source code. How is this making up excuses?
Second, I know plenty of companies where MySQL or Wordpress are not allowed to be used because they are GPL.
PostgreSQL is NOT GPL and thus is allowed.
The fact that some companies have incompetent legal teams isn't a good argument against the GPL. There's no reason you can't use Wordpress to back a commercial website.
There are concerns that doing so, in the context of a monorepo, might infect your entire codebase — including the components that have nothing to do with the control-panel, and so never touch the Wordpress stuff — with GPL.
Of course, you can architect code to get around that, e.g. building your shared architecture that the control-panel uses as a library, and then pulling that lib in from your isolated Wordpress control-panel codebase. But enterprises don't want to be forced into (possibly sub-optimal) architectures just because one component legally requires it. They'd rather just not use that component, so that their architecture can be totally determined by what's best for their development.
I realize this is an ideological position (and not necessarily a popular one on this board) but that's the point of the GPL. If certain companies lose out because they're afraid to share, then they have no one to blame but themselves.
The pitch is that the world of free software can, collectively, build better software for everyone sharing, than the proprietary islands who selfishly cut themselves off from that ocean.
At a previous company I worked at legal considered it cheaper to reinvent the wheel if necessary to avoid GPL software than to potentially have to deal with a lawsuit around GPL being used in our software.
Even LGPL software was considered off-limits, because one wrong linker flag and now it's statically linked into the resulting binary...
Then, to get compliance, you just have to fix the linker flag. Providing the source code is not the only way to get compliance.
I think they appear to be surviving just fine - they are switching forks not shuttering their doors or even withdrawing ES compatibility.
Suddenly changing licenses to increase how restrictive it is absolutely is a nightmare scenario to a lot of businesses and it quite rationally is so. We share software and build on each other's shoulders, to reject that and build everything from scratch is silly and fiscally irresponsible.
I'm very confused why this is so strongly stated when it's likely that most of us have, at one point, sat down with two pieces of libraries and gone with the less useful one due to licensing issues.
So, they don't run Linux, don't use glibc? That can't be all that common? (I mean sure, there's the bsds.. But still..).
We do run Linux :)
But there is a difference between building on and building with.
> Which is completely new to me
I work in a consultancy for enterprise-scale customers - every.single.one prohibits the use of GPL libraries and frameworks.
As for MySQL, notice how no one builds anything on top of it (like TimescaleDB and others did with PostgreSQL), but just uses MySQL as-is without modifying it, also rendering copy-left moot.
Personally I am happy about their decisions, as it stops the bleeding caused by big monopolistic corporations while protecting the freedom of users like me.
Even without that, how many would have adopted it without it's permissive licenses? Probably none of the major cloud offerings, which would have severely limited growth given the continuing sig ificant shift from on-prem to cloud during that period of time.
Counterfactuals like this are difficult to answer. However, more permissively licensed software does seem to reach widespread adoption more easily. Elastic's download rates seem to have approached hockey stick growth trend around the time of AWS adoption, and Elastic's revenue significantly increased afterwards as well. It might have been on track for that anyway, but 5 years of linear growth did turn sharply upwards around AWS.
It seems like Elastic as a company has a decent probability of faster growth and more revenue due to market penetration and name recognition assisted by AWS usage.
Had Elastic been restrictive.from the start, it's entirely possible that the industry would have focussed on and driven improvements in something like solr instead. As you said, elastic solved problems people had, it's not unreasonable to think they would have solved them some other way without elastic.
Which seems to negate the core argument Elastic is giving for the change.
I'm hardly a major contributor but I have provided some fixes over the years. This license is a bit of a deal breaker for me and I'm not likely to volunteer more time on this. I charge premium rates for commercial software development. I'm sure most bigger companies would also not be very eager to put any time and effort into this code base under this license where in the past this would have been less controversial with e.g. legal departments doing due diligence.
No amount of "clarifications" is going to fix that. It's not a problem of people stubbornly misunderstanding: it's people disagreeing with them. They'll be dealing with negative press and FUD for years to come around essentially any press release, announcement, or other bit of news in the foreseeable future. IMHO it's actually bad for business. And it seems the stock price is trending down the last few days; so maybe it's not just me. Not to late to correct this mistake, just saying. I assume share holder value was the driving force here. Not served by any of this apparently.
As for forks, I don't think Amazon has the right people currently to do anything more than very minor/trivial updates. I doubt that they will fix this by putting a large product team together to actually try to innovate. That would involve poaching people from Elastic or other projects. Easy to predict, because so far they haven't in the two years of opendistro having been around. I don't know about crateDB but I think they are a fair bit smaller than Amazon; so it's doubtful they can keep up much. I guess for them, the key thing is actually keeping up with Lucene updates.
SSPL is still open source, it's just aggressively copyleft. (Note that Crate is upset about them going GPL, not going SSPL.)
As for the question, depends how much their code diverges... Shoudln't be any reason Elastic can't continue to absorb contributions made to Amazon's Apache-licensed fork, until they move too far apart for the code to be relevant.
Crate is upset because they've created a significant derivative work which complicates their business under the new license. Apache 2.0 is intentionally designed to support this kind of thing; which is why it is a popular license for companies working together on OSS projects.
Copyright transfers like Elastic is practicing are of course a big risk for such companies. IMHO, any such projects has long term challenges. Most long lived OSS projects have a multitude of copyright holders. E.g. re-licensing Linux would be very impractical as it as many thousands of contributors some of whom have passed away (i.e. their families inherited the copyright). That's a strength of that project; it's forever stuck on GPL2, which provides enough wiggle room that the likes of Google, Samsung, and many others can use it to create software for their products.
Amazon's work is of course less significant but this license is designed to prevent derivative works from being likely to succeed commercially. Even Amazon's use which technically did not involve actually modifying the source code (not a single line, they used a binary OSS release provided by Elastic and added plugins) would be illegal under this license. That's not open; sorry.
I want to clarify one aspect here. Elasticsearch and Kibana will be dual licensed under the Elastic License or the SSPL license, not only SSPL. The distributions that we provide will be licensed under the Elastic License, which does not have the copyleft requirements that are sometimes concerning to legal teams.
That is, if you download our distribution and run it in your infrastructure, you are subject to the Elastic License. The same license that most (90%+) of our users are already running under today, when they download our default distribution from elastic.co. This is why we say the vast majority of our users are not impacted by this change.
Note: we are considering making changes to the Elastic License to simplify it. Please see more on our [blog][0].
Disclaimer: I am on the Elasticsearch team and work for Elastic; I welcome any and all feedback.
[0]: https://www.elastic.co/blog/license-change-clarification
The fact that most users use the basic license may well be explained by two things:
A) Your website offers it as the primary download.
B) Elasticsearch without the basic licensed modules lacks severely when it comes to security features. It doesn’t even offer basic auth, let alone TLS or similar.
The Elastic license is also a change from Apache 2.0, and is also not an OSI approved license as far as I am aware.
Doesn't the elastic license specifically limit use of software to basic features, prohibit modification of licensing control mechanisms etc. I haven't dug through it, but it seemed FAR FAR different than a normal open source license.
One question I had - is the Elastic license transferable? Ie, if you run a startup and are bought out with an asset buyout, can you transfer the stack including the Elastic licensed software to the acquiring party (assuming in the interim elastic has ceased to offer new elastic licenses so they can't get their own). Can you sell software built out on elastic licensed code and transfer the elastic license to the users so they can also use it? Or does everyone need to go back to elastic to get these licenses.
A lot of discussion about being open, but reading the details - seems far from open at first glance.
And for open-source users things definitively do not "remain as they are", given even your blog post admits it's not an open-source license.
But well, not particularly surprising you don't consider such users as "your users". Disappointing, but not surprising. Good reminder to check the CLAs required by any project I consider using.
Philipp from Elastic.
1. elastic.co
2. Products
3. Elasticsearch
4. Ignore three buttons, click the link "Or download and get started"
5. Ignore six buttons, click the link "Visit our downloads page"
6. Ignore twenty other links, find "The pure Apache 2.0 licensed distribution is available here." six paragraphs into the 'Notes' section.
7. Download it.
There's your any and all feedback, I'm looking forward to being let know _how_ welcome it was.
This is the most important statement, yet they don't explain why it kills their business model
Also, did I miss something in the SSPL wording? is the consensus really that SSPL equates to GPL? I can see the similarity to GPLv3 in the sense that you have to provide a lot of the environment on which you run the software,but the article just equates SSPL to GPL after the first paragraph, which is a pity imo.
> We would never have chosen Elasticsearch in the first place, had it been licensed under the GPL as some of our customers (and many large enterprises do by default) banned GPL licensed software from their application stacks for legal risks
You should feel fine using GPLv2 in your business, if your are really opensource company (if there is such a thing.) That's really an opinionated statement by someone who doesn't even have business, but please comment; why would they care if they're the ones making the FOSS plugins and will only see changes going back upstream.
Announcement from 2018:
> "We did not change the licence of any of the Apache 2.0 code of Elasticsearch [...] and we never will"
Edited announcement in 2021:
> "We did not change the licence of any of the Apache 2.0 code of Elasticsearch [...] and we never will¹"
> ¹ [...] we announced a further change to our licensing strategy.
1. https://anonymoushash.vmbrasseur.com/2021/01/14/elasticsearc...
They are free to use the contributions of everyone for any purpose they wish, but everyone else is curtailed and limited.
It’ll be interesting to see if their “eat the cake and have it too” move works. I hope not.
Reminds me of Trump trying to steal the election by accusing the other side of stealing the election. But in that case it was at least easier to find out who was lying...