Amazon’s quiet open source revolution
infoworld.com
infoworld.com
Edit: Also doesn't AWS have a history of Microsoft style EEE, where they embrace an open source project, make a service from it, extend that service in a non-open way and then push people to use their extended non-open service instead of improving upstream?
Hasn't this resulted in a number of projects changing their licensing to prevent it?
https://camo.githubusercontent.com/71f7e9147092c7d4b08df18da...
Worth bearing in mind that the only reason fork even exists was so that they didnt have to pay Elastic a cut when selling hosted ElasticSearch.
If they cared about open source they could start by not undermining the business models of companies that do more than just pretend to support it.
(Not to mention lying about "partnering" with them)
If allowed to continue this would kill the project. If only Amazon can monetize it, it doesnt get developed. The open source project then dies.
Amazon is no more doing the right thing here than they were when they destroyed diapers.com with a strategy of sustained predatory pricing.
The fact that they want a pat on the back for being "pro open source" now after pulling this kind of crap is just icing on the cake.
Notably, you could build GPL software on top of OpenSearch. You could not do so on top of ElasticSearch at the moment.
So, Elastic then chose to change their license to try and counter it.
> Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
Especially so, it seems, if those restrictions promote the growth of open source.
Elastic changed to a license that is completely open except insofar as it blocked Amazon from profiting from it.
>Notably, you could build GPL software on top of OpenSearch. You could not do so on top of ElasticSearch.
Notably you can't build MIT software based on GPL software either. This is 100% about MIT being a weak open source license with few restrictions and GPL being a strong open source license with restrictions.
Not coincidentally the very-much-also-open-source-license GPL also had a history of infuriating big tech for exactly the same reason - they hate restrictions put in place designed to help open source flourish.
E.g. Here's Google getting publicly grumpy about the AGPL: https://www.theregister.com/2011/03/31/google_on_open_source...
It didn’t just block Amazon, and even if it did, selectively blocking a single downstream commercial user would make it not open source.
The freedom of third-party competitive commercial downstream use is a guarantee against effective vendor lock in, and it is one of the important reasons to choose open source solutions.
Does that make it hard to monetize development of software by just rent-seeking in the “we don’t want to expend the effort to host our own things” SaaS market with a hosted implementation? Yes, but, too bad.
No, I guess it also blocked google, oracle, etc. from monetizing their work without sharing the proceeds from the quite considerable profits they make (something they seem happy to do).
Didnt block anybody else though, as you well know. Any regular user of Elastic faced 0 additional restrictions.
>Does that make it hard to monetize development of software
No. As Amazon demonstrates the software can still be monetized by the most powerful rent seeking players in the market like Amazon.
Does it make it hard for the companies not running monopolistic rent machines that actually build open source? Yes. MUCH harder. It would have killed elastic.
>too bad.
Yes, too bad for open source.
You can be anti open source if you like (it has made a lot of people mad, not just MSFT and Amazon) but don't try to dress it up.
> Does this mean that Elasticsearch and Kibana are no longer Open Source?
> Yes. Neither the Elastic License nor SSPL have been approved by the OSI, so to prevent confusion, we no longer refer to Elasticsearch or Kibana as open source. We updated our website and our messaging to refer to these products as “Free & Open,” and when talking about the licenses directly, we describe them as “source-available.” If you notice an area we missed, please let us know, so we can correct it.
I am not sure anyone should care what OSI thinks about a given license
To the corporate funded OSI. I know.
The same corporate OSI who joined in the character assassination attacks on Richard "open source comes before profit" Stallman for very similar reasons.
Could you expand on what you mean? Are they violating the GPL? What does it mean to do the bare minimum but not violate the GPL?
It's unrealistic to expect corporations to go above and beyond when complying with something like a code license. If you want someone to do something, put it in the text of the license.
To be fair, it's a tough sell to tell your management chain that you have to spend a week every month arguing and refactoring code just to make some internet people happy about it
Sounds like a weird management chain then.
Generally I've seen the conversation go along the lines of "we should 'contribute' these changes back, so the upstream people can check them over for bugs, and we don't have to write this stuff again for the next product".
This is completely false as they (amazon) want to market themselves has developer friendly and supportative of open source
Their actions however say something completely different.
Further Loads of companies and organizations go beyond the legally required bare minimum in all kinds of things not just open source. You have a completely warped view of human interactions if you want to boil all humanity, and all interactions to just what is "legally required of us"
I agree up to the point where they start going on about how much they love and support open source. At that point going beyond legal necessity should be a given.
If the legal requirement is X, and you do X+Y, there will still be people bitching about the fact that you didn't do Z, or that Y is really just self-serving, or something else. Someone someone will always complain about what you're doing because you're a profit-making enterprise.
The corporation decides to do that. They shouldn't advertise that they support charities if they don't actually support charities.
AFAIK (but I'm far from knowing about all examples), no, they have a history of taking open-source software, forking it out of the control of the creators, and extending it there without bothering to sync the changes back.
It is hostile to people that want to gain anything from their software, but it is not against the license.
Examples: AWS IoT Application - https://github.com/awslabs/iot-application - OOB app for monitoring IoT assets.
AWS IoT App Kit - https://github.com/awslabs/iot-app-kit - Library for building AWS IoT apps.
Amazon absolutely strip-mined open source.
In 2001-2006 Amazon was all built on open source and had ditched HP and Digital Unix for Linux, RedHat, Apache, Xen, etc.
It was abusively hard to contribute back to open source projects, including the Linux kernel, and you had to get extensive signoff and review to ensure that you weren't giving away any IP with your lock order bugfix or whatever. There were also constant reminders and threats to not engage in open source discussion in any way that could leak company IP which created an atmosphere where it was much easier to just not participate at all. It was definitely very easy to pull open source into the company and very, very difficult to participate going the other direction.
From my perspective when I was at Red Hat, Amazon was very much an IT consumer. Amazon was a _hugely_ important customer for Red Hat, even appearing on stage to give their testimony about the benefits [1]. It was a "lighthouse" customer that really opened some doors to selling Enterprise Software.
Part of what we sold to Amazon was being their connection "upstream". In 2001 it was quite unusual for an IT shop like Amazon to participate directly upstream. One benefit of being a customer of Red Hat's was: they will fix bugs and work upstream on your behalf.
It was a very reasonable approach in those days to use an existing contractual / support relationship with a vendor like Red Hat to deal with all of those details and complexities, which often required strong social connections among open source developers and maintainers working together upstream.
I think that over time as large companies shift to consume open source software directly from upstream, they have to build the capabilities and connections to participate directly rather than going through an intermediary like Red Hat. It can take a fairly long time to do that.
[1] https://www.computerworld.com/article/2577912/linuxworld--am...
To maximize demand for a product, make its complements free. For example, to maximize demand for mobile phones, make the operating system free and add a growing number of built-in default apps for free.
Surely, Amazon is thinking along these lines. Which Amazon products/services benefit from all the free code Amazon is contributing/releasing? AWS.
See also: https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
You can probably overstate the degree to which Amazon didn't contribute to and interact with open source communities 5+ years ago (and probably also overstate the degree to which there's been a complete sea change) but, as you say I think it's fair to say there's been a directional shift with more substantive activity in addition to what I'd mostly describe as talking the talk previously.
The industry as a whole has also developed into this incredibly complicated coopetition landscape especially around containers in the past decade.
The original apache httpd, for instance, was written by people working at different organizations collaborating together to build the web server that they needed. It wasn't something people did as just a hobby, although the individual contributors may also have had some altruistic and certainly cooperative motivations. It definitely wasn't something people did motivated by trying to make money off of selling apache httpd in some way.
Open source always requires self-interest to fuel it. The original model was people collaborating across organizations because it provided something useful to their organization -- exactly this. When this stops happening, because organizations cut budgets to the bone; because they gatekeep anything that might benefit competitors as well as themselves; because they can free-load off existing open source instead -- what other source of self-interest is there?
One is people trying to profit off of the open source itself, with "open core" models, or models where the license can appear open but tries to ensure that the creators maintain a monopoly on any profitable aspects. Or, we can just get lack of investment in open source everyone is trying to free-load off of, withering open source with burnt out maintainers.
Companies committing to open source because it saves them the time of maintaining it separately is the best case, and describes the environment of the "golden age" of open source.
It made me wonder how much valuable open source is powered, quietly, by people who are just used to being exploited.
> Open source always requires self-interest to fuel it.
That self-interest doesn't necessarily have to be profit motivated either. As one of the replies to your comment suggest, probably the majority of open source is maintained as a hobby by an invested/interested user. Open source does not have to have a profit motive to make sense.
But likewise, a profit motive is also congruent with open source. It greases a lot of axles, to put it nicely. There's definitely some disparity in open source contributors in terms of money being made, almost like the income gap between wealth and poverty in general. I wish we could do better at fixing that and helping the "little guy" that is contributing their soul into a project.
Still though, open source is driven by "selfish" means, in whatever form that takes.
There were a number of internal programs spun up with blank checks to mitigate a future log4shell. The Open Source program was one of those
https://github.com/awslabs/aws-glue-libs/tree/master/awsglue
https://medium.com/claimsforce/lakehouse-running-delta-lake-...
I ahve to admit no familiarity with Glue, but if that Java blob is bytecodes, shouldn't it be trivially decompilable?
Presto/Trino : Athena
It really wasn't. That was their whole approach to Elastic.
The tantrum they threw at Elastic when they relicensed was particularly cringeworthy.
I think it’s important to describe things accurately. Disliking something isn’t a “tantrum.” I think amazons blog posts were pretty normal speech and tl:dr’d to “we don’t like it, so now we forked it and will maintain under Apache.”
And they did that, so it’s actually kind of nice as I can’t use ElasticSearch under its new license but can use open search.
The fork is really half hearted with about 1/10th the resources Elastic dedicates to the mainline branch:
https://camo.githubusercontent.com/71f7e9147092c7d4b08df18da...
Meanwhile, the only effective licensing difference is that you can sell hosting with their fork - something that only benefits them.
When they had a disagreement Amazon chose to fork it. You can frame Amazon as "cringe" and "throwing a tantrum" but ultimately this is hardly either of those.
Lots of companies do similar or worse, I do agree. For instance a certain company that has obviously paid a PR company to go on an open source charm offensive (the article has all the hallmarks of pg's submarine) in this thread has strict rules about when its workers can pee. That's probably worse.
> […] obviously paid a PR company to go on an open source charm offensive […]
Please don’t do this here. Consider that others can disagree with you, or that you may simply be wrong.
> Please don't post insinuations about astroturfing, shilling, bots, brigading, foreign agents and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data.
Why not? That's immediately who I thought of.
Company 1 develops an open-sources a product, and in order to make money from it, provides that product as a hosted service.
AWS uses their market dominance to provide a similar service and eats Company 1's lunch.
Company 1 changes their license so that any other company hosting the product as a service has to pay.
AWS forks it and acts like the good guy.
I was referring to the PR whitewashing piece submitted (written by an ex Amazon employee, no less), not to anybody on this site.
Please ask questions in future before accusing and refrain from assuming malintent.
However , it is difficult in general for these open source companies to compete with Amazon's managed offering. The managed offering and its support staff eliminates the need for the tech service contracts that many of these companies to depend on.
ElasticSearch closed down the license and said "nope, no open source anymore because we have to make money". This was bad for the community, but not AWS fault
And if it wasn't AWS, it would've been Google. These sorts of open source development paid support contract based companies are having a hard time in general competing with the cloud.
I think it's fair for AWS to criticize Elastic for violating their ethos to make more money, but it's also pretty clear that the reason their OSS business model didn't work was managed cloud services like Aws.
And Mongo, Redis Enterprise, Databriks, neo4j, etc.
Their 'partner solutions' and not then directly competing with their partners is a win/win.
https://www.elastic.co/blog/elastic-and-google-cloud-announc...