E.g. https://aws.amazon.com/blogs/big-data/performance-updates-to...
Amazon EMR release 5.24.0 includes several optimizations in Spark that improve query performance.
Why have these optimizations not been contributed back to the community?
I don't see that happening anytime soon. There is still a market for hobbyist and boostrapped businesses who can't afford the insane prices offered AWS, Azure, GCP & Co. Storage, Computing and Bandwidth on these platform is inflated for the "enterprise and over-funded-startups" market.
Hetzner, Ionos, Scaleway and OVH are still order of magnitude below AWS in terms of prices. And the quality difference is in no way justifying the price difference. (Especially with Hetzer and Scaleway which are almost on par with cloud platforms)
> All changes to Elasticsearch were sent as upstream pull requests (#42066, #42658, #43284, #43839, #53643, #57271, #59563, #61400, #64513), and we then included the “oss” builds offered by Elastic in our distribution. This ensured that we were collaborating with the upstream developers and maintainers, and not creating a “fork” of the software.
With links on the issue numbers. Why should we not understand this as contributing back?
Someone pointed out elsewhere:
*Correction after looking a bit closer, I think Amazon has submitted at least 600 PRs, they only listed 9 in the blog post. That's better but it still doesn't change the fact their business model doesn't allow the companies they're building on the backs of to have a sustainable revenue stream. commit 8e413f85e8978da83db107aae51e5543d4779dba
Author: paulward24 <52216289+paulward24@users.noreply.github.com>
Date: Wed Jul 3 04:37:07 2019 -0700
Ensure to access RecoveryState#fileDetails under lock
Closes #43840Referring to the fact that the original 600 Amazon PR claim was rescinded in that thread
Sometimes step changes happen, and things are just different afterwards.
I see two related "changes in the world" here that are contributing to this current drama.''
1) the rise of SaaS and globally centralised cloud hosting.
2) the emergence of companies solely (or mostly) built on paying salaries to developers writing an open source software project. (Elastic/Mongo/Redis being the primary examples).
As I see it, the 30 year old "Open Source" principles didn't see these coming any more than the buggy whip industry saw the Model T coming.
Personally, I'd really like to see those three companies (and others/new entrants) come up with a workable new model for software licenses for some kind of "freely usable software" ("free" in the generic "no obligations" sense, not the GPL/Stallman "freedom" sense) written by salaried devs at for-profit companies. I think the world would be a better place with more software closer to the Stallman end of the license spectrum than the Oracle end. I don't think that's necessarily a given. It might well be true that in general, a for profit company cannot make enough money out of writing software and giving it away to keep teams of full time developers paid.
Also, I think through history every law was made in the name of making law more just, bringing the legal system closer to an ideal of justice.
We also know that simply throwing out law and bringing back case by case judgments (eg. what kings and lords of old did) just gets us further from that ideal, hence we work on our laws and licenses. But that doesn't mean we should forget that they are imperfect and sometimes they need updating.