The third wave of open source migration
blog.tidelift.com
blog.tidelift.com
When winter comes, those projects don't make sense anymore because cost cutting measures are in mandate. The same leader might even make the case for the Open Source alternative.
I've seen this enough times to know it is a pattern in our industry.
I guess, when you're on the rocket part of the hockey stick it seems that you can solve anything by hiring more people. A lot of leaders kind of forget what they sacrifice when they do that.
To a certain extent:
Create a corporate culture where people can get work done without needing to be "protected" from the rest of the company around them
Select against empire-building folks.
Push a culture of reusing and improving existing tools whenever possible.
Recognize and reward people for avoiding and eliminating technical debt, not just for building new things. What you recognize and laud, you will get more of. (There's a balance here: you need both kinds of people.) Hire more collaborators and less "ninjas".
If you have enough budget, empower people to experiment with interesting things without having to hide those folks under a corporate-level justification smokescreen. Understand that some explorations and experimentation will not pay off, and support that anyway, so people don't feel like they have to hide things until they're successful.
Encourage people to report organizational issues and organizational friction, and fix it as early as possible, so that people don't have to create "shadow infrastructure" to get their job done.
If you are making a lot of money and one of your employees has performed well, and never asked you to hire more people to do their job, give them a God damn raise
https://en.wikipedia.org/wiki/Trusted_computing_base
In-house solutions are not necessarily more secure, just more obscure/less discoverable.
I myself have used two of these projects heavily (Airflow, SuperSet) and am thankful to the teams for building such great products.
But did this make any business sense for AirBnB to spend on these? In this post COVID world, very unlikely this would continue.
The sad thing is that you often have to play the game defensively -- I've seen semi-decent organizations end up consumed by the 80% overweight departments because bigger departments end up with all the advantages just as a function of their size, even if they're almost completely worthless. Have to keep pace with the useless hiring just so they can't say "well it's our smallest group, only 5 guys", etc.
It already has a name; empire building
sometimes building something new seems obvious. i have to push hard to get people to back up that obviousness with evidence. (provide me with evidence that these existing solutions are not suitable before we build a new one from scratch.
sometimes building something new seems easier or cheaper than the effort required to evaluate alternatives, especially if the solution only takes a few days or weeks of work (take the obviousness factor above into account).
another factor is the seemingly common dislike of working with other peoples code. especially when it's known that modifications will need to be made. do we take this system with a lot of legacy code where we have to modify 20% or is it easier to start over and build what we need?
You could bolt both of those on top of OSS but that can end badly.
There's more OSS than ever but there is a huge lack of stability in most of it.
In way to many companies, engineers (including engineering managers) have to explain to a non-technical C suite that OSS/FOSS IS NOT FREE and NOT GUARANTEED TO BE CHEAPER than going with proprietary or self-written, minimal, tested and concise solutions in any given context. They can be, of course, and with some engineering investment they can be for the longest time in the feature, but every case is different and a "wave" certainly is not what just removes the need to do this kind of evaluation.
this is not a FOSS question. (i would not touch a closed source library as s potential dependency with a 10 foot pole. it's either FOSS or i write it from scratch)
this is a question of the ability or willingness to adopt and maintain 3rd party dependencies.
fear of FOSS could be added as another reason, but that reason is even worse than mere unwillingness to work with other peoples code.
in order to use a 3rd party application or library as a dependency you must be able or willing to maintain it yourself, in case you run into an issue with its support. (that's one of the big points of FOSS, btw)
you are right about unnecessary features being a risk, but again, that has nothing to do with FOSS as a choice.
this is a general problem with humans. They are "convince" that they do good even when they don't.
> sometimes building something new seems obvious. i have to push hard to get people to back up that obviousness with evidence. (provide me with evidence that these existing solutions are not suitable before we build a new one from scratch.
The main issue here is that the perceived cost of starting from scratch is very low especially when from his ebony tower the programmer ignores all the issues the previous programmer had to confront.
> sometimes building something new seems easier or cheaper than the effort required to evaluate alternatives, especially if the solution only takes a few days or weeks of work (take the obviousness factor above into account).
... days which become weeks, years , decades. Why is a software project (exept TeX) never complete ?
> another factor is the seemingly common dislike of working with other peoples code. especially when it's known that modifications will need to be made. do we take this system with a lot of legacy code where we have to modify 20% or is it easier to start over and build what we need?
If builders made buildings the way programmers make SW ... Why do SW cannot improve the code ? Why do they have to "invent" the wheel every day and in the end they obtain the same thing or even worse ?
I'll give examples: 1. KDE and GNOME. With every release the interface is different. Old programs do not work anymore - they have to be ported to new KDE 4 or 5 or 6 or whatever. They were at the beginning way ahead of windows (r) and Mac now they just copy. Same with GNOME.
2. Win 10 looks the same like Win 1.0. Functionality is the same. Will Win 31 look the same like 3.1 ?
3. Dll or .so hell.
May be that is why OSS works better with System Software or Framework like software instead of End User facing Apps?
I have more faith in bureau shaping, which is almost the same.https://en.m.wikipedia.org/wiki/Bureau-shaping_mode
My it dep is mostly run by an Oss linux geek. This is super cool
Last month, we lost a big project against SAP (budget 5m€): the company choosed SAP because their holding was willing to pay for the project. Last week, the same prospect came back to Odoo: as the holding could not afford such a project anymore, the company has to pay from its own budget. So, they choose Odoo (<1m€ budget)
I believe the next wave is a replacement of proprietary expensive business applications: ERP, SAS, BI...
Quite a few actually. If you're a software shop or digital agency, I think the Odoo Studio and marketing automation features are the big gaps.
For the latter, there's Mautic.
However, good CRUD builders are difficult to find. Seatable is promising, haven't evaluated it yet.
https://twitter.com/wdaali999/status/1161973951565881345?lan...
Basically, anything not open-source is not cool any more - SAS, matlab, SPSS. Kids are not learning these frameworks in school and don't want to use them. I see open-source taking over Data Science by the time this recession is over: Jupyter, conda, Scikit-learn, TensorFlow, PyTorch, RStudio, and even PySpark.
But it's really painful watching some of the smaller proprietary packages stay afloat.
High prices for hobbyists who are not students.
The Wolfram store does not even give you the option to pay with anything else than credit card (in Germany, other methods of payment are strongly preferred), so you have to use a reseller if you don't have or don't want to use a credit card for payment.
Side story concerning the previous paragraph: I got a voucher from Wolfram Research to update my Mathematica license via the Wolfram store for a price that seemed fair to me at that time. So I wrote to the support that I would love to accept their offer/voucher, but have no credit card. Wolfram Research support told me, this is not possible; I have to use a reseller in this case (for which the voucher does not apply). OK, I get it: Wolfram Research does not want me as a customer.
The vendor lock-in is so strong in this industry that octave could be 10x better and it wouldn't dent Matlab's market.
Hasn't this already happened by now?
No wonder the MBA’s view software developers as suckers.
Read through HN, and you see maintainers of hugely popular open source software that probably has saved corporations billions of dollars, burned out, mentally exhausted, living in poverty and begging for donations.
It is important to recognize the value of developer time too, though. There's a cost in dev time for setting up a "free" project.
That's why I think that any open source project that gets too popular will have to have a cloud vendor strategy, otherwise they'll get done to them what AWS did to Elastic Search.
I also thought it was interesting that the author mentioned support for the various application libraries. I know that there have been several "tip" type applications (gittip, gitcoin.co) that try to align incentives and allow open source developers to make a living.
This.
In fact sadly one of the marks of a successful OSS project is ability to pay yourself, and even perhaps commercial success. In Linus' day it was enough to have a whip round online to buy a faster Pentium machine, but these days it's a foundation and cloud offering.
I hope that isn't the case, but we'll see. Maybe the answer is niche operations that are too small or domain specific to be noticed by the big folks.
AWS's fork of Elasticsearch (ES) is a good example of the operationalization you talk about. If I was building a niche eDiscovery solution for the compliance market on top of ES I'm likely to want to offer that on AWS or another cloud provider at some point. If they offer their managed version of it, then I don't have to support it.
You can make obvious arguments like special extensions, or all compliance customers don't use the public cloud, but ultimately many users are going in that direction. Any type of general-purpose tool like ES is ripe for the picking by any cloud provider when it gets popular.
The only options I see are licensing it in a way that's prohibitive for adoption by cloud providers, or engaging them early to become their subcontracted maintainer for the product on their cloud so that you remain in the value chain.
Ultimately, it's not the type of business that should be taking VC money because the upside is, so limited when you have the typical successful outcome. The only way to get to an outcome that VCs might like is to use the income stream, or talent acquisition, from the open source product to pivot into something that isn't as limited. Doing that is really hard.
From there the strategy would depend on whether you want to stay small or not: To get bigger, you'd start going deeper into the open stack to scale things up and provide a wider array of services. If you stay small, your organization will necessarily be more focused on interfaces and compatibility while maintaining that top-end UX. In both instances there are plays for open source, but with different characters; the big company will tend to code-dump an enterprise toolchain, the small one will primarily be a contributor to a foundation project or open some of their internal interfaces.
Examples please? (I might be in that situation)
Unfortunately, there isn't a very good business model to fund developers to work on such applications, when the application itself is not a sell-able product.
These sorts of projects can work if there's a significant overlap between end-users and capable developers. If there isn't, then they're often woefully inferior to the commercial alternative.
What I write and share is directly re-usable to others.
We can walk a path and share our work, and others can take it and copy it immediately.
Whereas with UX.. Most of the time it's written for the benefit of someone else. And it's not something that can be used elsewhere very easily.. Often needing to be completely re-worked to be useful anywhere else.
It's probably always going to be harder to contribute design overhauls than bugfixes & features.
Closed source also sucks big at UX (see Office 365 for details). But M$ offers "integration" a thing which OSS cannot offer. Also , having enough lobby (corruption) or market share , the UX does not matter.
I think using these packages and projects requires more due diligence and planning on staff to pick and support, but I think the current highly variable support project by project works out well. And then for big stuff (Linux, Postgres, etc) some commercial support is brought in.
I’d much rather see more support for companies donating developer hours to patches and features. Some way to recognize in kind and labor contributions and expand recognition for these kinds of contributions. I think this works better for software than trying to get every company to pay into some support fund. If you want to pay structured licenses for everyone, there’s a model for that. Trying to shoehorn license fees on top of open source loses a lot of the efficiencies, I think.
I think you're missing the real value here of the support model and services a company like Red Hat can provide to large orgs.
It's not about having support for every package, it's more about having others do the hard, expensive work of presenting you a portfolio of projects known to work well together, so you can focus on adding your own business value rather than spending hours duplicating effort from others and debugging arcane issues.
To some extent you can't out-source everything (and I personally wouldn't recommend that. I think having some in-house experts is really important), but not everybody should roll their own OS, DB, container orchestration, etc.[1], and finding consistent options that work together can be difficult when you have huge, diverse, engineering departments with different values/priorities.
I don't disagree with you: I think contributions in either code or donations are a great way to support FOSS projects. I just think there is also a lot of business value in the support contract style method, because you're not just buying insurance, you're buying real value in the form of somebody presenting you with a portfolio of disparate open source projects that have been integrated and tested together.
[1] Note I'm not talking about "React v. Vue" which I agree support wouldn't make sense for.
I work at a DL/ML hardware company; and many here on HN would know that packaging DL libraries correctly is a nightmare and a half. In fact a good chunk of our value prop is offering an open source pre-baked bundle. It's great (and I've deployed it on some friends' machines and they love it) but it's, let's say three-quarter-baked and moving it into a "fully supported" model with domain-specific expertise isn't something that our company has figured out how to transition into (also not easy given our company is really small), and given business strategy and software strategy are independently difficult enough problems.
Do you (or any others) have any suggestions?
Obligatory (from 2014) "Why There Will Never Be Another RedHat: The Economics Of Open Source": https://techcrunch.com/2014/02/13/please-dont-tell-me-you-wa...
Luckily it's not the only path for growth... As I mentioned we're a hardware company so it's more like we want to wind up as a Dell (or in a worse scenario, IBM, which isn't too shabby)
To a hw company, I think the issue of how to monetize SW is to some extent clearer as well as pretty different than for a pure sw company. You're selling a tangible physical product, but customers are not interested in the raw hw but the complete package of hw plus supporting sw (drivers, SDK, whatever) + support. So you have to choose what is the appropriate model for you. E.g.
- Use profits from HW sales to develop OSS that makes the HW more useful to customers, and thus increases HW sales. For instance, one argument in favor of this would be that drivers included in the upstream kernel and user-space software in distros makes it easier for customers to use your HW. And you'll get OSS brownie points which might also help drive sales.
- Or make the SW free but not OSS, in case you're worried that your competitors could just take your OSS and use it with their HW.
- Or make proprietary SW that you sell in addition to the HW. I'm not sure customers care about how you split the total bill they're paying, and free (as in beer) software is certainly a lot easier to deal with for customers (e.g. license hassles). But again, it depends.
- Oh, and another potential advantage of the OSS SW model is the 'commoditize your complement' angle (do a web search on that phrase if your unfamiliar with it). tl;dr You can use OSS SW to undercut a pure SW competitor, as your income is protected by your 'HW moat'.
1. Having a product that solves a real need for the buyers (and preferably one they already know they have)
2. Getting that product "certified" with a partner, such as OpenShift.
I cannot get specific, but there are a few companies I've seen that developed a clever solution to a problem, packaged it into a Kubernetes Operator, got it certified with OpenShift, and because they had that logo they got the audience with the people with purse strings. At that point it's classic salesperson strategy.
A lot of big customers right now want to be able to point-click install stuff easily into their existing platforms/stacks, and then "check a box" in their list of requirements. If I were in your shoes, I would look at how people are already using it (OpenShift? AWS? Azure? Google Cloud?, etc and go from there. Getting "certified" is usually a mutually beneficial arrangement. It will take a bit of work with bureaucracies (which will annoy you greatly), but the payoff can be huge.
https://salsa.debian.org/deeplearning-team
For now the best way to contact the team is the debian-science mailing list:
https://lists.debian.org/debian-science/
An example of some recent work is this update on packaging pytorch:
https://lists.debian.org/debian-science/2020/04/msg00070.htm...
If you think you don't need that: use free riding flavours i.e. centos/ SUSE leap/...
Tech support can tell by looking at the system if you are a paying customer or a free rider. And sales can look at the download numbers and management servers if your installed base is what you claim it is.
If you are doing this at scale, you're going to end up paying way more in salary to employ people to do it than you would if you just went with Red Hat support. And since early in an effort you need more people, you would either have to lay people off or find new jobs for them.
I often swoop in and knock out tasks in hours that would take the company's infrastructure/SRE teams days or weeks to do, because I've done it a lot and I know what I'm doing. Its also less likely that I'll make a configuration error that exposes itself in prod, simply because I've already made those errors in the past and learned from that mistake. And when I do screw it up (which is a very, very, rare occurrence ;-) ), the company isn't scrambling to fix it. They call up support and we get it fixed ASAP. Again it's not for everyone, but it is for some people. I usually find that buying Red Hat saves a lot of money rather than costing money, which is why so many big companies do buy us.
Yet, "the rise of hosted cloud services like AWS, Google Cloud, and Microsoft Azure" is just "anti-pattern" for the subject of the article. Commercial companies that exploit (fuzzy term here but still) OS software.
But increasingly it's becoming a source of cheap labor. It used to be that you get a college degree and start a job. Now you need years of schooling, unpaid internships, postdoc and unpaid scientific contributions, an extensive GitHub page with open source contributions, etc to get the same job. The competition for better CVs will push individuals towards taking years of unpaid jobs against their will, which is negative.
Edited: And no one I know does unpaid internships -- most CS majors are making $20-40+/hr over the summer of their second or third years in college.
No discussion of anything signifcat:
* Are commercial corporations contribution back to FOSS software they use?
* * Additional functionality and bug fixes?
* * Grants/donations of money, hardware or even developer time?
... Amazon, Google, MS run mostly FOSS on their clouds, and pocket billions, but certainly don't give much back.
* Does "open source components" just mean FOSS inside but closed commercial outside, or do companies transition to making FOSS?
* What about hardware? Or at least, device drivers and firmware?
* What about all those SaaS and PaaS platforms even the article itself mentions? Their engineering setups, and software in particular, are mostly closed. Where's that promised "wave" for them?
Tech is dead.
https://www.zdnet.com/article/amazon-and-commercial-open-sou...
"Vendors developing those open source products started accusing AWS of strip mining, i.e., reaping the benefits of the products, without contributing back to their development."
https://aws.amazon.com/opensource/
For example running `git shortlog -ne` in the Linux kernel git repository will show a number of Amazon folks with many commits to their name.
Now, if they contributed back to FreeBSD, that would be meaningful, since they don't have to.
It would surprise me if Amazon use FreeBSD, I thought they use Xen & Linux KVM exclusively?
See https://twitter.com/cperciva/status/1211125881264934917 for one example of working with FreeBSD.
It's truly awesome that I can send an email to Amazon
saying "we're seeing an odd performance issue here" and
get back "here's a FreeBSD kernel patch I just wrote which
provides a 10% performance boost".
And people claim that Amazon never contributes back to
open source...
https://reviews.freebsd.org/D23322
https://reviews.freebsd.org/D23323
https://reviews.freebsd.org/D23324
https://reviews.freebsd.org/D23325When Amazon strip mines and destroys some projects, I don't think that's any better just because they need and do changes in the kernel
"Destroyed" was an exaggeration, at least as of today.
However I like and use some of the oss projects Amazon strip mines -- if Amazon instead paid the oss companies a part of want Amazon makes, that'd let those oss projects hire more people, improve the software even more -- and that I would appreciate, and could be made in a mutually beneficial way I think.
I watched a talk recently that argued that Amazon increases the size of the market available for the software the OSS companies are producing. So the pie increases in size and the result is likely to be more money available for the software, not less.
The OSS companies you refer to are more about using OSS as the new shareware, a loss leader or poison pill to sell proprietary software, their business model isn't about open source at all.
That's a good point.
> their business model isn't about open source at all.
(What do you consider open source biz models?)
> OSS as the new shareware
I think I understand what you mean. At the same time, in my case using only the OSS parts of the open core software, has been more than what I've needed
> I watched a talk recently
That talk sounds interesting, ... If you remember the name or speaker maybe I can find it?
Something that doesn't involve proprietary software, so support or pure-OSS SaaSS. Like RedHat.
> If you remember the name or speaker maybe I can find it?
I think it may have been one by either NextCloud or RedHat, I'll try to find it.
https://media.libreplanet.org/u/libreplanet/m/why-i-forked-m...
https://git.postgresql.org/gitweb/?p=postgresql.git&a=search... https://www.postgresql.org/message-id/92F458A2-6459-44B8-A7F... https://www.postgresql.org/message-id/9EF7EBE4-720D-4CF1-9D0...
We have thought about this quite a bit. The way to address this is to make offering your OSS as-a-service from day-1. Initially, you are small and AWS won't care. Once you become big, you should be able to compete with AWS on the service offering - afterall you know your code best.
The problem arises when the OSS vendors had a different business model (open-source, on-prem support only) and AWS is able to completely own the as-a-service market.
I actually expect Kubernetes to start offering the AWS services as CRD/operators, and not the other way around.
cloud is dead.
I hate that every single thing you buy requires an app and a login and a shitty proprietary driver.
(why does my mouse need a cloud account?)
In general hardware folks don't do software well. Software is just a checkbox.
Meanwhile there's a world of great folks out there that will make their hardware get up and dance given the chance.
Honestly they should commoditize software to increase sales of their hardware.
How much more open hardware do you want? I'll be thrilled once reasonably sized phones and laptops are reasonably straightforward to get with open hardware of course, but there is lots and lots of open source hardware.
The rumors of this bull market's death have been greatly exaggerated. The DJIA is still up over 18% over the last 3 years, or ~6% annually; barely a percent below average. The NASDAQ 100 has almost completely recovered its' losses from the initial COVID-19 crash. Major drivers of the market over the last 10 years like $FB, $AMZN, $MSFT, and $AAPL are through the roof. Granted that a lot of this is fed meddling, and yes a lot of people from the service and hospitality industry are out of work, but the primary engine of our economy is humming along. I would not be surprised to see a record Q3 this year.
The general move to the cloud rewards AWS/GCP/Azure with the two primary "winners" currently being AWS and Azure. This will continue as companies move to the cloud, but expect there to be a flattening of growth.
As for Facebook and Google, they rely on advertising, which relies on consumption. When there is 10+% unemployment and 20+% underemployment, consumption is highly likely to at least flatten if not fall off a cliff, particularly in elective consumption like high end electronics.
The share market is not indicative of the economy and is a lagging indicator of company revenues.