A response to the git.centos.org changes
redhat.com
redhat.com
If you accept that it takes a lot of effort to produce what Red Hat produces and you accept that you cannot buy food with a list of your open source contributions, then the people who work for Red Hat need to be compensated in some way.
If your answer to this is that there is a big, distributed pool of people willing to do this work instead of Red Hat then you don’t believe that Red Hat adds value. But the question remains: who compensates those people?
And this big distributed pool of people ready to do free work definitely doesn’t exist already. Well, it does, but it’s just Redhat. And they are rightfully changing that.
It could be the companies they work for (like is done with the Linux Kernel), but that doesn’t exist and starting it is a pretty big undertaking.
At the end of the day the comment in the post about the anger coming from people who “do not want to pay for the time, effort and resources going into RHEL or those who want to repackage it for their own profit” rings pretty true.
I don't think anyone is particularly surprised, or will miss them much. It's just kinda sad to see what was a reasonably happy opensource success story end with a giant corporation shutting down some of their releases out of protectionism.
No. That was their excuse for cancelling CentOS. If you remember, it was universally unpopular.
If RHEL is only for paying people, as McGrath from the pulpit so expounds, most people are gonna leave. The value was having a community, a good path, that extended from open source to paid support. If McGrath blows up one end of the continuum, the other end will certainly wither collapse & die. This is a deathmark. People are upset to see a vibrant world shut off and thus shut down, whatever the value/greed/whatever of the situation. A pillar of Linux has left the chat.
It's exactly these kinds of fiddly details that get weaponized when you want to derail a distro. It's not easy or frictionless anymore.
Either you don't understand what CentOS stream is, or you're being ridiculously hyperbolic.
It's starting to change, but for years people would look at me like I was crazy for running Debian/unstable aka Debian/sid on my machines/personal servers. Being willing to have volatile systems is a big barrier. Only having "dogfood this wip" or "pay us" as the two options is a radical ask, one that doesn't really seem survivable to me (rh/centos won't go away, but their mindshare seems likely to implode to me).
Last year, with RH cancelling CentOS, I switched to Rocky expecting another 8+ years of updates. We'll see how that pans out.
It wasn't worth putting up with so many free users for a paid product, just to have a small minority actually pay anything.
It used to be worth it for 20+ years. Now something changed.
For my servers I used to use 95% centos, 5% rhel. Now I’ll use 0% rhel.
I didn’t need support for most servers but I liked having comparability for when things grew into needing it.
RH doesn't understand this.
What Red Hat is doing now will probably increase their license sales in the short term. But they will do so by cannibalizing the entire Red Hat-compatible ecosystem. All the "freeloaders" provide RHEL with most of its share in the distro market. Kill them off and it will become a technical backwater, all the small and medium users will go somewhere else and there will be no on-ramp for new customers (as Red Hat is too technically different from what they have). It can take a while, they might get RHEL for their old systems but they'll go SUSE/Debian/Ubuntu for new stuff.
I'd say that they are literally doing what Travis CI and other companies have done in the past, they have no vision of growing their market anymore, now its all about cost cutting and squeezing the most out of their existing share while the music is playing. So no, I'm not angry about it, but on the other hand I can't see it as anything but a sign of irreversible decline.
1. The above market is pretty much "saturated" in the sense there aren't many proprietary UNIX systems left to convert to x86+RHEL. So not much growth potential there.
2. Oracle has their own RHEL rebuild that they aggressively(?) market to Oracle customers. And I guess it makes sense if you have a server to run Oracle DB on, to also use the Oracle RHEL rebuild to avoid the situation where you have a problem and Oracle and Redhat point fingers at each other and nobody takes charge of solving the issue.
3. Cloud. Cloud is obviously big these days, and it seems for one or another reason cloud deployments often choose some other distro than RHEL, be it one of the free rebuilds (CentOS, Alma, Rocky, OEL), or something else entirely like Ubuntu.
While I'm not privy to any RH internal info, it wouldn't surprise me if they're worried about the above trends continuing. Whether this recent change to make life for rebuilds a bit harder will materially affect anything, and if so how, remains to be seen.
(I have no personal experience trying to run anything like this, this is purely citing a previous comment made.)
Non-RHEL hosts are only supported is Red Hat has someone to talk to at the vendor or cloud provider, so that they can debug stuff that breaks only in their environment and can't be reproduced. This is done to cannot guarantee the level of support that customers expect but very few cloud provider can provide that. I am not even sure that IBM's cloud made the short list.
On other unsupported hosts we will ask the customer to reproduce on a supported host if we think the host is the culprit.
Probably not https://www.redhat.com/en/partners/certified-cloud-and-servi...
Best damn business innovation ever, renting people shovels to dig themselves a hole with!
We had Oracle databases running on Oracle Solaris running on Oracle SPARC servers, because the bosses were convinced of the synergies. When the problems started (with Solaris zone clusters) it was a happy merry-go-round of fingerpointing between Oracle support groups.
Oracle is on a rather short list of companies I'll try to avoid doing business with if at all possible.
Red Hat is more profitable than ever. It's about wanting EVEN MORE.
They want to be a little bit more careful with dogma that makes sense to their management, will get them patted on the head there, but is just nonsense seen from other perspectives - like my perspective contributing to their project for free.
They address that in the blog post:
"We don’t simply take upstream packages and rebuild them."
Rest assured they're adding good juju to your upstream code! This is 2000s-era Microsoft-level wordsmithing.
that's only a relevant statement IF you are trying to make your living off of it and they're rebuilding it without compensating you.
they're not saying rebuilding their opensource packages is a threat to the creators. They're saying people freely distributing the things they pay people to make is a threat to their ability to continue to pay people to make it.
I don't care if people repackage my open source code because I'm not trying to pay for food and shelter with it. If i was, I'd be screwed because every repackaging would be just giving away my work without compensating me.
'we' (and by 'we' -- I can only speak for the small slice of uk/eu companies I've worked at for the last ~18 or so years) were /forced/ to use redhat when running stuff like SAP, IBM WebSphere, Oracle, various terrible BI systems, ERP stuff, and so on for 'support'.
Then the cloud happened, and rhel/centos came along initally as we knew the tools, and we were running mostly the same stuff but on other peoples gear.
Slowly the application estate exploded and now I think it's fair to say we target k8s instead of 'rhel' as the place where our software is running, and rhel and it's brothers go absolutely nowhere near my k8s estate or any I've worked on (typically, ubuntu server nodes if I have to build my own, and alpine for contiainers).
So; people who are upset by the recent developments (and lets be honest, did you expect a thriving community around an IBM product?) -- why? What's rhel got that debian doesn't? Or ubuntu?
Debian (and by extension, Ubuntu) chooses to not fix a significant amount of security issues. This makes sense given their business models, but is an unworkable position for a huge number of enterprises that depend on Linux. Example: https://ubuntu.com/security/CVE-2021-45464
We actually _CANNOT_ use redhat for compliance reasons.
(We're using ubuntu LTS as it goes)
In the end, we just went with ubuntu for those nodes, and they all passed the certification. Shrug.
Since then, we don't even need the OS to be certified, since we are using confidential computing, and we stuck with ubuntu for our k8s nodes etc -- but we are forbidden from using rhel anywhere by our legal / compliance people now.
The bottom line really is plenty of auditors I've seen don't know how to check for vulnerabilities other than by checking a version... That's it.. Their tools or reporting only know package must have a version greater than x.y.z.
I have no idea who in their right mind would use derivative, it has zero benefits and is just build worse than vanilla Debian.
For each one of these packages, it's a crazy process to prove that the CVE they reported isn't actually there, and it delays our accreditation to the point where it was easier for us to change distro than go thorough hundreds of these..
Dealing with bad security audits isn't fun.
Maybe Ubuntu LTS does less security backports than RHEL, but that doesn't make it better per se (saying this as an Ubuntu user).
Short version:
https://www.redhat.com/en/technologies/cloud-computing/opens...
And by settled I mean "who cares as long as it's not Windows?".
Yes, I do not mean to sound harsh but I do not know how say this in a nice way. To me, this means we will still happily take work from our volunteers, but we will restrict other people from using this work so we can get more $. But thank you volunteers for keeping our payroll low and helping out our stockholders.
I really think this is another small step to what I believe is the corporate take over of Linux.
The one people have been predicting for decades because they refuse to understand how Linux works and/or they want attention.
None of this is new. None of this is even hugely interesting. Red Hat is one distromaker out of many, and nothing makes anyone use them or their products. Trying to conflate one distro with Linux as a whole is common among people who are either ignorant or attempting to troll.
I just can't see it being taken over by anyone, Valve or anybody else.
This is a failure of the Linux Foundation to push back on proprietary hardware vendors, one blaring example is Nvidia. Even Linus has commened on Nvidia.
The Foundation should stop allowing proprietary hardware in Linux, instead these vendors donate $ to the Foundation and we are stuck using this hardware.
Another example is Secure Boot, some laptop manufactures are no longer allowing Legacy Boot. Were is the Foundation on this issue. Instead Microsoft donates large amounts and nothing is said about who signs the keys. It is just about impossible for many people to install Linux (or a BSD) on some new hardware without jumping through Secure Boot Loops.
And we have Intel ME, totally closed, the Foundation did nothing to complain and open that up either.
I could go on all day, and I am sure others can bring up many of these issues I know little about.
I think you're wrong on this one. You seem to be mixing up UEFI vs Secure Boot. The elimination of legacy boot in favor of UEFI has nothing to do with Secure Boot being enabled. Secure Boot is a good thing. In this case Microsoft is just acting as a Certificate Authority. Mainstream distros like RHEL, Ubuntu, and Debian have signed bootloaders. They pay MS a modest fee for what is essentially CA services. If a hobbyist distro doesn't want to pay then they don't get the benefits. A lot of OSS developers don't even sign their code because they object to paying for a code signing cert. If you want to play in the walled garden you have to pay the toll. Despite this I've yet to come across a device that prohibits disabling Secure Boot.
I know if I need to get a new Laptop, I will need to either pay twice the average for an open system that meets my needs, or spend months looking for compatible hardware.
So far the ~8 years old Laptop I have is working fine with Linux.
That's just a willful misconstruction - they're absolutely contributing to upstream, as the post makes clear.
That includes expert users who don't need enterprise support. If you make them use something else, then that will become the most battle-tested enterprise distribution.
It also includes developers. If they run on your platform, then it will get native support. If you make it easy for them to test on your platform, then it will get first-class support. If you make it difficult and they feel like second-class citizens, you will get no support.
Open source is about community, and RedHat has made it clear that the only community they care about is companies willing to pay for expensive per-server licenses.
With the rise of the cloud and containerized workloads, the business model of providing long-term support for on-prem servers is declining, and they are just milking the customers who remain.
This is exactly what they seem to think. In this blog post, they mention that the open source community will simply revert back to being hobbyists and hackers if they don't make this change. This is because Red Hat's management right now thinks the open source community is entirely comprised of Red Hat, Red Hat customers, and random hobbyists and hackers. Other business models, and even SUSE/Ubuntu, are seemingly invisible to them.
I can't help but wonder what this person is thinking. I get that they look at Oracle Linux (or maybe even Rocky Linux) and think, "there's money we're losing". Is that really the whole story?
I don't see how, at the end of the day, this is going to help Red Hat. There's no way Oracle is going to start shelling out dollars to IBM for licenses. Maybe Rocky Linux will lose some customers, I doubt it's going to be enough to make it worth the time, effort and negative press that RedHat is suffering.
I wonder what they’re seeing/thinking that we’re not
We were looking at Rocky Linux, but that no longer makes sense in light of recent news. So along with moving all of that CentOS infrastructure to something else, it no longer makes sense to stick with RHEL either and they are now going to lose our money.
We thought we were on a good proven path of paying for support where we need it, and using the downstream alternatives when we didn't. Turns out we should have more seriously considered a move to Debian...
I'm not really shocked Red Hat are trying to squeeze more money out of their existing audience, everything they offer is eye-wateringly expensive nowadays. I almost choked when I got a quote back for the Ansible automation platform, before installing AWX for free.
AAP is crazy expensive, just like RHEL. I think they are going to lose a lot of customers and dollars due to this, both immediately and even more so later.
It's basically if a car company had a free car on every road, hoping it would drive people to buy a car for themselves, but people are just using it as their car, avoiding buying one because the free one exists.
So why would you think they would now go RH everywhere, especially after losing Cent and having to switch to Rocky/Alma, instead of going to Debian/Ubuntu or SUSE and not having to worry about it?
Not to mention that new admins and even current admins using Cent/Rocky/Alma at home to learn and play - now they can't.
There definitely was a platform effect, and now they have killed it.
* Red Hat was a bastion of open source
* Red Hat sold out to IBM
* Red Hat stopped being Red Hat, and started being IBM by focusing on $$ over open source
* Red Hat reputation degrades as $$ are put first, killing off centos, now this, just downhill from here. >
I don’t understand the “killing off CentOS” thing. I’m still using and it updates continuously.
Before the "killing of CentOS": CentOS is based off of RHEL (basically being identical with packages, just not "officially professionally supported", I think)
After the "killing of CentOS": RHEL is based off of CentOS (with CentOS-Stream being "the staging" for RHEL packages)
But I actually have no actual clue what haseen going on there and have just been somewhat following this entire "drama" with bbit of interest, so I might be completely wrong in how it actually work
That contradicts to reports like https://news.ycombinator.com/item?id=36484207
"Additionally, CentOS Stream updates often lag behind RHEL updates. This is because Red Hat won't commit an embargoed security update to CentOS Stream until after it ships in RHEL, so the developers responsible for the update will sometimes forget to commit it to CentOS Stream until a week or two after it's shipped. You end up in a weird position where you get most updates faster than RHEL users, but you often have to wait to get critical security updates. "
If not embargoed, however, they are still applied to Fedora first. But most fixes of that severity are embargoed anyway and therefore the patches simply cannot be applied to public source trees without breaking the embargo rules.
Doesn't Red Hat do the same thing for many packages? Do they substantially contribute to every single project they distribute?
I wonder if this will lead to the creation and mass adoption of GPL v4 explicitly forbidding this loophole(and others which have popped up over the years).
More analogous (perhaps? I don't know) would be if someone forked an open source project you maintained and slapped their own brand on it.
If you really don't like it, to the point where it's an actual problem, then it's probably a hint that you picked the wrong license.
Also, RHELs main point is giving a much longer support life cycle than the original dev, so they are contributing for sure. Heck Python 2 is still supported on RHEL 8, even though It's now EOL.
Are they seriously claiming they contribute to every one of the tens of thousands of packages they include in their repo? I'd like to see some proof of that.
If they're taking any open source app, and just including it in their repo without at least 1 RH patch.. then they are "simply tak[ing] upstream packages and rebuild[ing] them".
RedHat absolutely makes money from upstream OSS contributors, and not all of those projects get money or patches from RedHat.
But RHEL by default doesn't have as many packages as Debian or even Fedora. See recent announcement about LibreOffice. Red hat can't by default ship a package that it can't fix and support.
This really is the Only Sane Way To Go, as you don’t want to support your patches forever. And reapply them on every release.
Eventually they'll get it right.
So... completely tone deaf.
> I feel that much of the anger from our recent decision around the downstream sources comes from either those who do not want to pay for the time, effort and resources going into RHEL or those who want to repackage it for their own profit. This demand for RHEL code is disingenuous.
The demand for RHEL code is genuine. The expectation of a return on investment is the disingenuous part. GPL does not give you the right to deny others a genuine copy of your code. If someone pays for a RHEL license and then mirrors all the GPL packages, they would feasibly be complying with all the licenses in play. Putting up all these roadblocks and assuming self-righteousness is how the community loses trust in a platform like this.
> There is CentOS Stream the binary deliverable, and CentOS Stream the source repository. The CentOS Stream gitlab source is where we build RHEL releases, in the open for all to see. To call RHEL “closed source” is categorically untrue and inaccurate.
Where I could just go get their OS myself?
They managed to do that just fine pre IBM acquisition
They are doing just fine with it, but they want MAOR.
What makes this extra worrying is the important role RH plays in the linux ecosystem. When they get fully IBM'd, what will happen to Gnome, Systemd, Flatpak, etc?
A better pricing model. Nobody is opposed to paying them. Most people used Cent because of the cost. Instead of trying to capture that value they intentionally refused to capture it.
But now I realize he was right the whole time.
https://www.gnu.org/philosophy/open-source-misses-the-point....
I am not a Red Hat user or customer (I've been living on Debian and using it at work for nearly 15 years now), but I can perfectly understand their decision to not play along so nicely with EL rebuilders any more.
Look at Amazon, who, until very recently at least, simply took what RHEL provided, used that to lure customers into their "RHEL-compatible" cloud platform, far away from any need to compensate Red Hat in any way, and contributed back into RHEL and its upstreams... what exactly?
I can see this only become worse with other cloud heavyweights throwing their 800+ pounds behind other EL downstreams (such as Microsoft with Alma Linux) to continue this play, which will bleed most of Red Hat's established revenue streams - and with it, RHEL itself, and all the projects Red Hat is funding development of today - dry in the long run. They are trying to stop the bleeding, and I sure hope they will succeed. There is a LOT of Red Hat-funded software in Debian that I actually depend on, and if Red Hat withers away, chances are those projects will not survive either.
That said, I think this is still insufficient to thwart them.
I predict RedHat will wither away and get shut down in 10 years. Neither Amazon nor Oracle will pick up the slack. I would not bet on any projects that they sponsor.
This is a death spasm. First of many.
Yes, let's do look at Amazon! Because if RH is targeting Amazon Linux, they kind of missed the mark; Amazon, like Red Hat, now simply draws from Fedora as their upstream. All this does hurts is the old versions that were already on their way out.
The whole, literally entire point of open source is copying and redistributing.
Open source is a threat to open source companies? Yeah kinda. Many of them make money selling things that are free and, as he says “not adding value”.
Ironically Red Hat does add a lot of value but now instead of trying to sell that value they’re trying to put walls around what is free.
What if they don't patch anything, aren't they just "rebuilders" too?
> The generally accepted position that these free rebuilds are just funnels churning out RHEL experts and turning into sales just isn’t reality. I wish we lived in that world, but it’s not how it actually plays out. Instead, we’ve found a group of users, many of whom belong to large or very large IT organizations, that want the stability, lifecycle and hardware ecosystem of RHEL without having to actually support the maintainers, engineers, writers, and many more roles that create it. These users also have decided not to use one of the many other Linux distributions.
This is a very important section a lot of the comments aren't mentioning.
It seems the potential loss of new user flow for them is worth it, as the majority are just using it as a way to avoid paying for RHEL, which was never a free product.
> The CentOS Stream gitlab source is where we build RHEL releases, in the open for all to see.
So there shouldn't be *any difference* between this repo and the final RHEL binaries right?, at least on tags/releases.
But what about unplanned security patches, are these also built in the CentOS stream repos?
In red hat case, how many of those freeloading users using alma and rocky linux will actually convert into red hat customer? How many red hat customers they piss off because they also use those distros and decided to transition away from red hat in the future?
Instead of that model, they'll just use Debian/Ubuntu or SUSE, and then RH gets nothing.
When Larry Ellison created Oracle Linux, he said this explicitly: given RH's then model, Oracle can take RedHat's work for free and sell it for money.
I'd go a step further and say a lot of the anger is general angst at the _everything_ of the world, but RHEL's changes provided a focusing point in the nerd world. Or at least that's how I've come to terms with my weird interest in this story (and Reddit nonsense and the submersible story).
When everything sucks, it seems very easy to misdirect that angst at whatever is pitchfork-trendy.
-Red Hat Linux 9 terminated, 2004.
-CentOS acquired, 2014
-CentOS terminated, 2020
-Production patch visibility restricted, 2023
Really?
https://www.theregister.com/2023/06/23/red_hat_centos_move/?...
https://www.jeffgeerling.com/blog/2023/im-done-red-hat-enter... https://news.ycombinator.com/item?id=36479882
They built their world on the work of others shared in kind under the GPL, what are they thinking at this new IBM wearing RedHat's face?
To Quote: The modern definition defines free software by whether or not the recipient has the following four freedoms:[8]
The freedom to run the program as you wish, for any purpose (freedom 0). The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this. The freedom to redistribute copies so you can help your neighbor (freedom 2). The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.
No Red Hat unless you pay them for it. If you pay them for it, you can't redistribute it.
Citation please? I've sat in enough conference rooms full of lawyers to know the GPL explicitly forbids this, so it would be bordering on insane if this is what they're proposing.
IANAL so I can't say whether this constitutes a "further restriction on redistribution" per the GPL, but evidently RH lawyers don't think so.
IANAL, but that doesn't break the GPL as far as I can tell.
You are allowed (per the GPL) to redistribute the sources that they give you on request with the binaries. But as soon as you do that, they will cancel your support contract so you can no longer get any updates (source or binary) as well as support.
Is it legal? I've read informed opinions both ways
Sure, GPL will let you redistribute but their T&Cs say that if you do, you’re cut off from their portal (or something similar to that effect)
Unauthorized Use of Subscription Services. Any unauthorized use of the Subscription Services is a material breach of the Agreement. Unauthorized use of the Subscription Services includes: (a) only purchasing or renewing Subscription Services based on some of the total number of Units, (b) splitting or applying one Software Subscription to two or more Units, (c) providing Subscription Services (in whole or in part) to third parties, (d) using Subscription Services in connection with any redistribution of Software or (e) using Subscription Services to support or maintain any non-Red Hat Software products without purchasing Subscription Services for each such instance (collectively, “Unauthorized Subscription Services Uses”).
> ....
> There was a time, not too long ago, that Red Hat found value in the work done by rebuilders like CentOS. We pushed our SRPMs out to git.centos.org in a neat package that made them easy to rebuild; we even de-branded it for them. More recently, we have determined that there isn’t value in having a downstream rebuilder.
All in all, this line of commentary really shows to me that FLOSS software and community is at diametric odds with commercialism and capitalism.
One side (FLOSS) is about building and working together, even if there's bumps and bruises. In the end, everyone benefits because everyone is helping each other. FLOSS is obvious area, but so is the fediverse like Mastodon, Matrix, Lemmy, and others.
The commercial/capitalist way is to erect walls and gates, initially set them low/no barriers, and then over time extract money for access to the arbitrarily gated areas. We see this in reddit, discord, github, gitlab, redhat, etc. And then those companies will start out open, and slowly close each avenue until people are either forced to pay up, or leave. Cory Doctorow talks about this with "enshittification". "Internet of Shit", for IoT goods, is a similar anti-feature-fest where you pay more and get less over time.
maybe they should just grandfather RHEL (only support current releases for the 10 year period, no new LTS) and if clients want a security patched newer version of Linux, offer consultancy to help them switch to Oracle Linux. and then Larry will have to actually do the work lol.
maybe at a later point they can offer support for a bug-for-bug compatible rebuild of Oracle Linux :-)
https://drewdevault.com/2021/01/20/FOSS-is-to-surrender-your...
"Simply rebuilding code, without adding value or changing it in any way, represents a real threat to open source companies everywhere. This is a real threat to open source, and one that has the potential to revert open source back into a hobbyist- and hackers-only activity."
What I'm getting from this is that Red Hat has a mythos that they are saving the Linux ecosystem from the days when Linux was just a hacker's OS. But this mythology is used to galvanize the rank and file Red Hatter into doing what management decides is the best direction for the Linux ecosystem.
Seems like the engineers who claim to earn $500,000 don't want to pay for their own software...
McGrath seems to be fessing up to removing access to binaries, and has his hard-truth he wants to try to deal, but but it seems like the story goes significantly deeper or the anger/frustration wouldn't be anywhere near as high. It's not clear to me from a quick browse of the repo what backports/security fixes are/aren't available.
When Oracle messed with Java LTS AWS did Corretto LTS. When Elastic did Elasticsearch license change, Amazon/AWS did Opensearch.
They don't need to support esoteric hardware (at least other than their own).