I'm Done with Red Hat (Enterprise Linux) | Hacker News Reader

I'm Done with Red Hat (Enterprise Linux)

jeffgeerling.com

519 pointsstargrave394 comments

I don't think this sort of emotional, loose-facted "hot take" style response is real constructive. I'm unhappy about this decision by Red Hat, and am also very concerned about the trajectory RH is on. For the first time in well over a decade I'm re-evaluating which ecosystem to base all my work (and the company(ies) for who I make decisions). I don't believe that Red Hat's leadership cares a whole lot about open source anymore. If they do, they've demonstrated that they are willing to harm open source in order to (short-term at least) increase sales. (congrats salespeople, you finally won and killed CentOS. And congrats to all the people who used to tell the salesperson "I can run CentOS for free, why should I buy RHEL?")

But that said, as far as I know, one guy used the term "freeloaders" and the context wasn't clear at all who he was talking about, and certainly not clear whether that is one person's opinion or the company's opinion. I find it incredibly unlikely that he was talking about people who used CentOS and actually contributed to the community. But either way, attributing that to Red Hat as a whole is completely unfair and unproductive. Every big organization is going to have at least one person with an opinion they don't agree with, and painting the entire org with the brush of one person is fallacious.

What's the goal here? Is it to polarize into good and evil sides? Get people to dig in and defend their ground for fear of the "other side" seizing their words or decisions and holding it over them?

I like a lot of OP's content, but I am a little worried that he's a little too immersed in the Youtube success formula of tapping into emotions (particularly rage and anger) and it's driving him to a very emotional take on this issue.

> But that said, as far as I know, one guy used the term "freeloaders" and the context wasn't clear at all who he was talking about, and certainly not clear whether that is one person's opinion.

Yeah, Red Hat wasn't the one that called them "freeloaders", this was language The Register used in an article:

>> Having given the move time to successfully eliminate most of the clones, Red Hat then killed off its own official free version of its paid-for flagship product. Instead, it switched to offering a free test version, an announcement which it made accompanied with lots of positive-sounding language about community involvement, and so on. In fact, what it was really doing was cutting off those that might be seen, from its perspective, as a bunch of freeloaders. The move was accompanied with free production deployment of RHEL for developers – but only up to 16 machines.

https://www.theregister.com/2023/06/23/red_hat_centos_move/

There have been two instances I have seen where the word 'freeloader' was used by someone internally at Red Hat.

And there are a number of examples in public comments (one recently in the Fedora mailing list[1]) where you can easily substitute the phrase 'freeloader' for a longer description of the downstream communities. The same sentiment comes across either way.

[1] https://lists.fedoraproject.org/archives/list/devel@lists.fe...

Hi there. I am the author of the Register article that you quote:

> this was language The Register used in an article

So, in response to that, I'd like to point out a few things.

1. This is, I feel, now widely and well-established language in this discussion. For example, note its use in the comments to this article about the discontinuation of CentOS Linux three years ago: https://blog.centos.org/2020/12/future-is-centos-stream/

2. Others are using it in the context of this story; for example, here: https://fosspost.org/red-hat-is-shooting-itself-in-the-foot-...

3. The direct personal reason that I used the phrase "a bunch of freeloaders" is that this is a Douglas Adams quotation, or rather, paraphrase. I am not an especially big name, but those who do know my name may recognise it: I was once the president of ZZ 9 Plural Z alpha, the official Douglas Adams appreciation society. I was interviewed quite widely in media around the world at the time that the Hitchhikers Guide to the Galaxy film came out. I used the phrase in order to refer to the Douglas Adams line about "a taxi service for a bunch of degenerate freeloaders." For reasons which I trust are obvious I removed the adjective.

So amazingly primitive they even think a digital watch is a pretty neat idea.

Ford?

Yes?

There's an infinite number of monkey outside, who want to share a single digital watch.

More Arthur.

Many NeXT machines have had their hostnames after Hitchhiker characters. My favourite name been “zaphod”.

If we’re very lucky, will Red Hat read some poetry to us first?

What a bizarrely appropriate screen name. I wonder if it will be friends with me?

Thanks for the clarification. I'd just like to add it wasn't a knock on your journalism or a suggestion it was a pureposeful misrepresention, just wanted to re-cite the authors citation in his Youtube video here

This article is almost identical to the YouTube video just released, which I think partially explains the tone / style.

https://www.youtube.com/watch?v=kF5pyVUQBH8

I'm personally impressed that Fedora is fully open source while being released by a for-profit company. I would have expected, when learning about this ecosystem, for Red Hat releases to be proprietary for 5 years, and then have those versions switch to a FOSS license, just as a way to ensure they can monetize their work. But Red Hat, at least in the past, sounds like they did something amazing, being able to go foss at the same time as making money.

I understand the red hat distro itself has stricter terms

Not sure how they will fare under IBM, but the history of this company is pretty cool. I'd love to read more, if somebody knows of a really good article or writeup

> I'm personally impressed that Fedora is fully open source while being released by a for-profit company

You shouldn’t be. Fedora is a community project. RedHat, while a huge contributor, does not maintain all the packages therein.

In the end, RHEL is a fork of Fedora so it’s reasonable for the contributors to Fedora to annoyed by this change in the ability to make OS rebuilds of RHEL.

I'm not saying you're wrong, but that paints an overly simplified picture that ignores history.

AFAIK RHEL came out of Red Hat Linux and Fedora came later, then at some point they made Fedora the upstream. But it has never been a "fork"[0] and there was no clean cut, it just transitioned around.

[0]: OK, debatable - depends on the way you use the word fork. They're still Linux distros with common roots and packages were first introduced here or there. My main point is just that it's not like "Here's Fedora, and then later someone forked RHEL off" but more like RHL -> RHEL -> Fedora -> RHEL

You are quite right to point out that I glossed over the RHL -> RHEL -> Fedora -> RHEL history. Thanks for clarifying. Yes, I was referring only to today’s community contribution workflow and what I considered the fair but now previous tacit agreement between RedHat and the community of spec patch contributors and packagers. We contribute to the packaging repo and RedHat makes that repo, even the “forked” RHEL branches, available in real-time, minus embargoed security fixes.

It's not too surprising when you realize that Fedora is really just a testbed for future RHEL iterations, once the packages involved have been field-tested and hardened. It's similar to how Debian branches progress (unstable -> test -> stable).

RH takes a lot of liberties with Fedora, sometimes those go at the expense of the users. I wish they would put it in a separate foundation, because as it is some redhatters think they are special i.e first class citizens when it comes to contributing to the distro, so a lot of valuable hobbyist contributions go unnoticed.

Regardless whether Red Hat said it or not, why not call it what it is? Not every CentOS user was a freeloader and the same is true for its spiritual successors. However, a large majority of CentOS users were not contributing back in any way to CentOS or the larger FLOSS community. Secondly, a substantial chunk of those users used CentOS because they wanted all the benefits of Red Hat EL without paying for it.

That pretty much sounds like freeloading to me. Copyleft open source licenses allow freeloading [1], but they also allow Red Hat to gate source code. As long as they fulfill the license obligations towards their customers, it's fine.

Red Hat puts enormous amounts of money in the FLOSS ecosystem, I think it's completely fair for them to nudge customers with deep-enough pockets to pay for subscriptions. For other users, there are plenty of other options.

[1] For non-copyleft licenses, Red Hat are of course not obliged to provide source code.

> As long as they fulfill the license obligations towards their customers, it's fine.

This cannot be stressed enough. If we are unhappy with this outcome, maybe we should be taking another look at the licenses we choose when we release our work to the world.

License situation is still unclear to me. GPL explicitly does not allow to forbid further redistribution. RedHat does that. I'm not a lawyer, but from a layman PoV this seems sketchy.

I'm not going to analyze if this is illegal or not (I don't think it is). It has been discussed elsewhere already.

But even if it where illegal, the only people in a position to sue would be the contributors of the upstream projects that get packaged into RHEL. And I don't see anyone having the appetite for that.

I contribute a lot to QEMU and there I'm practically working side-by-side with redhat devs. They contribute so much (maintainership, code review, testing infrastructure and much more), suing them wouldn't occur to me in the slightest.

Don't think of it as suing your coworkers, think of it as suing your coworkers corporporate overlords?

I think you're missing GP's point (or possibly I am but I don't think so). They aren't saying they are buddies and so they wouldn't want to sue. They are saying "Red Hat pays a lot of people to work on this, possibly to the point where the project would be dead without them. They're not just packaging up somebody else's code and shipping it"

I adamantly agree with GP. There are important projects that wouldn't exist without Red Hat, or if they did they would pale in comparison to the proprietary solutions and wouldn't be seriously used. Qemu/KVM is a big one. Without Red Hat's investment, it would all be VMware and anyone wanting to (seriously)_virtualize on Linux would be using VMware.

IANAL, but that doesn't mean that Red Hat does have to give you the source code. They only have to provide the source code to customers to which they distribute the binaries. Of course, for GPL code, their customers could redistribute the source code.

AFAIK, the current kerfuffle is about providing RHEL source packages through git.centos.org. They currently only provide RHEL through a subscription and they fulfill the GPL if they offer their RHEL subscribers the source code.

>Of course, for GPL code, their customers could redistribute the source code.

The fact that the subscription agreement does not allow this is exactly what we're discussing here.

The GPL does not say that they are obligated to sell you the next version.

The entire point of the GPL is to let you own the software you have bought and, examine, learn and change.

The rule Red Hat have brought in is that if you exercise that right then they don't want to continue selling their software to you. That is totally within their rights, it's not like they are suing you for illegal distribution.

It doesn't prevent you redistributing what you have already been given (binaries or source), it does however allow Red Hat to not give you future iterations.

They can't prevent you, because that would violate the GPL. However I'd say getting your subscription revoked as a penalty means it is not 'allowed'.

The intent of the GPL to allow distribution to third parties seems pretty clear, and a company refusing to deal with you after you exercised a right explicitly granted by the licence definitely seems like a 'dick move', despite being legal.

> The fact that the subscription agreement does not allow this is exactly what we're discussing here.

There's nothing to discuss. What Red Hat is trying to do IS ILLEGAL and GPL explicitly grants the right to all their subscribers to gave RH the finger.

From GPL Section 8 "If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term."

It is obvious that the "subscription agreement which denies the right to redistribute" is a "further restriction" that GPL legally allows users to simply ignore.

Also Red Hat violating GPL is unquestionably clear from the Preamble: "To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others."

RH asking users to surrender their rights and revoking any subscription for redistributing is clearly violating its responsibilities as demanded by the GPL.

It's even better because not only can you ignore RHEL's infringing terms, but RHEL's license to use GPL code has been terminated.

You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and *will automatically terminate your rights under this License*. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

> a large majority of CentOS users were not contributing back in any way to CentOS or the larger FLOSS community

So what? There is no obligation to contribute back in any way.

Which is exactly what I said. The license does not require you to contribute back. The license also does not require Red Hat to give you the source code, unless they distribute the binaries to you (through their subscription service).

The difference is that RedHat changed the status quo. Perhaps not illegal but worthy of complaint and contempt.

Don't underestimate the impact that even someone using CentOS and not contributing back can still have. It's still a person or group of people learning the Red Hat way. Lots of people (myself included) started on CentOS, and later bought RHEL when the need came up because I was already super familiar with RHEL because of my years on CentOS.

Increasing proliferation of usage is a huge factor in driving growth.

Based on what I can tell Red Hat agrees with that and has gone through great efforts to invest in and evolve the CentOS project. The current state of the project is CentOS stream which is still a LTS enterprise distribution and has an efficient dev cycle that prevents the delays seen in years past. Doesn’t centos stream fill the niche you are describing?

It comes close, but doesn't fill the niche because according to Red Hat, CentOS Stream should not be used in production. People do and for the most part they have no issues, but as long as the maintainers themselves are telling people not to run it in production, I would never recommend somebody to use it for serious workloads. And if you can't use it for serious workloads, then you should invest your time into a different ecosystem that can be used for serious workloads.

If you’re running something in production you’re almost certainly making money off of it, so maybe you should send some of that money to RedHat for licenses?

> If you’re running something in production you’re almost certainly making money off of it

That is an extremely incorrect premise. There are tons of things in production that people don't make money on. Tons of new products don't make money for many years. And even if it is profitable, profit margins are often thin and RHEL is not cheap.

I have about 10 services in public-facing production right now (on Alma), and only one is profitable, and that runs in a container anyway (the host is not RHEL, just my container). I also have about a dozen or so personal/private services that I use for myself and my family. Those don't make any money either, I spend about $80 per month to run them. If I had to pay for a RHEL license for each of those, I'd just have to shut them down.

Which seems more reasonable to you: move my life to a different base that does support my niche, or shut everything down?

So what if a company isn’t profitable? Do you think that companies don’t need to pay for electricity or rent just because they’re not profitable?

As for your “personal/private services”, aren’t those covered for free under the RHEL individual developer subscription?

> (the host is not RHEL, just my container).

Are you sure that is really a supported RHEL configuration?

As far as I know, to get support unless you have a pretty special (and more expansive and expensive) support contract you're supposed to run a RHEL host and a RHEL base layer for the container images.

Your configuration looks like running standard RHEL over something not-RHEL, but without anything besides binary compatibility; so why not run RHEL UBI?

RHEL UBI has a much more lax license that you can almost pretty much use and basically redistribute freely, and you can get support for the RHEL components on it.

You almost certainly not if you are in academia, why do you think Scientific Linux ever existed?

What do they do there so it’s suddenly not suitable? Introduce bugs on purpose? Break dependencies just so? Put in insecure defaults? If you are not an organization that requires an actual compliance department to be able to work for governments and the military, how is it different?

People have been doing this, or running Ubuntu LTS without Pro or other forms of paid support from Canonical.

According to Red Hat, CentOS Linux should not have been used in production either.

> That pretty much sounds like freeloading to me.

I'll bite. The value in RHEL was supposed to be in the support. CentOS helped establish and maintain RHEL as the defacto standard that everyone targetted and prevent the development of another. I mean if Debian becomes the default that actual reduces RHEL's value proposition for their paying customers. Reinforcing a platform's dominance is contributing value.

"CentOS helped establish and maintain RHEL as the defacto standard"

Not exactly. Maintain, perhaps, but Red Hat Linux was the standard before RHEL existed and everybody wanted a clone of Red Hat AS/RHEL to continue enjoying the free ride.

Let's also not forget that CentOS had a rocky (no pun intended) few years before Red Hat stepped in later on. A lot of folks gloss over the fact that there were periods of disarray with CentOS that involved some long gaps between updates, a whole kerfuffle about a founder disappearing and the domain ownership being in question...

CentOS, as a project independent of Red Hat, existed for about 10 years. This included several long periods of slow updates to minor versions and long waits for major versions of RHEL. (e.g. it took almost 3 months for CentOS to push out 5.6 after RHEL 5.6, and even longer to get CentOS 6 out of RHEL 6).

Before Red Hat took over CentOS a chief point of criticism was that CentOS took too long to update and the core group wouldn't let new folks in to contribute. (See: https://lwn.net/Articles/435744/ and https://web.archive.org/web/20190402181426/http://www.linux-...)

Now, users have Stream which is pretty much everything you'd have wanted except the ability to claim that it's exactly specific RHEL release. In my book a major improvement, unless what I want is the ability to get support for RHEL from vendors without actually paying for RHEL.

A CentOS Stream release only gets updates as long as its equivalent RHEL release is in its "full support" period. In practice, this means that the support duration is cut from 10 years to 5 years.

Additionally, CentOS Stream updates often lag behind RHEL updates. This is because Red Hat won't commit an embargoed security update to CentOS Stream until after it ships in RHEL, so the developers responsible for the update will sometimes forget to commit it to CentOS Stream until a week or two after it's shipped. You end up in a weird position where you get most updates faster than RHEL users, but you often have to wait to get critical security updates. I would be wary about using a system like this in production.

> forget to commit it to CentOS Stream until a week or two after it's shipped.

Or months, even. See httpd and php right now, and the CVEs solved this year. CentOS Stream doesn't have it, Alma has.

CentOS Stream is on a different httpd patch release, it gets the fixes from upstream instead of backporting them. If you have a specific CVE number that you're thinking about, I can check the status internally.

CentOS Stream 8 is on:

* httpd-2.4.37-54.module_el8.8.0+1256+e1598b50 (released 2022-12-08)

* php-7.4.30-1.module_el8.7.0+1190+d11b935a (2022-08-04)

Almalinux 8 is on:

* httpd-2.4.37-56.module_el8.8.0+3560+c8e5e57e.6 (released 2023-04-27)

* php-7.4.33-1.module_el8.8.0+3477+f828cbb0 (2023-01-13)

So meanwhile, for httpd the following was released:

* 2.4.37-56.6 - Resolves: #2190133 - mod_rewrite regression with CVE-2023-25690

* 2.4.37-56.4 - Resolves: #2177748 - CVE-2023-25690 httpd:2.4/httpd: HTTP request splitting with mod_rewrite and mod_proxy

* 2.4.37-56 - Resolves: #2162499 - CVE-2006-20001 httpd: mod_dav: out-of-bounds read/write of zero byte; #2162485 - CVE-2022-37436 httpd: mod_proxy: HTTP response splitting; #2162509 - CVE-2022-36760 httpd: mod_proxy_ajp: Possible request smuggling

* 2.4.37-55 - Resolves: #2155961 - prevent sscg creating /dhparams.pem

For php:

* 7.4.33-1 - rebase to 7.4.33, fix: due to an integer overflow PDO::quote() may return unquoted string

There are already issues filled in bugzilla:

* https://bugzilla.redhat.com/show_bug.cgi?id=2217408

* https://bugzilla.redhat.com/show_bug.cgi?id=2217409

Thanks, I've forwarded this.

_EDIT_: looks like it's not intentional, there are bugs that got to MODIFIED state despite the updates having not been pushed, and then got stuck there.

That makes CentOS much less useful. Did they lie when they said it's upstream of RHEL?

They dind't lie, just kept silent of all uncomfortable truths, it soured my image of the RH the way they handled this

The value in RHEL is not just in the ability to call Red Hat for support. I submit that much more of the value of RHEL is in the packaging and maintenance processes. Selecting software, making sure it's enterprise ready, patching it, backporting fixes, QA and regression testing, etc. From my view inside the machine, Red Hat is still committed to an opensource development model, and I don't see that changing any time soon. The thing Red Hat is trying to do is find the balancing point between capturing, and giving away value, as Scott McCarty explains in this post: https://opensource.com/article/21/2/differentiating-products...

Since before I became a Red Hatter, I have used CentOS for my personal servers and projects. I have struggled with CentOS Stream for that application, for example packaged kernel drivers from ELRepo. I can understand why many are upset about this. As a casual user, I am not upset, because I still have lots of options (RHEL Dev sub, Fedora, Ubuntu, Arch, etc.) If I had given a lot of time and effort that benefited the EL community, and in turn, Red Hat. I would probably be more upset.

I find myself torn on this issue. On one hand, I really want there to be a vibrant Enterprise Linux community where nobody has doubts or trust issues in participating. On the other hand, I also want Red Hat to continue to have a viable business model around RHEL so that the community continues to be able to benefit from all of the expertise that it gets from all of the people who get to make it their main focus because they can rely on a steady paycheck by doing so.

I think that when Red Hat was growing fast, it was easy for them to have a laissez-faire approach to downstream rebuilds. Now that the business is contracting, people have had to start making hard decisions. There's no more difficult decision to make, than to lay off people. I would not want to have a leadership role at Red Hat right now. I don't expect it's much fun.

I really worry about the impact that this has on goodwill. I think most people at Red Hat are also concerned about this. The ecosystem around RHEL will not be the same. EPEL, ELRepo, and all of the other communities that have flourished around the downstream builds are going to be impacted. Trust, when broken, is very hard to rebuild. Jeff's comments ("fool me once...") underscore the rift that is driving away good people. I worry about the long-term effects of this. I hope that this will not "kill Enterprise Linux" as many have suggested. I do believe, though, that it has been wounded.

Disclosure: I am currently employed by Red Hat.

I think that Redhat’s been failing at the PR part of their business. I agree with all of your points about their motivation and value. They’re a valuable part of the community.

From an outsider’s perspective, they were merged with IBM, “killed off” CentOS, and restricted access to the sources of their distribution within a relatively short span. It just looks bad and makes everyone feel like they have to justify that Redhat is still a good company.

From my position, I don't feel any IBM influence. In fact, if Red Hat were still a publicly traded stock, I believe the shareholders would have had much more influence than IBM is exercising right now. That said, from the outside looking in, I know that a lot of people have a negative view of IBM, and I can understand how they would come to the conclusion that this has something to do with IBM.

Fully agree on the "PR Problem". It's tough to "control the narrative" in OSS. Based on my observations, I think Red Hat's approach is: Deliver the facts, ride out the storm, pick up the pieces and move on.

> I believe the shareholders would have had much more influence than IBM is exercising right now.

Well, you answered yourself why it went as it did. Don't you think answering to your shareholders is what keeps the company competitive and on the right course?

Not necessarily. The economic situation has been tough for everyone and maybe the shareholders could have asked for reducing the work done on Fedora or Stream for a short term benefit?

I don't see any link between shareholder demands and business viability, other than "keep the business profitable". What makes you think shareholders are more familiar with the market a company operates in than the company itself?

In fact, with tech companies, the shareholders don't usually demand "profitable", they usually demand double-digit revenue growth.

Interesting you link to opensource.com, I read that RedHat fired the entire team behind that site during the recent layoffs. I see that they might have found an alternative funding source though:

https://opensource.com/article/23/6/new-developments-opensou...

How much is red hat paying kernel developers not directly on its payroll again?

They're the freeloaders.

The term freeloading is stupid.

There are customers who genuinely don't need and don't want support. And there are customers who do need support and should pay for it.

It's that simple. Don't bundle them together as all freeloaders.

Redhat offers self-supported versions though, for half the price.

> Copyleft open source licenses allow freeloading [1], but they also allow Red Hat to gate source code

GPL does not allow for gating the source code. Full stop. You are allowed to charge a nominal fee for the distribution of source code. For example, back in the day it may have to be mailed to you via floppy diskette or CD's so you would be charged for the media and for the time for someone to put that together. Walnut Creek used to sell CD bundles of all source code for something like $20 back in the day and you'd get several CD's of source and all the make files needed to build it.

>You are allowed to charge a nominal fee for the distribution of source code.

You added the word "nominal" but the GPL offers no guidance for how large the fee can be. RedHat appears to be breaking norms here but probably not breaking the license.

GPLv2: You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee.

GPLv3: You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee.

> GPL does not allow for gating the source code. Full stop.

RedHat is getting around this by saying "If you ask for it, we'll give you the source code for a high fee. If the source code we give you gets released to others, that's totally fine! We'll just immediately terminate all business dealings with you forever, if it does." GPLv3 has some language that very arguably might prevent this, but CentOS is licensed under GPLv2 and no one has found a clause which makes this behavior obviously improper, yet.

From GPLv2, section 6:

> Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein.

How does "we'll just immediately terminate all business dealings with you forever" not sound like a retaliatory restriction on the recipients' exercise of their rights?

It doesn’t impose a legal restriction on their right to do those things. It reserves the right to refuse service for Red Hat. Hopefully a lawyer can successfully argue your point of view and win in court but I don’t personally see it myself.

> How does "we'll just immediately terminate all business dealings with you forever" not sound like a retaliatory restriction on the recipients' exercise of their rights?

A enterprise has the right to refuse customers if the reason isn't discrimination of race, color, religion, sex, sexual orientation, gender identity or nationality.

"I violated a contract I made with them" isn't one of those.

This is like the tech communities version of the Ellen drama. Ellen is far from the worst person in Hollywood and Red Hat is far from the worst ISV, but because they’ve built their image on being the “good guy” anything they do contrary to that elicits a strong emotional reaction.

> I don't think this sort of emotional, loose-facted "hot take" style response is real constructive

I do. A basic pedagogic principle is that conditioning works better the closer the consequence is to the action.

They should be afraid of backlash at every turn.

People must be ready to actually walk away, otherwise you get into a Reddit-type situation in which everything ends up just like before.

> A basic pedagogic principle is that conditioning works better the closer the consequence is to the action.

> They should be afraid of backlash at every turn.

> People must be ready to actually walk away, otherwise you get into a Reddit-type situation in which everything ends up just like before

I actually agree with all of that, but I don't think the hot take gets us there. You can have backlash and people walking away, while staying fact-based and even tempered. In fact I think that's far more effective because then (Red Hat in this case) will see the logical connection. If it's just emotional stuff then it's easy to dismiss the quitters as irrational fringe people that aren't reasonable anyway (and I don't think that's far off from the reality).

> But that said, as far as I know, one guy used the term "freeloaders" and the context wasn't clear at all who he was talking about

One guy? The sentiment of CentOS users being freeloaders is shared by a lot of people, inside and outside of IBM.

I remember when the original announcement came out that CentOS was being killed in favour of CentOS Stream. Lots of IBM (Red Hat) employees were defending the change by painting CentOS users as getting a paid product for free. Lots of people here on Hacker News and Reddit and other places were defending it as well.

Don't try to rewrite history.

The sad part is the short nearsightedness. I know tons of RHEL engineers who only got into RHEL because they could essentially learn large parts of RHEL for free on CentOS and then go on to work professionally on RHEL. I also understand that a lot of courses to become official RHEL engineers/sysadmins use CentOS for the classes.

This is not true. Red Hat Enterprise Linux is currently available at no charge to students and non-commercial users for the use cases you mentioned. And it has always been available in that manner. It even comes with additional (paid if used commercially) red hat products on top of the OS itself.

I have a tough time believing they didn't know that considering red hat basically advertises training on the free version of RHEL. They want you to pay to take the RHCSA, which costs much more than anybody would realistically spend on a personal OS for learning.

https://developers.redhat.com/articles/faqs-no-cost-red-hat-...

The "free" RHEL licenses still require dicking around with subscription manager, and creating/maintaining a Red Hat account. It may not be much overhead if you already have paid RHEL instances, but for somebody who doesn't it's a stumbling block.

I think it's great that Red Hat offers free dev subs and I applaud that, but I think they underestimate the friction that creates.

Geerling is very good at getting his content to the top of lists. I had the same sentiments you have at the second post on this issue with Redhat. Occasionally, he'll post something that piques my interest, but most of the time his YT style "marketing" is just too over the top. I know he's doing it specifically to get those metrics, and I hope it works for him and the audience it gathers is what he's looking for. Since he doesn't change, I assume it is. I'm sure he sleeps at night just fine

YT style is well expressed. this social media style is click bait and rife of misunderstanding/misinformation.

this is how he is. this kind people ain't mine. the style of publications is YT based, clickbaity and aimed at social media: grabbing the attention and polarizing. this is why I was done with Geerling months ago. it does not help with the conversation/community.

It is exactly the right response.

I'm sympathetic to Jeff's position, but this is I believe the third post from this individual about this topic to reach the HN front page in the last week.

At this point I feel like there's no more blood to be wrung from this stone. Jeff is free to take his ball and go home with it - just like Red Hat, in fact - but I think we're past the "this is notable news" phase and another post from Jeff about his frustrations doesn't really further the conversation.

I'm pretty sure Hacker News isn't the target audience for this post.

It wasn't.

I usually write a blog post separate from the script of my video (this requires a lot more time, but I feel like audiences like HN deserve better writing than a YouTube script). But in this case I pasted the script almost verbatim (just adjusting formatting).

True enough. If it were I suspect OP wouldn't have glossed over the entirety of Red Hat's contribution as Red Hat would grab a copy of "Linux", add Red Hat magic sauce, RHEL comes out. Reminds me of the "how to draw an owl" post :-D

Personally, I don't mind it. It's a big deal in the world of open source, and as someone who was mostly uninitialized about CentOS and RHEL after all these years, I find these posts somewhat enlightening, even this one. I'll grant you that it's probably not necessary to have three posts or have them all go to HN's frontpage, but all-in-all it doesn't seem like such a big deal either. The people voted, I suppose.

Exactly, who is this Geerling guy anyway? ;-)

(I'm kidding, of course, but a few days ago I had to google him...I don't follow many Tech YouTubers' channels anymore)

To quote Jeff:

> Here's how it used to work:

> Red Hat would grab a copy of Linux They would add magic sauce that makes it Red Hat Enterprise Linux

> They would release a new version.

> They would update a source code repository with all the data required to build it from scratch

A few remarks. The "add magic sauce" part is gone, all the secret sauce, or black box machinery is out in the open in CentOS stream. Red Hat no longer does any RHEL development behind the corporate firewall, and does everything out in the open in CentOS Stream. There is one caveat for security CVEs, but effectively those go into CentOS Stream and RHEL at the same time when the CVE embargo expires.

What this comes down to is wilfully ignoring CentOS stream. All the clones of RHEL can simply track CentOS stream, and even replicate all the "Secret Sauce" that is the CentOS Stream CI/CD workflow. The process of replicating RHEL is so much easier today than ever before.

Two thoughts:

> The process of replicating RHEL is so much easier today than ever before.

Doing so bug-for-bug requires more effort than it did a few weeks ago. And they're basically doing that work in parallel with Red Hat itself (duplication of effort).

Also, making a change like this (forcing Rocky and AlmaLinux to change in the middle of the 9.x release cycle), when the previous change (killing CentOS) was _also_ in the middle of the 8.x release cycle, with not even 24 hours warning... makes me feel this change was not _only_ made "to make CentOS and RHEL development better."

If that were the case, the changes would be announced with at _least_ weeks (if not months) of warning, so the community could plan for it instead of getting blind-sided... now twice in a row.

Why do bug-for-bug with RHEL at all? Why not turn Alma and Rocky into stable distributions in their own right?

If this pushes RHEL into oblivion, why should anyone care?

The Debian ecosystem isn't centered on Ubuntu, it's centered on Debian. Comparisons between the two ecosystems are invalid because of this simple fact.

It would be nice to see the "Red Hat" ecosystem recenter around something which isn't a direct source release of a commercial product. Alma and Rocky should rebase on CentOS Stream for starters, and then try to get closer to Fedora as an upstream.

(BTW, I speak as someone who did corporate systems administration for years and understands the use case of replacing RHEL licenses with free CentOS installs perfectly.)

> Why do bug-for-bug with RHEL at all? Why not turn Alma and Rocky into stable distributions in their own right?

Because the whole point is for them to be 'compatible'.

Many of the users are running software that needs to be on RHEL/CentOS/Rocky/etc or it won't be supported.

Sure, there are users who just want a stable OS, but the real point of Rocky/Alma is to have a 100% RHEL-compatible, but free distro.

There are many levels of compatibility. ABI compatibility is perhaps the least stringent, bug compatibility is the most picky.

I'm not certain that much more effort is required.

Untraceable uploads with Onionshare over a Tor browser would conceal the origin of source RPMs that did not bear fingerprints.

Once captured, all that is really required is the updated SPEC file, and any modified sources or patches (if I remember my RPM internals correctly).

Alma could maintain such an Onionshare instance. If the SPEC and patch files are specifically GPL, I'm not sure that Alma could be compelled to forego them.

Let me preface this by saying, Jeff owes Red Hat nothing and he's free to stop "supporting" RHEL for any or no reason at all.

That said, "grab a copy of Linux" seriously, gloriously, hilariously handwaves away so many things that Red Hat does to create a RHEL release.

Red Hat pulls together hundreds or thousands of upstreams to create RHEL, participates in many of them, tests all that together, helps partners certify software against it, etc. etc. etc.

It's true, Red Hat doesn't "own" the Linux kernel, but it's done a ton to help develop it over the years. But RHEL is not merely the kernel nor any single upstream. RHEL is a product that comprises thousands of packages all tested together and then released as a supported product.

What Red Hat is trying to guard is not the source code to any single or even groups of projects. It's trying to preserve and capture the value it created from all those parts. Coincidentally, that value is what businesses, competitors, and the community are clamoring for and not the source code alone.

They want that single point-in-time snapshot that everybody agrees on as a de facto standard because the overall community has never been able to agree on another workable standard that would allow targeting applications across the board. And that standard has a name, and it's RHEL. And it belongs to Red Hat.

You can have all the pieces and assemble them yourselves if you like. But nobody is entitled to certify it as (officially or unofficially) RHEL except Red Hat.

If that angers you, I heartily encourage folks to build Debian up as the standard we all certify against. Or start your own business that overtakes Red Hat and earns the place RHEL has today.

Mark Shuttleworth took potshots at Red Hat's business model with RHEL for years, and it seems to me that they're doing a very similar thing now with Ubuntu Pro by holding back updates to packages after 5 years and charging for updates to the Universe and Main repos.

I think the main thrust of all of this is to de-commodify Linux. To make RHEL really be in people's minds it's own operating system outside of Linux. It's hard to do when it's really just a ball of generic open source components. Hence, this hostile behaviour. They don't want to be Linux, they want to be RHEL and ask not about the man behind the curtain.

Just the opposite, I think. Red Hat is fighting the idea that the operating system is a commodity, and demonstrating that the RHEL subscription does have value.

Red Hat's competitors have tried really really hard to undermine RHEL as something of value. The cloud providers like AWS and Azure provide their own base Linux distros to run workloads on. They'd love to cut Red Hat out of the picture. Oracle wants to convince customers to give it all their money to run workloads, hence Oracle Linux -- that's based on RHEL.

If RHEL were, as you say, "a ball of generic open source components" then you wouldn't see all this wailing and gnashing of teeth about making it harder for Alma and Rocky to claim "bug for bug" compatibility with RHEL. Nobody would care.

Note that this has been going on for more than 20 years, and this isn't the first time. I've been writing about this on my blog, but we've seen this story before and the only thing that has changed are the names of the vendors / projects and the version numbers. (See: https://dissociatedpress.net/2023/06/26/red-hat-and-the-clon...)

Not the opposite. You're agreeing with the parent that Red Hat is viewing to de-commodify Linux. You just can't seem to see why this a bad thing.

Red Hat does not have the interests of the community in mind any longer, which was inevitable when IBM purchased them.

The parent comment is a bit confusing in that it says de-commoditizing Linux, which is one thing, but then talks about de-commoditizing RHEL.

Yes, Red Hat wants to show that Linux can be treated as a commodity except when it can't; that the work on a distro can be separated into parts that can be shared with the community and parts that people are willing to pay for; that rebuilders are reducing the value of the latter and the only game-theoretical outcome is that no one will have the money to do this work.

Without going into the discussion of whether that's correct or not, it's not a particularly new stance. It was the whole point of separating Fedora and RHEL in 2004.

I agree with most of what you said, but:

> If that angers you, I heartily encourage folks to build Debian up as the standard we all certify against. Or start your own business that overtakes Red Hat and earns the place RHEL has today.

This strikes me as a variation of the "if you don't like Apple or Google's rules, make your own phone" or "if you don't like Youtube's content policies, then start your own video company." It's not necessarily wrong, but it's wildly impractical. It certainly may come down to that, but it would be far better for everyone involved to reach some sort of happy medium point that doesn't result in mass duplication of effort and fragmentation. Before telling people to just fork the project if you don't like it, it would be great to address people's concerns as much as possible, and that can't happen unless people make their concerns known.

It's also, as has been demonstrated, wildly impractical for Red Hat to continue exactly as before without others undermining its business.

The thing is that everybody is only lobbying Red Hat to make concessions, but nobody's going after Oracle or Rocky or Alma or other clones to say "hey, cool it. Red Hat's trying to keep a business going here. You're undermining their business. If you keep this up, they're going to make it harder for everybody to use a free RHEL for non-business situations."

The difference, too, is that I (probably) have a business relationship with Apple or Google if I'm complaining about their wares. When I complain about iOS or Android it's because I've purchased one of those phones (or want to) but it has a defect (in my opinion) or user-hostile anti-feature preventing me from getting the value I paid for.

This isn't that. This largely seems to be people who are not paying for RHEL demanding Red Hat allow them to continue not paying for it but receive its value.

They could also shrink down to a more manageable business unit... except IBM paid $34 billion for Red Hat, and you can't ever decrease revenues or the axe drops :(

Well, remember too that IBM didn't pay $34b for just RHEL - a big part of the purchase was OpenShift which has been coming up in revenue and has RHEL as its underpinning. And I have a lot more to write and say about that in a series for my own blog, but certainly IBM isn't looking for RHEL revenue to go down...

And to be very fair to all involved, that's not even remotely unique to IBM. IBM is a public company and the expectation for public companies is that revenue always goes up. Always.

Red Hat had to live with that as an independent public company too. If IBM weren't in the picture and Red Hat were reporting poor subscriber numbers for RHEL, investors would be calling for heads because late stage capitalism. There's no more room for long-term thinking if that doesn't involve "numbers go up, every quarter."

Yeah, and regarding OpenShift, I wonder how well that play has been working for IBM. My guess is they were expecting to 10x or 100x OpenShift revenues as giant corporations standardized on containerized infrastructure...

...but then Kubernetes (sans OpenShift) seemed to do the 10x'ing while OpenShift was just what RH customers used if they weren't already exploring Kubernetes through some other cloud provider's offering.

"I wonder how well that play has been working for IBM."

In January IBM reported that OpenShift is pulling in $1 billion in Annual Recurring Revenue [1]. OpenShift existed as a product before Kubernetes but was rebased on Kubernetes with v3. That was released (I think) in 2016? So that's basically 6 years from $0 Kubernetes revenue to $1 billion.

Red Hat numbers overall are sort of blended in with the rest of earnings for their group, I think, so I don't see exactly what Red Hat's numbers are now overall. When I worked for Red Hat they'd break it out internally but I wouldn't be able to share that unless it was reported publicly...

It took Red Hat something like 12 years from IPO to $1 billion in ARR, as an entire company. Not sure when RHEL alone became a $1 billion ARR business, but you have to remember that Red Hat's numbers included JBoss and other software too when they hit $1 billion.

As far as I know, none of the cloud providers break out Kubernetes numbers alone. I don't know, for example, how much AWS racks up via its Kubernetes. Of course, AWS also gets some of the OpenShift business since some of Red Hat's customers are on AWS, there's ROSA, etc. (Then again, AWS also gets a slice of RHEL too...)

Worth noting that Google Cloud posted its first profitable quarter in April of this year. No idea what Google's market share for Kubernetes is, but if I had to guess I'd say that their distribution isn't #1. (If anyone actually can accurately measure this, anyway...)

If IBM expected OpenShift to have grown beyond $1bn by January this year, I'd say that was unrealistic. I mean, obviously because it didn't, but also because when IBM acquired Red Hat officially the total Red Hat revenue was about $3.4 billion and if I'm not mistaken "emerging technologies" (including OpenShift) accounted for only $225 million. [2] If OpenShift was, say, $100m of that then they have (in fact) 10x'ed it.

And, finally, I don't think OpenShift revenues reflect whatever services IBM has packaged for OpenShift as a standard deployment platform. So - if IBM has made it easier to sell other things because they only have to target OpenShift, and they're successfully pushing OpenShift into major customers, then it's a pretty good deal.

But for now they also need to protect their primary Red Hat revenue stream, and that still appears to be RHEL.

[1] https://www.ibm.com/investor/att/pdf/IBM-4Q22-Earnings-Prepa...

[2] https://www.theregister.com/2019/03/26/red_hat_2019_results/

>There's no more room for long-term thinking if that doesn't involve "numbers go up, every quarter."

That is obviously unsustainable. I get that they need to make a profit but there's no fundemental reason numbers have to keep going up and at some point they won't be able to go up anymore, even if the business remains profitable.

Yeah it is obviously unsustainable, but nobody knows how long it can last, and they don't really care. Whoever gets to helm the ship when the end starts will get fired by the board, and somebody new will come on board. When they eventually can't grow, they'll start acquiring other companies and using that as their growth. Repeat forever.

> It's also, as has been demonstrated, wildly impractical for Red Hat to continue exactly as before without others undermining its business.

> The thing is that everybody is only lobbying Red Hat to make concessions, but nobody's going after Oracle or Rocky or Alma or other clones to say "hey, cool it. Red Hat's trying to keep a business going here. You're undermining their business. If you keep this up, they're going to make it harder for everybody to use a free RHEL for non-business situations."

You make some really great points here. I think you've mostly won me over.

> It's also, as has been demonstrated, wildly impractical for Red Hat to continue exactly as before without others undermining its business.

How was that demonstrated? RH became, famously, a billion-dollar company under the old model; why would continuing that not work?

CentOS never offered paid support. Rocky and Alma are trying to clone the whole package.

With that being said, I do think it falls into the category of "problems that probably wouldn't exist if CentOS was still around"

Rocky and Alma are new, but Oracle's been doing OEL for a long time. Although I'll grant that adding more players to the mix may have exacerbated things, or at least giving RH the impression that things were getting worse, which I agree is rather a self-inflicted problem since I suspect the logic was "RH killed CentOS, so we need to replace CentOS, which takes resources, so we'll sell optional support for our clones".

Yup, there is a market for cheaper support. The problem is RHEL licensing (price and hassle). If CentOS was around, RH could offer cheaper support for it. That was the solution to this problem, not kneecapping their platform.

> This isn't that. This largely seems to be people who are not paying for RHEL demanding Red Hat allow them to continue not paying for it but receive its value.

Well... I am not so sure about that. As I wrote above, it's actually easier to make a copy-cat of RHEL today by leveraging CentOS Stream than anytime before. So it's not about costs, but perhaps more about the inconvenience of switching off the old workflow for the new hotness.

Red Hat has no technical self interest in perpetuating an old legacy service considering that CentOS has itself migrated to CentOS Stream. Please remember that maintaining that CentOS git repo was really a self-serving activity, to help CentOS project rebuild RHEL packages using the clunky old workflow. In reality supporting the old workflow was never an act of charity, Red Hat did that for whatever business reasons following the CentOS take-over. Certainly the old workflow benefited other folks beyond CentOS community, namely the other copy-cat rebuild distributions.

And certainly, as you have argued, there is the side-effect of the old legacy service benefiting copy-cat distros, and by extension the cost-free user community. But as I argue, not necessarily opposing your view, but offering a counter point... it's more of an inconvenience that some folks seem to wilfully or genuinely misunderstand. The misunderstanding is that Red Hat is killing copy-cat distros, and they are not. Super smart & motivated people will still copy-cat Enterprise Linux, just like they always have. And certainly, Red Hat probably has near zero interest in helping any competitors rebuild packages, or compose the distro. But it's a stretch for folks to argue that ending one workflow in favor of another is slapping the faces of loyal free-loaders. The nexus of ideas/arguments from the one starting point to the wrong conclusion is just tenuous at best, so I'm not surprised some people would construct a straw man argument narrowly focused on short-term outcomes. Folks just need to get on-board with CentOS Stream, or setup some other way to rebuild EL packages. It's not that hard to reason about this topic.

> Red Hat pulls together hundreds or thousands of upstreams to create RHEL, participates in many of them, tests all that together, helps partners certify software against it, etc. etc. etc. > It's true, Red Hat doesn't "own" the Linux kernel, but it's done a ton to help develop it over the years. But RHEL is not merely the kernel nor any single upstream. RHEL is a product that comprises thousands of packages all tested together and then released as a supported product. > What Red Hat is trying to guard is not the source code to any single or even groups of projects. It's trying to preserve and capture the value it created from all those parts. Coincidentally, that value is what businesses, competitors, and the community are clamoring for and not the source code alone.

That's a point that (unfortunately) seems to have been lost along the way. A big part of the reason why people use(d) CentOS was because of the confidence they had that it'd be stable, functional, receive timely security updates, etc. And the main reason that was true is the work RedHat put into RHEL.

That said, it's not entirely a one-way street. The widespread use of CentOS meant much wider support for RHEL by open-source packages (and some closed source ones) than a locked down, limited availability RHEL would have had. I know of places that use RHEL because CentOS is widely available, and of places that would never bother to support RHEL if it weren't for the availability and (near) ubiquity of CentOS.

> What this comes down to is wilfully ignoring CentOS stream. All the clones of RHEL can simply track CentOS stream

You are willfully ignoring that CentOS Stream is upstream of RHEL. CentOS was downstream of RHEL.

Oracle Linux, Alma Linux, and Rocky Linux are all downstream of RHEL.

Feel free to argue all day about how similar CentOS Stream is to RHEL. Go ahead and argue how much it doesn't matter to you that it's now upstream.

The fact of the matter is whether it's upstream or downstream does matter to a lot of users and companies.

It does matter that IBM is trying to block access to open source code and is trying to restrict its subscribers from redistributing that same open sourced code.

IBM is causing extreme harm to the open source ecosystem that they profit from.

> The process of replicating RHEL is so much easier today than ever before.

I would argue that it was easier a few weeks ago when they made source rpms available publicly.

> [^1] Red Hat seems to be upping the limit to 240 sockets per developer as of this writing, but nothing official has been announced yet. https://twitter.com/fareszr/status/1673145072714665984

Hey I'm the one who wrote that tweet.

Mike, who first noted this on Mastodon, just updated his tweet (toot?) to say that his contacts at Red hat confirmed that this is a bug, the dev Subscription is still limited to 16 servers.

RHEL has a major ecosystem advantage related to drivers that they may not be aware of. They risk ruining this, as I will try to explain.

At the time industry started to take Linux seriously, RHEL was the dominant distro. As a result, and by accident, RHEL+derivs became the primary target for commercial hardware drivers. As an example - it's easier to get obscure low-latency network and packet-capture cards working on RHEL+derivs than on other systems. RHEL+derivs is the assumed standard when you talk to firms that make that kind of stuff.

As a result of driver support, redhat derivs became the standard for commercial deployment. Redhat did not get this volume. The deployed volume is with all the Centos/Rocky/Alma that is deployed in hedge funds, prop firms, oil search grids, etc.

As I see it, this driver situation is the real value in the Redhat ecosystem. The userland is a bit of a mess, the package manager does not impress me. But I deploy RHEL-derivs because drivers work with no hassle.

If Redhat were able to cancel the derivatives, they would trigger the tipping point where industry ceases to treat RHEL+derivs as the standard driver target. Firms using centos on five thousand servers are not suddenly going to start paying USD 350 / year / server where previously they paid nothing. They will move to Debian, and eat the pain of the transition. Commercial driver culture would swiftly follow. That culture change would not take years. It would take days.

I think Redhat's per-server sales model is naive. USD 350 / year gets you a license with no support. This probably gets some commerce from small-business, and a bit more from firms with traditional service expectations.

But is that really where the opportunity is?

When I set up a data-centre presence, I want to PXE-boot each server. This way I can modify the system image for the grid by creating a new PXE image and rebooting hosts. Getting to this setup is fiddly. I wish there was an off-the-shelf solution from Redhat that did a good job of it. USD 2000 per year per site per 100 servers, including support for the PXE device itself. Redhat could coexist with the derivs if it took this path. If they built this as an appliance, that product could become ubiquitous like firewalls and network switches.

If you don't mind, what makes more niche hardware drivers work / build better against RHEL?

I suppose RHEL uses basically the same kernel, with patches that don't alter its interface too substantially. So I presume that a driver in source form, or even partly in binary blob form, should build and work approximately equally well with any stock kernel.

Beside the driver developers apparently using RHEL / CentOS (so on them the drivers are known to work), what am I missing?

In my past experience as a sysadmin, some vendors will only release drivers (as binary blobs) for RHEL. EMC PowerPath comes to mind, granted that's kind of a moot point now that the FOSS kernel MPIO drivers are good enough, but I'm sure there's other such examples that exist to this day.

It is both drivers and the userland tools that you get in the tarball. This stuff is often developed by people with a focus on hardware. They will build something to work well on their machine but may never have used a different linux distro. The userland tools might assume compile options in libraries will be as they are in redhat. There will be hard coded paths where there shouldn’t be. The source may not be available, it might be static binaries. That kind of stuff.

> I suppose RHEL uses basically the same kernel, with patches that don't alter its interface too substantially. So I presume that a driver in source form, or even partly in binary blob form, should build and work approximately equally well with any stock kernel.

No, I think that's exactly the difference; Linux (in)famously has no stable ABI for drivers, and regularly makes changes that break things if you don't recompile against the updated kernel. Part of the value in RHEL was that they froze significantly more ABI surface, which made it possible to write proprietary drivers that would work across at least minor kernel updates without needing to be changed. That won't work on an upstream kernel because upstream doesn't go out of their way to freeze that ABI surface.

Yes, exactly this.

Maintaining any kernel modules that are not in the Linux kernel tree is extremely painful.

Almost every new kernel version breaks the older kernel modules, e.g. device drivers, by moving definitions between kernel headers, by adding or deleting function parameters, or by adding or deleting structure members.

Most of these changes are very poorly documented, so anyone who does not follow daily the kernel development is clueless for instance about which values should be put in the new function parameters or new structure members in order to obtain the same behavior as in the previous kernel version.

The kernel developers who make these breaking changes do not bother to write upgrading instructions for the benefit of those who must maintain an out-of-tree device driver, so those may have to waste a lot of time with reading the kernel sources, to discover what must be done to make the old device drivers compatible with a new kernel.

The kernel devs argue that the goal should be to get in-tree so these changes are taken care of for you by the people driving the ABI changes, and I happen to agree with them in that regard.

Being admitted to the tree is not treevial, so to say, and may take years for more complex stuff.

Nearly every Android device ever made runs on an outdated kernel that can't be upgraded anymore. From a platform perspective, I think they made the wrong choice when the majority of their install base has turned into abandonware.

Targeting a RHEL kernel version is quite different than targeting the same official kernel version as they backport a huge number of features, resulting in drivers that may work in RHEL but not upstream, or the reverse

In my experience RHEL has been better at backporting patches and maintaining a relatively stable kernel. This means that third-party software that targeted it is more likely to work over time. Canonical by comparison is absolutely terrible, and seemingly just blindly imports whatever is thrown over the wall from kernel.org even in their so-called "LTS" versions, and now after running an innocuous "apt-get upgrade" command your frobnitz device no longer works and/or you can't rebuild the driver for your frobnitz device due to incompatible kernel changes.

Some vendors also explicitly detail exactly which distro and versions they support ("RHEL or Centos x.y"), so even if you can get things to work on another distro, a vendor can easily deny your support request with the usual BS line: "that configuration is not supported. Switch back to RHEL x.y or SLES and let us know if you can reproduce the issue! La-la-la have a nice day, buh-bye! phone click".

The Software Freedom Conservancy legal analysis of RHEL and GPL that OP links to is the most interesting part to me. https://sfconservancy.org/blog/2023/jun/23/rhel-gpl-analysis...

Some discussion yesterday: https://news.ycombinator.com/item?id=36452101

Wishy-washy language there about skirting lines doesn't make that a very good analysis.

You can use others' GPL code in your products and charge money for it. You have to "publish" the GPL'ed sources (and any sources of yours that derive from GPL'ed code) on demand -- publish as in: if someone asks, you have to give it to them, but there's no requirement that there be a public download page or anything like that, and you can send the sources out in DVDs or flash drives or whatever media you like, and you can charge nominal amounts for the media. I.e., you don't have to make it easy to get the sources. And you don't have to provide the built product for free.

You can do a lot within the boundaries of the GPL. If IBM/RH does that, so what?

One can even do this sort of thing w/o GPL. For example, SQLite3 is in the public domain, yet the SQLite Consortium exists and makes what I imagine is good money for D. R. Hipp and his employees with zero competition from forks precisely because SQLite3 is very difficult to credibly fork, and SQLite3 is difficult to fork because the better test suite for it is proprietary and secret.

To a business, open source is a tool. Even for individuals, open source is a tool. A young person just starting out and with few resources might make useful software open source so as to gain notoriety and better employment, or even to start a business.

Yup. It goes from "These businesses are abusing the GPL and software freedom!" to "Well, they are following the GPL but that doesn't result in what I want!"

This is why we have GPL 2 and 3, for example.

Disclaimer: I work at Red Hat, but nowhere near RHEL.

it's not just an issue of what "I" want, it's an issue of what the GPL is intended to do. If it doesn't result in the four freedoms, but the GPL allows it, then the GPL has failed to do what it was designed to do.

From the text of the GPL itself:

> Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things.

RedHat is trying to sell software that is licensed under the GPL -- which has to be, because the base of it was written by others and the only reason RedHat is even allowed to use it or sell it is under the GPL -- and try to prevent users from having the freedom to re-distribute the GPL software they received, with or without their own modifications.

They are trying to subvert the GPL. They know they are. That's why they have all this language where they say they aren't intending to limit your rights under the GPL -- it's just that they'll refuse to have you as a customer if you exersize them, that's all.

Whether it is legal or not is a question for laywers. If it's legal, and it succesfully prevents users from redistributing GPL RedHat, then it's an accidental loophole, and has subverted the GPL's intent and design, as stated in the GPL itslef.

> and try to prevent users from having the freedom to re-distribute the GPL software they received, with or without their own modifications.

> refuse to have you as a customer if you exersize them, that's all.

refuse to -continue- to have you as a customer, that is. you are still perfectly entitled to the source code of, and distribution thereof, what you purchased.

The GPL doesn't create an obligation that you get continued access to future updates.

I probably should leave the conversation at this point (not because I don't think there's merit in discussing it with you - just don't want to muddy the waters).

you've fallen for the arguments and ways of thinking of your employer which is not a good thing.

> ... to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things.

when i'm not allowed to be your customer anymore i don't have "the freedom to distribute copies". you're putting a limitation on me - that's not freedom. or specifically, regarding "that you receive source code or can get it if you want it": i can't get it if i want it when you're refusing me as a customer.

or citing https://www.gnu.org/licenses/gpl-faq.html: > If you commercially distribute binaries not accompanied with source code, the GPL says you must provide a written offer to distribute the source code later. When users non-commercially redistribute the binaries they received from you, they must pass along a copy of this written offer. This means that people who did not get the binaries directly from you can still receive copies of the source code, along with the written offer.

> The reason we require the offer to be valid for any third party is so that people who receive the binaries indirectly in that way can order the source code from you.

so: there is explicitly no limitation allowed on the receiving of the source code. Red Hat is putting a limitation on it.

so for me it's clear that Red Hat is in violation of GPL, at least how the GPL was meant. maybe it's not legally waterproof because it's not formulated in a way that's withstanding a legal case against Red Hat but still: it's meant that way. and i hope and pray that in such a case the GPL (and every other license in use) is updated specifically against that (mis)use case so that Red Hat can't do such abominations.

> you've fallen for the arguments and ways of thinking of your employer which is not a good thing.

I recognized that as a risk, hence the disengagement, for sure. I also know that not just in matters like this, I am overly prone to analyzing and lawyering things, perhaps to too great a degree.

> when i'm not allowed to be your customer anymore i don't have "the freedom to distribute copies". you're putting a limitation on me - that's not freedom. or specifically, regarding "that you receive source code or can get it if you want it": i can't get it if i want it when you're refusing me as a customer.

I will say I absolutely get this perspective here and how it applies to the "spirit" of the GPL, and the way you phrase it is good in recognizing things as disparate but also intrinsically intertwined.

I will also say (on a personal level, I of course don't speak for RH) I think any such things should be addressed at the root, i.e. in the GPL.

> I will also say (on a personal level, I of course don't speak for RH) I think any such things should be addressed at the root, i.e. in the GPL.

Yes, but there's no time machines allowing time travel into the past.

> or citing https://www.gnu.org/licenses/gpl-faq.html: > If you commercially distribute binaries not accompanied with source code, the GPL says you must provide a written offer to distribute the source code later. When users non-commercially redistribute the binaries they received from you, they must pass along a copy of this written offer. This means that people who did not get the binaries directly from you can still receive copies of the source code, along with the written offer.

So this is very interesting. Say I want a copy of source code for X and you tell me you have it, but you're not an authoritative source of source code for X: how can I trust your copy, if I choose to get it from you?

Redistribution of sources is a nice right to have, but most people prefer to get them from authoritative sources as long as they're available. As long as IBM makes these sources available for a nominal fee, they're within the GPL, and people who want them should just go through that process.

> so: there is explicitly no limitation allowed on the receiving of the source code. Red Hat is putting a limitation on it.

IBM can't legally not provide a copy of the sources for a nominal fee. And they can't limit what you do with them as far as licensing goes, but it may well be that legally speaking they can terminate rights not under the GPL. Maybe this is just a loophole in the GPL. Maybe the courts will accept it, maybe not.

The GPL does quite intentionally not require you to provide a public download page (or git access, etc), and this was more or less discussed and a conscious decision by those who came up with the GPL, agreed. Albeit in pre-widespread-internet days when a "public download page" wasn't a thing yet (and the equivalents were much more expensive, thus why they might not have wanted to require that burden). But still. I agree this is more or less intentional and explicit in the GPL.

But that the GPL lets you take GPL software, add your own features on top, sell it, and tell customers that if they redistribute it themselves (as the GPL explicitly allows), you will refuse to sell them software in the future ever again? Nope nope nope. I think it's not entirely clear if the GPL allows that or not, but if it does, it seems pretty clear to me that it's inadvertant, and not what the GPL was intended to do.

At any rate, this is a different thing, and to me another level of shadiness, than simply pointing out that the GPL does not require an entity to distribute the source via public download links -- this is about the entity trying to prohibit users from re-distributing the source themselves, and the GPL really was intentionally designed to try and avoid such prohibitions, that's kind of it's whole reason for existing.

> But that the GPL lets you take GPL software, add your own features on top, sell it, and tell customers that if they redistribute it themselves (as the GPL explicitly allows), you will refuse to sell them software in the future ever again? Nope nope nope. I think it's not entirely clear if the GPL allows that or not, but if it does, it seems pretty clear to me that it's inadvertant, and not what the GPL was intended to do.

Does "Nope nope nope" answer the first question with "no", but then the "I think it's not entirely clear if the GPL allows that or not" answer it with "maybe"? Or was the "nope" think just a wish? I believe the GPL is silent on that. You might be right that this would be an inadvertent loophole, but as you know, it's not trivial to fix this sort of problem.

Nope nope nope means this is a different quality of thing, the things you are trying to put in the same category are in different categories.

the GPL, by it's own text explaining itself, is intended to ensure that users of GPL software always have the right to re-distribute it. It says that right in it.

If your point is that IBM has enough lawyers and money to prevent users from redistributing GPL software, and they may have figured out how to exersize a loophole that will be difficult to do anything about it, especially cause of all those lawyers and money... right, indeed, sure.

If you are suggesting that we should all consider this just fine, and you don't care what the GPL intended to do, and you don't think anyone else should either, and we should all aspire to make money by getting away with subverting the GPL... okay, thanks for sharing? But as for me: nope nope nope.

> If your point is that IBM has enough lawyers and money to prevent users from redistributing GPL software, and they may have figured out how to exersize a loophole that will be difficult to do anything about it, especially cause of all those lawyers and money... right, indeed, sure.

It's not. The that IBM has here is that they're doing is not lawyers and money, it's that their customers depend on them -- vendor lock-in if you wish.

> If you are suggesting that we should all consider this just fine, and you don't care what the GPL intended to do, and you don't think anyone else should either, and we should all aspire to make money by getting away with subverting the GPL... okay, thanks for sharing? But as for me: nope nope nope.

I'm suggesting something else entirely. First, that if IBM is legally right but you/we/they don't like it, there's voting with wallets, or sucking it up if you're stuck with IBM. Second, I'm suggesting that open source is a business tool for everyone from a poor individual to a multi-billion company -- a tool rather than an end in itself.

Now, I agree that open source as an end is fun ("look ma', what code I wrote!"), but people still need to put food on the table. Just like fine art, where artists want total freedom of expression, but often produce the kinds of art that will sell well because... it's nice to not be dependent on charity. Even back in the days when artists had patrons, they still had to appeal to the patrons' tastes, and if they wanted to revolutionize art they had to convince the patrons that that was a good thing.

If using open source while putting food on the table is incompatible with the freedom for users to redistribute open source software, than the aims of the GPL has failed.

Which is possible.

Where we disagree is that you are trying to summarize them all as the same thing -- either you are doing something that violates the license in a way that can be enforced in court, or it's all just the same category of doing what the license allows as a tool while putting food on the table.

The difference, I am suggesting, within that category, is simply that the GPL was very specifically designed to allow users of GPL software to redistribute that software without restriction, and Red Hat is trying to prevent this, while using GPL'd software.

It's as simple as that. This makes it different than just any generic "I'm within the letter of the license while trying to maximize the profit I can make from using this open source".

Whether it is within the license or not is not clear, only a court can decide.

Whether it violates the intent of the GPL is pretty clear, it says so right in the GPL.

Of course, nobody has to care about the intent of the GPL, but you don't have to consider open source an "end and not a tool" to care about the intent of the GPL. The choice is not just "I think open source is a political movement rather than tool", vs "I am fine with companies using GPL software to do things the GPL's whole reason for existing is to prevent, if that's what they need to do to maximize their profit, cause we're all just maximizing our profit here, whatever you can get away with is fine."

Open source is a tool, and GPL open source is a tool that preserves the right to modify and/or redistribute it, which is why some people choose to use it or license under it. I do understand that those rights are of no concern to you, right.

> people still need to put food on the table

IBM has no problem putting food on its table. It's doing so by using open source products to accomplish it.

On the other hand, IBM is trying to prevents others from putting food on their tables, using the same open source products.

IBM is painting themselves as the victim here[1]. You and many others have taken the bait.

[1] https://www.redhat.com/en/blog/red-hats-commitment-open-sour...

Don't forget that the FSF was selling binaries as a fundraising opportunity for a while as part of the Deluxe Distribution they'd sell for $5000, and before that Stallman was selling copies of GNU Emacs on tapes for $250.

Selling binaries has always been a business model.

selling binaries, yes.

Trying to prevent people from redistributing the source: no.

(Yes, in an internet world, it's hard to make money selling binaries if people can redistribute source, maybe. the FSF was selling binaries in a different context, for better or worse. You are still welcome to sell binaries. You are even welcome to charge for distribution of source. But the fundamental goal of the GPL is to give users the right to redistribute source without restriction. The FSF never tried to limit that.)

The IBM MBAs have finally taken over Redhat. Anything to squeeze another dollar out, even if it's counterproductive in the long term.

We knew it would happen. Honestly I'm surprised it took this long.

Said MBAs are in it for their annual bonuses. You'll see this in many companies today: go-nowhere/soon-to-be-canceled projects are launched -> bonuses are awarded -> people move on -> rinse and repeat. Only fools stick around for long-term gains. On the other hand - maybe the workers have finally cracked the corporate secret.

I hear a lot of this sentiment, but from what I see both internally at rh and news publicly available there is nothing to back up what you are saying. The decision seems to have been made by many long-tenured folk. So what if the opposite of what you’re saying is true?

Perhaps long-tenured folk that are now incentivized differently from how they were previously?

I work with some long-tenured folks and am myself one (at a different open source company) and let me tell you that incentives are not the same as they were two decades ago -- and we've not sold out to fucking IBM.

Redhat people, as good as they may have been in the past, are now IBM people and if they aren't operating like IBM people they will be replaced by people who do.

> Also ignore the fact that Red Hat builds their product on top of Linux, which they didn't build and don't own.

I feel like this ignores the fact that Linux's continued success is, in large part, a direct consequence of its ability to compete for government contracts by way of Red Hat pursuing compliance and funding development to that effect, in addition to a massive amount of Linux development that Red Hat does directly fund.

Like, yeah - they're not the only reason Linux is successful, but RHEL's success definitely played a big part in making it suitable for the datacenter.

No, not really. Ubuntu and Debian are far more prevalent in the datacentre than Redhat.

Linux became popular through grass roots deployments and then companies cashed in on that. There seems to be a new narrative that it was the companies that started the ball rolling but it's not so.

The companies absolutely got the ball rolling. Sure, the grass roots started the ball rolling, but that roll was not going to get Linux anywhere without the kinds of investment that Redhat and others brought in the second half of the 90s.

Depends on what you mean by "anywhere". If your only goal in life is to see Linux take over the datacenter, well you should be happy. But if Linux ok the desktop is more of your goal, then I think that the corporations have actually been a detriment because they prioritize changes that help the server ecosystem, even if it makes the desktop shittier.

Red Hat did way more than "cash in" considering how many open source projects it has created or funded directly or even indirectly by employing a very large number of their maintainers.

Some reasons why Stream is not replacement for Centos alike (Alma, Rocky, ...).

1. Stream can't be used as base to build el-compatible packages. There is no any guarantees Stream doesn't break ABI compatibility.

2. Stream has no large (several years) support cycle and can't be used as a stable system. Yes, there is a big community who don't need a paid licensed support from a RH and ready to help with bug reports and testing.

How long typically do packages percolate in Stream before making their way to EL?

The time delta to the next minor version, usually. RHEL has a release in the spring and the fall. The last pair of releases were in May and November. If something landed in CentOS Stream in between May and November, it will probably be part of the November release. If it's after that, then it'll be part of the next spring release.

It's not particularly complex, in my opinion.

> There is no any guarantees Stream doesn't break ABI compatibility.

This is incorrect. Or, sure, there are no _guarantees_, but any such break would also break a future RHEL release, and is therefore a bug.

2. Five years.

AFAIK, there is no rule RHEL is strictly based on Stream.

RHEL is branched from Stream and released from the branch every 6 months.

I challenge you to find _one_ package in RHEL (as of the git.centos.org c9 branch) that reverted an ABI breakage that is in Stream, without said reversion having been applied to Stream as well.

> Five years

So basically 2 times less, nice.

Isn’t that the same amount of time Rocky would be supported?

I 100% support and appreciate Jeff's (and everyone's) right to take a stance when it comes to how they spend their time. We have a finite amount of time available to us, and should spend it on what's important to us, especially as that changes. See Jeff's blog post on "Saying No"[1] that really hits home with why this is important.

What I'm having trouble understanding is the overwhelming use of IBM-related anecdotes (regardless of historical truthiness, and I'm not making any claim that it's wise or unwise to be wary - that's to each their own), reframed statements painted to appear like Red Hat's made brash statements about its community, and the general gall needed to make statements such as "tell your employees to stop [doing a thing]".

I get that this event may have felt like a violation of trust, and that a violation of trust probably hurts the most. To that end, I suppose emotional responses make sense. But it would have been significantly less cognitive load (on ones self, as an open source maintainer/contributor) to just pull your support and move forward.

[1]https://www.jeffgeerling.com/blog/2022/just-say-no

Not going to touch the hair on fire tone of this in general... but one thing worth mentioning is I believe the "freeloader" comment meant Rocky, Alma, Oracle and the likes are the freeloaders who are repackaging and RESELLING without contributing anything.

If you read between the lines in 2020, this was the next logical step coming. I think they should have done both changes back in 2020 and put a wider emphasis on the developer program with free subs, and removing pain in the ass subscription limitations which is why people want to use Cent/Rocky/Alma to begin with.

> RESELLING without contributing anything

I think you are underestimating the value of providing support to commercial users.

You mean, the valuable support they are increasingly shifting to Colombia, Philippines and India?

Who cares? Every country has skillful, qualified information technology individuals. Additionally, a quick google search shows it looks like RH does offer US only support if a customer is willing to pay.

https://www.redhat.com/en/resources/security-strict-data-han...

So, what Red Hat already offers.... Was Alma and co doing upstream development where they can prove it? Was Oracle patching upstream first or only in oracle linux? Was Rocky offering anything unique, value adding, or were they just copy pasting and selling?

It may sound loaded, but I'm genuinely asking, because from what I've seen the answer in regards to value add is pretty ambiguous.

https://lwn.net/Articles/929582

AFAIK Oracle employs major contributors to btrfs and XFS. I'm mostly interested in filesystems, so I can't speak for other subsystems (seems like they do a decent amount of work on core kernel development — the really important stuff like schedulers and the memory subsystem — second place just below Google).

Oracle Linux is also not a pure RHEL clone. They customize it as they desire and ship it accordingly. For example, they have an alternative kernel for their distribution with different features.

But they also have RHEL native kernel version for version if you're to choose to run that instead. It's just that RHEL kernel is usually very old and UEK kernel has more modern features and improvements in it.

Overall while they have some extra packages available they are on top of RHEL clone and you don't have to use them.

If you build your business on top of GPL software, then this sort of repackaging/reselling is just something you have to live with. Want full control? Build your own OS and license it under the terms you want.

Apparently not.

Is there anything to stop RedHat from pulling patches from Rocky? Is Rocky's code not available? Rocky isn't doing anything that CentOS (and RedHat project) isn't as far as I know.

The problem isn't Red Hat getting them, the problem is a Rocky "customer" would have to just wait for Red Hat to release the patch and then wait for rocky to re-build, and re release.

I was quite upset with the abandonment of CentOS. Now RHEL is pushing even more. But, let's forget the ethical quandary they walked into.

As others pointed out, this is self-defeating business direction.

Vast majority of government Linux systems in several countries was CentOS and RHEL. Lot of the improvement can be directly traced to FOSS improvements, outside of RHEL. Majority of the recommendations to use Cent & RHEL in government systems came from the users and developers.

It is clear today that users & developers (aka customers or consumers) can make serious impact to a brand by simply refusing to do business with the organization that upset them.

Pivoting from CentOS/RHEL to another distro is significantly easier then pivoting from a tangible product or store.

I'm surprised at all of the drama surrounding Red Hat's decisions. Red Hat is a for-profit company and not a charity. They (Red Hat) don't make any claims of offering RHEL for free to the whole world. If you want that, Canonical's Ubuntu may be a better option for you. Am I missing something?

You’re missing the fact that they are pedaling GPL software, and thus they are inclined to be offering it for free to the whole world, definitionally.

I'm asking this non-facetiously: Is there a wikipedia page or a written record of all the people/corps who've been taken to task in court over a GPL violation, or a large company such as IBM? If yes, was the penalty commensurate?

This appears to be a decent though seemingly incomplete list with outcomes

https://en.wikipedia.org/wiki/Open_source_license_litigation

Thanks, this is just what I was looking for.

It is incomplete however. For example, there was a long-running lawsuit against VMware that was never resolved. [1] Was sort of an odd case and pretty much no one else in the industry wanted to go near it.

[1] https://www.zdnet.com/article/linux-developer-abandons-vmwar...

Thanks for the pointer actually.

The source of GPL software can be sold, it just has to be available to users of the software.

> offering it for free to the whole world, definitionally

That's definitionally not true. Quoting https://www.gnu.org/philosophy/selling.html

] if you are redistributing copies of free software, you might as well charge a substantial fee and make some money. Redistributing free software is a good and legitimate activity; if you do it, you might as well make a profit from it. ...

] Except for one special situation, the GNU General Public License (GNU GPL) has no requirements about how much you can charge for distributing a copy of free software. You can charge nothing, a penny, a dollar, or a billion dollars. It's up to you, and the marketplace, ...

That's why the linked-to essays says "Technically, the GPL allows [a paywall]" from the text.

No, you are very much encouraged to sell GPL software:

https://www.gnu.org/philosophy/selling.en.html

No they only have to offer source code to users of their software. In order to use their software you have to buy a license. They have been giving it away for years but are not obligated to. When GNU first came out they sold their source code for thousands of dollars and this is the format GPL was written to enforce.

Auto makers, TV makers, and others use GPL software left and right and they don't, or rarely, offer a easy way to obtain it. How's it different?

You could run the things on CentOS just fine, but when you had need 'a supported solution' you just bought RHEL and did all the same things, just against 'a supportrd solution' and RH got their money.

> Auto makers, TV makers, and others use GPL software left and right and they don't, or rarely, offer a easy way to obtain it.

To be clear, if they don't provide source code of GPL software to customers, that's an actual outright violation of the license.

Correct. But don't you have to be an owner of their products to ask for the GPL'ed code?

No, anyone can definitely email them and ask for the code. If they refuse then thats a GPL violation and the copyright owner can sue them. The Software Freedom Conservancy are also working on a case using the legal theory that downstream recipients of GPLed code are third-party beneficiaries of the GPL agreement between the copyright holder and the redistributor, and as such, they are entitled to the source code and can sue for it. Hopefully they win, send them some donations towards legal costs if you want to help out.

https://sfconservancy.org/copyleft-compliance/vizio.html

> No, anyone can definitely email them and ask for the code. If they refuse then thats a GPL violation and the copyright owner can sue them.

Curious; I was quite sure that only recipients of binaries (basically, users of the program) were entitled to get source code, but the relevant part of the GPLv2 (https://www.gnu.org/licenses/old-licenses/gpl-2.0.html) at least looks like:

    3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following:

    a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, 
    b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, 
    c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.)

which does indeed seem to suggest that if you're not preemptively shipping source to customers along with the binaries then "any third party" can ask for the code. Which is interesting context here; it would be interesting to hear an actual lawyer's reading of the situation, because that feels like such a big difference that it should have come up already.

> Red Hat is a for-profit company

... built on top of software and labour contributed by others.

What you're missing is how the open source ethos works.

>... built on top of software and labour contributed by others.

the irony of Rocky, Oracle, and etc just flat out ripping off Red Hat.... which is the exact move that caused this change lol

If they sold for profit and started putting their changes behind a paywall, that would be terrible. But they're just benefiting from Red Hat the same way that Red Hat benefits from upstream Linux. Red Hat is the one misbehaving here.

If they don't like it maybe they should change their licensing

Isn't that basically all for-profit software though? not to mention exactly how most of the people on this board have paying jobs?

I would argue that it's exactly how the open source ethos works

It would also still go in the other direction as well would it not? Meaning contributions via paid Red Hat devs would still benefit the wider Linux community, even as they close off the free forked versions.

I'm not sure exactly how much Red Hat contributes to Linux though but if I remember correctly it's quite a bit. Maybe Red Hat making more $$ = more devs. Or maybe this is just a net negative for the ecosystem code-wise (as opposed to just hurting the current users of the forked OSes) as it pushes more devs/software away from a very popular 'platform', reducing exposure, free online support on forums, testing, etc.

Red Hat is a huge contributor to Linux. They also are responsible for making it a trusted OS for people coming off traditional Unix in the 90s. They aren’t doing anything on the backs of others.

Red Hat are relying on innumerable folks who do the work of certifying that their products work on RHEL, potentially creating lock-in for Red Hat.

Most of the code they ship isn't written by them, whatever metric you choose.

If that's the bar to being a good open source actor then who is?

You say that and it feels like Red Hat never contributed a single line of code to any FOSS projects or doesn't hire any developers to work on FOSS.

It doesn't really matter if they've contributed or not. They still need to abide by the terms of the license.

In what way is Red Hat not abiding by the license? My understanding is that Red Hat is making the source code available to their users via their customer portal. I don't think that they have an obligation to make their source code available to non-users.

They have an obligation under the GPL to allow end users to redistribute. Terminating the subscriber agreement for redistribution is punishing the user for exercising his right granted to him by the GPL.

I had missed this point earlier. Thanks for clarifying. This does indeed look like RH has put themselves into a quagmire.

End users are allowed to redistribute the source they received from Red Hat at their hearts content. What Red Hat is terminating, from my understanding, is the future access to new binaries and sources due to the service contract becoming void.

Your server will keep running, you'll have the sources for all the server's binaries, but no more support.

I mean, they’re within their right to pull support, but people are equally within their right to point out how that is shooting themselves and the RHEL communities in the foot.

This. What a good opportunity to compare: the (GitOps!) packaging workflows, build server security, software supply chain integrity controls, issue tracking / triage, wiki, documentation, kernel patching, cloud fuzzing / integration testing, and baseline MAC and DAC policies of the stable kernel patchset OSes within budget for schools, hobbyists, after workers, and corporations who can and for some services maybe should afford an SLA.

On worthwhile investments of time differentiating our offering in InfoSec and Operating Systems,

FWIU (RH) OpenShift (and MicroShift) does k8s containers most correctly in terms of separate SELinux contexts per container, which we should probably have for browser tabs, too. Do (a) browsers, (b) Cloudflare Runners, and (c) Docker WASM runtimes run WASM tasks without container-like process isolation; all as the same user and cgroup and context?

> which we should probably have for browser tabs, too

This would be incredible

It's pretty sad. Red Hat should be optimizing for "maximum number of dollars over 10 years", not "maximum control over RHEL-compatible distribution". The second goal leads to more support for their competitors, and ultimately less people buying Red Hat subscriptions.

Anecdotally, we'll have to support more Linux varieties instead of comfortably mandating RHEL-compatible.

None of the people who hold IBM stock will be holding it in 10 years. None of the executives at IBM will be there in 10 years. Why would they bother?

I don't think the author is expecting charitable behavior: it's more that Red Hat was deriving extraordinary amounts of value from his work, and was (partially) compensating for that by making his (and others') support for RHEL easier by providing them will access to RHEL.

In other words: Red Hat's behavior here is almost certainly going to make end-level support for their OS worse, not better, all for a tiny slice of their non-paying install base.

Red Hat offers free subscriptions to developers. How doesn't it help in this case?

Because its a pain in the ass and going along with it is essentially signing yourself up for the next time they want to try squeezing for money.

The sign-up path isn’t clear: When I signed up, it took a day for all the parts of Red Hat’s infrastructure to become aware of me. That delay wasn’t made clear at the time.

It expires every 12 months, and you have to take action to renew it, again with a not-very-clear path. It’s not possible to renew early (at least, I didn’t see how).

It adds extra friction: You don’t get a custom ISO or the like. Instead, you have to register your system during installation. It’s an extra step you don’t have to take.

There’s a subscriber agreement you must agree to, which not everyone wants to do.

It is worth adding that they can also take this way or reduce it at any time.

It isn't trustworthy in the same way that the downstream rebuilds were.

The point is that this is a stupid decision. It will turn people off Red Hat, and at the same time it’s incredibly annoying and inconvenient for a large number of Rocky/Alma/etc users.

> It will turn people off Red Hat

Will it turn paying customers off Red Hat? (Honest question)

No (paying Red Hat customer).

Most likely yes. Can't speak for other organizations but the one I work for licenses RHEL on our critical servers and uses CentOS/Alma on all non-critical infrastructure. Obviously the reason that is done is because it's easy to support what's essentially a single OS. With this new change, if Alma/Rocky go away we'll be looking at either Suse or Ubuntu most likely and RHEL will go away completely to be replaced with the licensed equivalents of Suse or Ubuntu so that we only need to support a single OS.

OpenSuse LEAP is 100% binary compatible with SLES. You can build an entire dev environment with LEAP, clone it and run a script to make it SLES.

Ubuntu is pushing snaps to hard IMHO. Suse is moving towards immutable OS with flatpacks, but doing so in a much more responsible manner that is not trying to lock down the ecosystem.

Current customers? Probably not, but it will hamper the flow of future customers for sure.

No, it won't unless the customer is also using upstream stuff elsewhere. But it will close an avenue of new customer acquisition. Can't say how often, but it happened that someone needed support for an upstream software (which is a big no-no) and ended up buying a downstream license.

Not directly but it might cause some upstream vendors and projects to rethink weather they want to recommend Red Hat as preferred platform and if the hosting vendors turn off the Red Hat clones that's going to ripple back to Red Hat, as a lot of corporate linux users will gravitate to the distro that's recommended by the application vendors.

Part of how canonical challenged Red Hat is that they deliberately made it really easy for a developer to run Ubuntu on their workstation and test environments.

I'm still waiting on their IPO to find out how well this strategy is working for them.

RedHat used to have a good open mind approach with CentOS, but then they got acquired by IBM and started switching their efforts somehow.

That cuts both ways; RH doesn't owe anyone free (gratis) software, but they're also not entitled to good will or free work/contributions. Or users or customers, for that matter.

The issue here is the GPL and what does it provide, freedom wise , for a system that is built of other contributors who also released their work under the GPL. It is a bit funny to me that red hat wants to kill off clones of their os and plenty of people think this is ok and seeing it as freeloading because red hat gives back which I would say they may be required based on the GPL and their business model. Could the Linux foundation move to subscription base and charge red hat based on they also give back and suddenly require subscriptions? This though isn’t the first time I have seen the same story so I am less worried. I remember red hat initially moving to their enterprise release and there was quite a few months of unknown what people would do after rh9 - I suspect this will also work itself out. If red hat wants to be more closed source they should build off of FreeBSD or something that can be less opened.

Genuine question (I'm not intimately familiar with all the terms of the GPL) -- does the GPL require you to release the source code to anyone and everyone (even non-customers)?

No but it gives every customer the right to redistribute the sourcecode to software they bought if that code is covered by the GPL under the same terms as RedHat got from their upstream.

It looks like RedHat might be trying to avoid that clause by threatening to stop selling any software to people who might use that part of the GPL.

I had missed this part earlier. Thanks for clarifying. It certainly changes the picture.

> Canonical's Ubuntu

Why the suggestion for Canonical?

They've also been doing shady shit again recently. For instance, adding advertisements for their products to existing cli tools.

I haven't been tracking SUSE for a few years now, but they're they're not doing shady shit? Hopefully there's at least one good option in commercial-linux-land. ;)

Probably because Canonical does promise to make their software freely available to the whole world.

Hmmm. "freely available to the whole world". Is that really the wording they use?

Asking because their recent advertising added to the apt cli is promoting some kind of security packages. Seems like people need to pay for those?

I believe that was for Ubuntu Pro.

Ubuntu Pro provides some extra support (five more years) for packages in Ubuntu's existing (gratis) Long Term Support (LTS) releases of Ubuntu.

Hmmm, not sure about that.

The wording was along the lines of "There are security updates for package XYZ. If you join <something> (maybe its Ubuntu Pro like you mention?) you'd have access to them."

That's not really a message that should be showing up on a box running 20.04 LTS, which is years before its EOL date.

Yeah, I saw it too.. we used a precursor to Ubuntu Pro so I'm sure my message was less obnoxious.

I think they're also offering patches for some commonly installed third-party stuff like nodejs via Ubuntu Pro.

I actually quite like Ubuntu Pro for the fact I can send a developer a laptop and know that there's 24/7 support from Canonical. I was a little dubious at how good they'd be, but they were able to diagnose the problem and provide a fix.

> I think they're also offering patches for some commonly installed third-party stuff like nodejs via Ubuntu Pro.

Yeah. That rubs me the wrong way. Like, they're clearly entitled to pay developers money for whatever they want.

However, they're paying developers to develop patches they're not sharing back with their upstream in a timely fashion:

1. Without those patches going through the upstream channel(s), there's no real mechanism to push back on patches and aren't good enough (for whatever reason)

2. It feels like they're taking advantage of the rest of the OSS ecosystem that is writing software / developing fixes and providing them in a timely fashion

Is there a free Ubuntu Pro clone, like Alma and Rocky for Red Hat?

No idea personally.

Perhaps the reaction has something to do with a product for which the USP is reliability and stability changing the terms on which the product is available very suddenly and without warning twice in the last few years?

The clones did contribute mind share to RHEL. That will now be lost. We don't know what the consequences of that will be.

PS: I think people should support Debian. A non-profit project with clear charitable status in many jurisdictions and a fully public development process. No sudden shifts if you use stable in production (or even oldstable).

Time to rally around the flag here. This is an incredibly hostile move by Redhat - and the attitude that only they provide value in the ecosystem is incredibly toxic and a direct threat to what open source stands for.

Without a doubt. Not sure if this is purely from IBM + Corp or Redhat sales folks finally getting over the whole idea of FOSS

Why not extend to Fedora? At the risk of starting a distro religious war…

After many years away from the RedHat ecosystem I recently tried to stand up Fedora 37 and 38 to test it with Willow Inference Server.

After hours and hours of poor and conflicting docs, random stability issues, various additional software repos of questionable quality, etc I gave up and proclaimed it officially unsupported.

Why anyone would want to waste their time being a tester for a commercial product is very strange to me when there is Debian and even Arch which (for a rolling release) seems to be substantially more stable, better documented, etc.

I understand why it exists and it has its place but absent a few specific scenarios I have no idea why it would be a first choice for anyone.

Fedora has been one of the most stable distributions I've ever used (I used it as my main system for ~6 years, but moved to another distribution now for unrelated reasons). It's also mostly developed by the community — Red Hat provides the infrastructure, but not much actual development.

Fedoras big strength is the update speed.

The problem with distros is that you're always picking an update cycle tradeoff. Debian has decades old versions of packages that they'll backport security fixes into till pretty much the end of time (RHEL also does this but is well, commercially backed while Debian is entirely volunteer driven, which has opened up Debian to maintainer shenanigans a couple of times).

Ubuntu is more up-to-date by virtue of simply importing the Debian backports repo every 9 months.

Arch Linux meanwhile runs on "whatever the latest upstream is", and with that you have pretty much 90% of the actively used Linux distros covered.

All of these models are untenable for someone who likes to stay up-to-date with the latest version but doesn't want to drop down into the occasional shell hell that Arch provides.

Fedora kinda... sits in the middle of that. It updates to latest version every 6 months, has a generally sane-ish release cycle that follows the most popular scripting language (Fedoras cycle is specifically staggered to allow for python releases to coincide with it every other release). That in turn makes it the best OS to recommend for Desktop Linux if you're new to Linux.

As for why not Arch/Debian/Ubuntu for beginners: Arch requires wanting to maintain the OS for the sporadic breakdown or strange default setting (not to mention the shell being an expected skill beyond running package management commands), Debian is too out-of-date to be useful and Ubuntu is only slightly better than Debian.

> Fedora kinda... sits in the middle of that. It updates to latest version every 6 months

It's even more in the middle, Fedora can update to the latest versions of packages as long as its not a major package and its an fairly standalone package. Fedora pulls new kernels as an example.

It's release timetable also suits GNOME, Fedora comes out about a month after every GNOME release, with that version of GNOME.

Decades? How many?

Just a quick note: debian stable has a new release every two years. So in the very worst case "decades" means like three years. The horror, the humanity, just remembering how software used to be three whole years ago causes me to shudder.

> just remembering how software used to be three whole years ago causes me to shudder

Well, yes. I agree with that statement even though you made it facetiously. Backporting security fixes only for bugs that have made enough noise to warrant it is a horribly janky hack. And, just thinking about all the time that has been, in my opinion, wasted on custom code that Debian maintainers have had to write in order to backport fixes makes me shudder.

Eh, there is huge demand for that static-ness, see windows ltsc, red hat, suse, debian stable, ubuntu lts etc. . Those customers actively don't want unstatic releases and pay lots of money for people to "waste" their time. So it doesn't matter whether it is a "janky hack" or what your personal opinion on that is.

That can still be ages depending on the maintainer spats.

When I say maintainer spats, I'm referring to shit like how avconv was given undue credence for ages because the split happened right around a Debian release and the packager for ffmpeg on Debian was on the avconv side, which caused a false deprecation notice to be present for quite some time.

And yes, with how quick some software moves, two to three years can very much result in a mess of hacks and patches.

> And yes, with how quick some software moves, two to three years can very much result in a mess of hacks and patches.

If you don't like the stable release, you could just run testing/unstable.

> Debian has decades old versions of packages that they'll backport security fixes into till pretty much the end of time

I don't know what filthy corner of the internet you get this misinformation from, but it is completely wrong. Please stop.

To be clear, I have nothing against them doing that, I'm just saying that this is how Debian generally operates. Their extended support model still allows for backporting fixes into jessie of all things, which released almost 8 years ago.

It's really good that they do that, I respect the niche they work in, but Debian has the nasty knock-on effect of having a lot of guides made for it by third parties that recommend really outdated versions of debian which can be a nasty suprise for someone new to Linux.

(It also results in notable forks like Raspbian having a poor update cycle - I remember in particular that it took about 2 years to move from the regular EOL of wheezy to jessie.)

> Their extended support model still allows for backporting fixes into jessie of all things, which released almost 8 years ago.

So your evidence that they don't update things is that they backport new fixes to an EOL version? Even if that made sense, 8 years is 0.8 decades; I think a lot of the pushback is that you said "decades old versions", which is to say "versions of packages that are at least 20 years old", which is wildly out of line with reality.

It's just rhetorical hyperbole, you're reading too much into it.

Rhetorical hyperbole that's inaccurate deserves to be called out; that's not really a defense.

If that is what you meant in your original post, then you presented it in a very manipulative, dishonest way. You alluded to Debian only offering decades old software versions and nothing newer, while the truth is that by default, users get fairly recent versions (e.g. KDE/Plasma is only one point release behind upstream), and one has to go out of their way and jump through hoops to get a version of Debian with software that is several years (not even a single decade) old.

This middle ground of sorts is what I was alluding to with "a few specific scenarios". From what I recall Fedora is the distro of choice for Linus? So it's clearly not completely terrible and I never said it was. That said, the creator of Linux is about as far as you can get from a typical/average user...

Per usual HN there are plenty of conflicting anecdotal reports and wildly differing perspectives on my position here and as I noted in another reply I absolutely could not care any less when it comes to people selecting the best tool for their purposes/opinion/experience/worldview. Not being steeped in modern Red Hat (I have an RHCE cert for RedHat 8 around here somewhere) but having decades of experience across many other distros the difference in experience as a "new user" with Fedora was striking. Also note that "new user" isn't exactly accurate here either - the install process was great, the UI is clean, etc. However, as soon as I got off the beaten path the quality of experience (again, for me personally) dropped to among the poorest I've seen in years with Linux distros.

Back to another anecdote for me, after decades I can count on one hand the number of times I thought "wow I could really stand to have a newer system python release" (as one example) - even when using pretty long in the tooth Ubuntu 20.04.

I'm not saying that doesn't happen and doesn't have it's advantages. It's a good point generally but I'm as confused as I am because it's never been even remotely close to a deal-breaker in my decades of doing all kinds of random stuff (from embedded to desktop to server to cloud). I don't know what people are doing to call them "too out-of-date to be useful" but again that's just me - I'm sure you and others have plenty of reasons for this position. I've just never encountered anything close to it.

Python specifically has such famous and nightmarish packaging and dependency issues I more times than not just throw everything into whatever official release Docker container for the version I need and call it a day. Great? No. Hacky? Yes. Practical? Absolutely.

So you tried to run your app on it and ran into trouble, and you conclude that Fedora is an unstable hunk of shit?

Also, this attitude that Fedora is just a tester for a commercial product is absurd. I suppose this is the same logic that makes CentOS Stream just a "beta" of RHEL (but strangely, these same people don't consider RHEL a "beta" for old CentOS even though consistency would demand this).

Anecdotally, I've used a lot of distros over many years, and Fedora has been by far the most stable and "just works" of any of them. There were a couple of rough years with the Nvidia driver, but I blame Nvidia for that.

Old CentOS wasn't downstream from RHEL, it was a clone. While stream is upstream for RHEL.

Yes it was a "clone", but it was "downstream". In fact Alma themselves just a couple days ago in their post pointed out the CentOS Stream is "upstream" while Alma is "downstream" of RHEL. CentOS always lagged RHEL by a little bit of time. Big security things it was usually days, but other times could be weeks or even months on new minor version increases. The only way for it be just a clone would be to near-instant rebuild, which has never been the case.

I agree with this assessment. Happy Fedora user here, but I am a bit nervous about the I Mification of RedHat and the potential long term ramifications for Fedora.

No idea why iOS corrected “ramifications” to “I Mification”. Too late to edit, though.

My conclusion was, essentially, that if I can't get our application to work with Fedora in a way that is practical, approachable, and sustainable for our "average" user it should likely be unsupported. I should also add I was working with at least one person who has a tremendous amount of experience with Fedora and they were taken aback by the whole experience as well. I should also note our application, specifically, (more of a long term support appliance) would likely represent an undue support burden because of the fairly rapid Fedora release cycle. As a random tangent, how is DNF /still/ so ridiculously slow? I don't get it.

Ubuntu/Debian is our main development platform so of course it "just works" there (in addition to various other Debian/Ubuntu derivates). However, when I repeated the same exercise for Arch (as one example) - a distro I have zero experience with - the process was completely painless and we had it implemented and documented in something like 30 minutes. Even though it is a rolling release the Arch wiki has substantially better documentation alone than anything I could find in the Fedora ecosystem (random hodgepodge blog posts for the last three major versions).

Again, I've been using Linux near-universally since 1997. I've had these debates (religious flamewars) plenty of times before and I have no interest in rehashing them. That said I don't know how anyone could debate my central point (let alone call it "absurd") - that Fedora is a testing ground for a stable LTS, commercially supported product (and one that with recent events is even more so). This is all right there, clear as day, in the official documentation[0]. So maybe not "absurd"?

I suppose I just don't understand the general preference for Fedora (other than momentum and familiarity) when there are plenty of other choices around where the primary goal is a free, stable, and open release.

On that note - IMO the best thing about open source is choice. Use whatever works for you, why would I care? I do not.

[0] - https://docs.fedoraproject.org/en-US/quick-docs/fedora-and-r...

Fedora is really great. I use it on desktop for over 10 years now. Very stable (like 2 issues in all that time). Much better than alternatives.

Same, I distro hopped for a while across various ubuntu and ubuntu derivatives, always ran into weird issues, stability problems, etc (and still do at work where we use ubuntu). Installed fedora eventually and have been using it ever since, outside of a packagekit bug like 7 years ago that didn't clear the cache and ate my disk space, it's been pretty smooth sailing.

Oracle appears to have substantial influence over AlmaLinux at this point:

"In the immediate term, our plan is to pull from CentOS Stream updates and Oracle Linux updates to ensure security patches continue to be released."

I would be curious to know if Rocky will do the same, and what this will mean for binary RHEL compatibility and the direction of these platforms.

Did you miss the part about careful curation to ensure it's still 1:1 RHEL and not with any extra Oracle stuff, etc.?

Oracle has no influence over AlmaLinux.

Sounds like it is time to fork and own it. Just as legal as what IBM is doing and people might vote with their feet rather than pay the IBM tax.

There's enough of a community to own this. Jenkins is a good role model. So is Libre Office. Oracle was once on the other end; now they are on the receiving end. So, I'm guessing they might be interested in being on the other end this time. Between them, Amazon, and maybe a few smaller companies there should be enough to pull together an independent fork, a and leave IBM to piece together long term support by themselves. The goal would be to have a stable long term version but I'd say IBM's involvement might be optional at this point. And what's left of Red Hat might be tempted to jump ship. Shape it like a foundation and make sure that there is no single corporate owner that can change their mind and you have a fine basis for decades to come.

Red Hat has even been making this easier for the last few years...

Back in 2003, when they discontinued Red Hat Linux (RHEL's predecessor) and locked binary RPMs behind a subscription (to similar outrage), the clones didn't have an easy time. RHEL's build process was behind closed doors and doing rebuilds meant reverse engineering the proper build order. This is why CentOS major versions took quite a while to appear after the corresponding RHEL releases (and made CentOS a major effort).

Since Red Hat assimilated CentOS, and culminating with CentOS Stream becoming the upstream for RHEL itself, the build system became unified.

This is highly valuable. That clones aren't taking advantage of this to jumpstart themselves into independence (and ensuring the Red Hat ecosystem has a future, with or without Red Hat) is just sad.

We should be calling Redhat what it really is these days - IBM. It’s not the old Redhat that many of us used to depend upon - IBM chipped away and it and pretty quickly too, they’ll run it into the ground and flog it off at a loss.

We started leaving red hat and red hat derivatives (centos) in favor of Debian when they were bought by ibm, assuming things wouldn’t turn out for the best.

While I was never a huge red hat fan, I do regret what’s happening now - I think that having a major corporate backer of open source code was very important. Other companies have since at least partially filled the gap, but still, I don’t like this. I guess that the red hat of the 90s is no longer the same company- time flies.

For all the RHEL to Debian refugees, here are some Ansible roles. I made the jump when they killed CentOS:

https://github.com/liv-io/ansible-roles-debian

I'm a "second generation refugee from XIV". XIV was a storage company started by Moshe Yanai, it pioneered many things in storage world. It was the catalyst that created many talented storage programmers that created their own successful companies: Infinidat, XtremIO, Weka, Elastifile, and many more.

IBM acquired XIV... and ran it into the ground. To anyone who had this kind of experience with IBM the possibility of RHEL dying under the guidance of IBM was almost as good as certainty.

My guess is that RHEL in not such a long time will take it's proud place somewhere between zLinux and AIX, and most will never hear from it again.

Why can't AlmaLinux and Rocky Linux drop their bug-for-bug compatibility commitment, and become stable distributions in their own right, based on CentOS Stream?

I think this ecosystem would gain a lot from these distributions having their own identity, instead of being just 1:1 copies of RHEL.

Change my mind. ;)

Many enterprise apps are distributed as binary release that compatible with RHEL, and people want to run them without using RHEL, especially in situations when it doesn't make sense to purchase RHEL licenses.

I understand that scenario, having lived in it for many years. However, in practice, that doesn't require bug-for-bug compatibility.

The proof of this is that such software tends to run perfectly fine even across different RHEL major versions with just a few compatibility packages.

I don't use red hat based distro myself so I'm curious: if you want to run an old binary built for, say, RHEL from 5 years ago, can you run that binary on centos stream without dependency issue?

Don't know specifically about CentOS Stream, but I used to run things like IBM DB2, IBM TSM, and a few other stuff that was built for RHEL 4.x/5.x on RHEL 5.x/6.x.

One thing that I like about RPM is how most packages specify dependencies based on contents, not on package name or version. So, even if a package changes name, if it still contains the required ".so.x.y.z" library, it will work.

PS: I don't understand how people can complain about rpm/yum/dnf, 20 years ago it was already years ahead of what apt/deb is today.

Well then these apps now have to support something else if they don't want to lose customers who don't want to use RHEL.

RHEL (and co) is largely a worthless trainwreck, technologically speaking. The values that the clones provide are "lets you run your old RHEL shitware from clueless vendors", and "lets you tell clueless customers that yes, against better judgement, they can run your software on their box-ticking RHEL systems".

Without RHEL compatibility, they are nothing. And without them, RHEL customers have no software to run and RHEL certification means nothing.

I'm done with everything related to Redhat after Redhat8(no it's not RHEL 8, it's their non-enterprise distro before it's ditched and Fedora was invented as far as I can recall), yes it's about two decades ago.

I since switched to Debian ecosystem, and never looked back. I was pissed off enough to never use any rpm-based distro ever since.

Yep, don't understand why the hate now and not back then when Redhat commercialised the hell out of everything. Even RH7 was bad trying to push their Satellite crap. However I note back then there was no Ubuntu and Debian was always lagging far far behind.

I'm running Debian for a week now and it works great. I only regret not jumping the RH ship earlier.

>Red Hat seems to be upping the limit to 240 sockets per developer as of this writing

-- EDIT: I read twitter, seems it may mean number of machines as opposed to open sockets. But a very confusing post.

I stopped with RHEL 2 months ago at work, I use Slackware at home.

What does that quote mean, if I get a RHEL developer license, am I limited to the number of open sockets ? That almost goes back to the per user licenses in the old UNIX Days. If I ever had to use RHEL, I would now say no just because of this.

> am I limited to the number of open sockets

I assume they mean CPU sockets.

I was thinking the same, if so that is a direct line to how AIX is provisioned, limitations on use and pay more to get more.

I the author of the post says, time stop RHEL support. I wonder if/when the FSF will weigh in.

>TYPO: I the author of the post says

I meant to say:

Like the author of the post says

In Red Hat Subscription Management I can see that each entitlement for the "Red Hat Developer Subscription for Individuals" product is good for a system with up to 128 sockets. And you get 16 (or possibly 240, or possibly this is a bug) entitlements.

It's now back down to 16, aww!

> I use Slackware at home.

Slackware was my first distro decades ago, and I have lots of fond memories learning Linux with it. How is it these days? What makes it your distro of choice at home in 2023?

Not OP, but: stable, with up-to-date packages. Configuration and maintenance does not change drastically between versions (there's a paper manual, it's on its second revision).

I can be heads-down on something for months and not run updates, and then run them and they apply fine and don't hose anything.

Long support for releases, both officially and on slackbuids.org

Plus, IIRC, Slackware may provide custom support for unsupported releases for a fee.

>If for some reason you are unable to upgrade or handle your own security patches, limited security support may be available for a fee.

From https://slackware.osuosl.org/slackware-11.0/ChangeLog.txt

From my end user perspective Slackware 15.0 works fine - up to date(ish) packages with what might be described as 'conservative' (old school) administration. For server use it might be best to have a look at the Slackware forum...

https://www.linuxquestions.org/questions/slackware-14/

and perhaps post a few questions once you have evaluated the basic OS.

Build scripts for software outside of base is at

http://slackbuilds.org/

There are various third party scripts for automating installation for software outside of the base.

Slackware ftw. Came back to Slackware on pretty much all workstations/laptops around 10 years ago. Some servers are also running it but mostly I've switched to OpenBSD.

Whether machines or sockets, it doesn't matter, 240 is an insanely large amount to be given out like that. I'm flabbergasted that people are complaining about this.

Red Hat is done with you also. IBM right now only cares about paying customers.

Shocking

And another comment section full of people who don't understand the GPL but are very confident about their wrong takes.

Agreed. I don't know why, but I expected more from HN commenters.

Would it be feasible to sed-replace the RHEL and/or Fedora selinux and container-selinux rulesets for use with other Linux distros?

AFAIU only SUSE can run both AppArmor and SELinux?

And browsers are running as unconfined in selinux with like all major distros; even on ChromiumOS (which was based on Gentoo, Gnome, and Chrome) where WASM or a paid shell (or 15-30% cut from the Play Store only) is the only way for the kids to Python on the Chromebooks we bought them for school.

Wouldn't it be great for them to not have to switch OSes and distros throughout the day.

I don't know about "sed-replace", but you can download the RHEL selinux-policy source package from https://gitlab.com/redhat/centos-stream/rpms/selinux-policy

I hope I'm not misunderstanding this, but isn't the current arrangement perfectly valid under GPL? GPL states:

- If I provided _modified_ binaries

- I also have to provide the sources

That's what's happening now for people that can download the binaries.

What happened before was:

- _Anyone_ could download the sources

(Not a GPL requirement).

So, now, IIUC if you can download the binaries you can download the sources. If you can't download the binaries (not a customer, for instance) you can't download the sources.

The contract termination is not related to GPL in this sense.

GPL grants you the right to redistribute those binaries and sources. The contract is terminated if you exercise your GPL rights.

One of GPL clauses is that you can't add more restrictions. Terminating a contract seems like a restriction to me.

Also, terminating a contract is very different from not renewing one.

Nah. GPLv3:

> You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License.

Given that there are a mix of licenses in use on Red Hat's distribution, they could be in a world of hurt. There's GPL, GPLv3, LGPL, CDDL, Apache, and so on. These all have different terms. GPLv3 in particular is the one that I think gets legally interesting, and I think it would be a neat court case.

Courts have recently taken up issues like non-compete agreements, and of course the enforceability of various contracts and licenses has always been a topic for interesting court cases. This would be the same way. Is the GPLv3 actually enforceable? Can you legally have a license or contract forcing action on the recipient of a piece of IP?

To say that RH did nothing wrong might actually come down to the legal fund at IBM of course, since courts are neither fair nor free.

The issue with the GPL isn't about being able to download source code. It's unambiguous that they are complying with that requirement. The issue is the sentence "You may not impose any further restrictions on the recipients' exercise of the rights granted herein." They're violating that by disallowing the people they do give the source code to from publicly mirroring it (on pain of terminating their contract).

Just gonna plug Suse that basically does what Red Hat + CentOS (not stream) used to be. Even more so because OpenSuse Leap is now binary compatible, not just source/ bug compatible like the RHEL flavors, the binaries in the Leap repositories are the same ones that are in the SLE repositories and you can switch from Leap to SLE with no issues. Amazing distro, it runs my home lab and would not change it for anything.

RHEL isn't a choice for a lot of us. If you work in finserv/government (especially DoD), many/most of the time RHEL is effectively the only option.

That probably won't change will it?

Probably not, at least not initially. Hopefully we see more usage of distros like Alpine out in the security field.

Why not SLES?

SLES is worse than RHEL in my eyes. Their support is horrendous, absolutely the worst in the industry. Their early move to btrfs caused so many headaches at client sites. I haven't trusted their technical decision making since.

If the SFC loses this case I wonder if this will lead to the creation and mass adoption of GPL v4 explicitly forbidding this loophole(and others which have popped up over the years).

I will never understand why someone would choose to use RHL over Debian flavored distro unless you want the enterprise support they provide. In which case, you reap what you sow.

Honestly surprised to see so much chat about RHEL here. I'm sure there are reasons to use it, but it seems so niche. Like 90% of cloud stuff is running Debian or Ubuntu.

Given their customer base, I certainly wouldn't be putting any special effort into supporting RHEL unless I was being paid for it.

How many times is everyone going to be so "shocked" when a corporation screws them over in a business descision?

Why would you have picked Red Hat in the first place?

Choose an independent distribution for more technical and end user oriented project trajectory.

Archlinux was the poster child for this until it basically fell to Red Hat with systemd adoption.

At this point Voidlinux is the most robust independent distro...

What about the small companies that started using CentOS then grew to needing enterprise support and switched to Red Hat? I'm sure they considered this before making the change, but I always assumed this was at least part of the strategy. Now there will be no free entry point that can be converted with a simple repo-swap.

Our team has seen 2 breaking changes in the past month from RH that caused problems in production. One in RHEL7 and one in RHEL8. It's definitely making us question the wisdom of the RHEL LTS approach. The LTS part is not what we're seeing.

Same, RHEL says you should be compatible against all minor versions and it doesn't matter if it's 8.5 or 8.6. Upgrades can happen without you noticing, but our software would only run on specific minor versions without investing time and development energy. Several installs I've worked on required an OS rebuild because the version crept into an unsupported minor version.

I don't mean this antagonistically and I realize I am lucky to have this perspective maybe, but all of this just reinforces a thought I've had for a long time, I feel bad for people forced to use these types of enterprise-y Linuxes.

There is nothing wrong with RHEL, it's all about the support contract (which you want or even legally need as a big company).

RH / IBM is trying to hit the partner companies of RockyLinux, AlmaLinux and so on. E.g. for RockyLinux there is a company called CIQ which sells enterprise support, which itself may or may not already be a dick move towards RH since they are chipping at their business without really contributing to it.

I've heard people on HN say you can't build a huge business off open source. That Red Hat was a one-off.

Maybe this supports the narrative? Assuming it goes poorly I mean.

Why? They did have a good thing going; that they then went and decided to shoot their own foot doesn't invalidate the earlier success.

I can't personally judge yet how much this will affect their business, bad for optics and community and all that sure. But regardless the narrative still implies they were already successful (aka "a one-off success"). Businesses rarely last forever. A decade plus of success is still a big success.

The point is more about the extreme difficulty of building for-profit businesses strictly off Linux-style OSS. Where people will pay for premium versions supported by paid dev teams. Otherwise it's just a glorified consulting business and not really an OSS software selling business.

I'd be interested to hear the motivations for doing so. Whether they strong downward financial pressure or were stable + merely wanted more growth.

Did they shoot themselves in the foot? It was a lot easier selling RHEL licenses when people were running on-prem workloads. These days the support contract many enterprises want is from their cloud provider, not their OS vendor.

Redhat will simply find themselves not supported if none of the open source projects can run CI against their distro.

Open source projects can get RHEL for free.

https://www.redhat.com/en/blog/extending-no-cost-red-hat-ent...

That seems like the same difficult process, just without paying. People need to be able to just get their docker image from dockerhub or similar

Video version of this rant:

https://www.youtube.com/watch?v=kF5pyVUQBH8 https://news.ycombinator.com/item?id=36488156

I havent seen a single server or workstation running rhel in the data science field, the most used distro in ds already was ubuntu and with the launch of pro covering the universe repository for security updates, I cant understand why redhat is making this move, cant see the endgame.

Glad to see redhat go down the shitter. I was never a fan, but that's not the reason to hate on redhat. Everything is behind a paywall or a license. They just seemed like another Microsoft, especially when they started licensing Linux on at a per CPU core level. I think the only thing going for it is their support, but in my 20 years in the industry I've only ever used vendor support like once or twice, I've usually overcome obstacles on my own. At a previous workplace we had Ubuntu without licensing or support. The company tried to make us switch our infrastructure to redhat because they wanted indemnification. So we instead purchased Ubuntu support/licensing and again - never once did we use it or need it. Virtually everything we built we figured out ourselves. So when it comes to vendor support, I usually don't give a shit.

My interest in RHEL+clones waned when a kernel update didn't play nice with ZFS, DKMS notwithstanding, and then dropped to zero when searches for configuration information began to inevitably lead to paywalled pages on Red Hat's site (also a problem for Wildfly vs. JBoss).

Vendor support is pretty crappy. By the time you're done gathering all the logs and running out the response time clock you've usually figured it out. Then at least half the time they do help, it's wrong because you're environment has some special requirement.

> I don't think this sort of emotional, loose-facted "hot take" style response is real constructive.

Of course not, but it gets you a lot of attention.

I think any GPL-licensee who charges for support is in no way encumbered by the GPL if they withdraw that support for any reason they choose.

What is the actual issue here? It's not explained well.

Is that the CentOS Stream is not a bona fide community distro, but simply a sneaky name for something that is actually "RHEL Beta"?

The article refers to subscription paywalls; so I'm assuming they apply to this CentOS Stream thing, which indeeded makes it totally-not-a-community edition.

Which makes it a dickhead move for it to be using the name.

Technically, it would way more sense for a bona fide community OS to be the upstream for the commercial one with the paywalls and subscriptions, than to be making community editions downstream. But it looks like CentOS Stream isn't it, so the point is moot.

> What is the actual issue here?

RHEL source code may now only be obtained via the Red Hat customer portal rather than pulled from Git.

This means that RHEL rebuilds (which _want_ to be downstream of RHEL) now have no way to obtain sources, since in order to use the customer portal you have to agree to not distribute the content downloaded from it to anyone else.

> The article refers to subscription paywalls; so I'm assuming they apply to this CentOS Stream thing, which indeeded makes it totally-not-a-community edition.

There's no paywall in front of CentOS Stream. Anyone can download it for free like any other Linux distro.

> Technically, it would way more sense for a bona fide community OS to be the upstream for the commercial one with the paywalls and subscriptions, than to be making community editions downstream. But it looks like CentOS Stream isn't it, so the point is moot.

CentOS Stream _is_ upstream of RHEL. It's far more of a community OS than CentOS ever was. Paywalls and subscriptions are for RHEL not CentOS Stream.

The problem is that the traditional CentOS community does not _want_ to be upstream of RHEL. But they also don't want to pay for it...

> They used to have edgy ads quoting Ghandi and ... It's funny they used to be so devoted to that Ghandi quote ...

It is Gandhi, not Ghandi!

oops, mis-typed that!

I wish I could get rid of RHEL. Forced to use it - and pay through the nose - with a state agency I'm doing contract work for.

Could somebody who maintains a particularly important open source software change the license to say basically “you must release your source code to anyone who requests it free of charge whether they are a customer or not”? Yes Redhat is a for profit company, but them deciding to make a business off of software that has licenses that might change to their detriment is their problem. Oh wait is that what they are doing to others now too? Well I guess that’s what happens..

Free has never meant monetarily free. Open-source means you can recompile, and thus maintain after the EOL of the provider, and code secrets aren’t secret. This is already a step above proprietary software.

For example: Wordpress themes are OSS. You must pay the gatekeeper, but you can inspect the code.

But free lunch is something else. Which OSS developers can also require, I’m not discussing that.

Open source means you can redistribute the code having received it. If you aren't allowed to do so, the program is 'source-available', not open source.

This is very enlightening! Thanks for sharing it. Ok, I can now see how RH might have backed themselves into a corner with the license.

Yea selling an entire OS that consists of a ton of other peoples open source projects—often paid for by other companies to have somebody work on a thing — then when people say “hey we should get a copy of the small tweaks you did to all of those bits” and they say no, seems like a situation where changing the license on enough critical parts force the OS maker to not withhold the software.

What would stop Redhat from saying “we’ve decided the only customers we care about will pay us 50k to be a customer, therefore anybody who doesn’t pay 50k cannot get source from us.. and anybody distributing the source code owes us 50k..”

> What would stop Redhat from saying "[...] and anybody distributing the source code owes us 50k..”

The GPL doesn't allow this.

When people buy RHEL what are they buying? I always thought that when buying RHEL you are buying their support, build infra, signatures, testing, etc. You're essentially buying the boring stuff- NOT the actual open source code which is available freely to anyone.

You can just as easily change your license to one which does not allow corporate use. This has the same result, the last previously licensed version of your project is forked by corporations. I.e. if your goal is to stop providing free use of your future work you're always welcome to it, but don't hold your breath that these companies will suddenly clamor back as a result.

It won't be exactly the same result. If the license is just not compatible with commercial use, the software just won't be used ever.

But yeah that clause would make it the same, because it's insane to demand that a company has to release all its source code just to use some open source library.

IBM trying to shake some loose change out of Red Hat users and copying Elastic's homework with the licensing switcheroo?

Will Red Hat start discreetly tagging source code releases per subscription to find out who is sharing source code?

If they haven’t already, I’m sure you could expect them to. That’s the only way to really police the policy.

I've read a suggestion that that actually would violate the assorted open source licenses in question, which generally require some version of giving the user the same source as was used to build the binaries they got. Of course, they might be able to just do it on the more permissively licensed packages.

So all it would take is a couple of separate releases of the SC, and you can diff to see the changes.

Yeah, a few well intentioned people could buy copies of RHEL and compare source code releases.

Nowadays I feel like a fool , why should I do open source contribution? To enrich these big corporation that thank you with that behaviours and pissing a you professional category ?

- They stole your code embedding it in close commercial crap without credits for the authors nor source code publishing;

- They stole your code using it to create AI appliance that are absolutely not intelligent but act like indirection layer to nullify the license and than they say that developer can be substitute but those "AI" appliances;

- As Mr. Geerling point out, now they shamelessly pass on the GPL license without fear of retaliation;

I hope that the community will do its move, first of all updating the GPL to explicitly forbid this loophole, I'm talking about paywall and illegal restriction of the source code distribution. Moreover , I think some kind of open source licence enforcement organisation should exist. Licenses without enforcement are hot air. For plenty of that organisation Open Source programmers are only simpletons , not dedicated peoples building their fortune. IMHO, I would like to see the possibility to bill for commercial use of open source code easily with license that explicitly cover this area and, again, organisation doing enforcement. Time is a priceless resource and IMHO is inconceivable that corporation etc reduce open source programmers to the role of slave workers of third world countries.

I think this is just the beginning of a whole wave of open source behind paywall type of deal. Ubuntu put the Amazon store app in their distro, for Christ's sake.

The only downside I see to this (I don't care to get involved in the ethics or centos or "freeloading" debates/conversation) is gatekeeping / whistleblowing may get much harder. Granted, I don't care to or have ever looked at RHEL source code, but there is a lot of independent observers out there looking at code that runs servers and desktop software for very important things. Projects in my area that come to mind are scientific Linux and those spins that were directly supported by fermi and the like, and were based on RHEL. That code got put on all kinds of servers and desktops in very important applications.

Fantastic opportunity for someone to found Blue Hat.

What are my alternatives?

People talk about how pre-IBM RedHat was somehow a good player. Perhaps that is true on the Linux side, where there were more eyes. Outside of Linux, I can say they were definitely toxic after acquiring JBoss, locking down previously open-sourced features, shipping bugs that could only be fixed if you had access to the enterprise contract, deleting content from the Wiki that they wanted behind the paywall of the enterprise contract. Even if some of that started pre-acquisition, it certainly accelerated post.

It's amazing to me that anyone upstream would lift a finger for RedHat at this point.

IBM behind the scenes squeezing every drop of value out of Red Hat…sad to see but this will continue

Can someone in Red Hat's engineering org walk into the CEO's office and bitch slap the common sense back into them? Bc I am sick of seeing Red Hat being sacrificed to the alter of "shareholder primacy."

Embrace, engulf, extinguish.

Looking at you as well, systemd and Pottering.

I am an embedded engineer and this act of engulfing is also stifling me and my effort to provide a reliable and easy-to-maintain Linux/GNU away from the likes of RedHat.

Is this a clickbait title? Or do I misunderstand the word enterprise?

They are not done with using Linux for "a project or undertaking, typically one that is difficult or requires effort." or "a business or company."

They are no longer going to use a specific version of linux that was made by "a business or company."

Do people really care that much about linux distros? Between 2 decades of personal use and a few years of professional Fortune 20 use, linux seems to be linux to me. (Could just be that I have stuck to Debian flavors)

Red Hat Enterprise Linux is the name of a product.

Geerling supports a massive collection of Ansible playbooks, which goes pretty far beyond anybody’s personal use of Linux and would explain why this is a bigger deal than you or I ditching support for a distro.

I've always heard it as Red Hat. Weird to see the parenthesis.

I would not have cared if it said 'I'm done with red hat enterprise linux'

I took it as 'I'm done with enterprise linux'

The parentheses means they're done with RHEL the OS and RH the company as a whole.

Geerling is a big name in Red Hat. He’s responsible for most of the popular Ansible packages out there.

> Do people really care that much about linux distros?

They sadly do. There's a whole wave of consultants and engineers tied to the RHEL ecosystem and wouldn't be able to install without typing "yum".

"sadly"?

Your second point is valid but it seems most developers today can't install anything that isn't an npm package

> "sadly"?

In response to the origin post that Linux is Linux. Kubernetes and Docker tries really hard to hide this differentiator of Linux distros.

> but it seems most developers today can't install anything that isn't an npm package

It gets worse than that. It's a daily struggle to explain the different between npm, yarn and pnpm for example. If only they understood what is "an npm package".

Was rpm -i deprecated ? /s

dnf, yum is slated to be deprecated.

https://access.redhat.com/documentation/en-us/red_hat_enterp...

Yes, we care about flavors of Linux in the enterprise. RHEL is an industry standard distribution that nearly all vendors support. We have a significant amount of commercial software that is only supported on RHEL or its derivates. Not running on a supported distro? You get no support. That’s a non-starter for mission critical systems in the enterprise.

We used to be able to run this software, fully supported, on CentOS. IBM pulled the rug out from under us. Rocky and Alma rose to the occasion, and the various vendors supported one or both, since it was still RHEL minus branding. Now IBM is cutting off those distros also in an attempt to force everyone into RHEL licenses and support contracts.

But because it IS Linux, after all, a lot of enterprises like mine have NEVER needed support for the OS itself. We just needed our vendors to support their software running on it, and we needed vulnarability patching on the OS, something which CentOS, Rocky, and Alma always provided.

If I had my way, 100% of what we run would be on Debian. I have Debian running everywhere possible. But not all 3rd party applications support Debian. I’m hoping that one consequence of IBM’s move here will be to drive them all to Debian.

Yeah, I'm hopeful the five year support window for Debian LTS releases now will help drive that migration. I suspect though the lowest-effort path for migrating from one commercial vendor to another will be towards Canonical, given Ubuntu's LTS releases and existing infrastructure around commercial support / certified hardware compatibility lists / etc. I may not be a huge fan of some of their decisions over the years but as my objections have tended to be technical rather than business/contractual/ethical I think they'd be nicer to deal with than IBM or Oracle, heh. :)

There can be quite a number of distro-specific quirks to bear in mind if you're targeting more than one, enough so that as a downstream product vendor it's nice to only have one or two ideally slowly-moving targets (say, RHEL and Ubuntu LTS) to test on/document/support instead of dozens. In turn it's also nice as an end-user to have a platform with long periods of stability/support so you're not having to rebuild/retune your systems' foundational bits every year.

(I have no particular dog in this fight. I mostly left the RedHat ecosystem behind a long time ago, but I understand it's still a big deal for a lot of vendors and clients. From an individual dev/user perspective, I agree that more or less as long as whatever is a posix-y environment, I can figure it out and get my work done, whether it's solaris or slackware or whatever haha.)

Reply on news.ycombinator.com