The future of AlmaLinux
almalinux.org
almalinux.org
Now they can actually be a real distribution and a true member of a community. They can make contributions, fix bugs, and add extras if they choose.
Now they will be “based” on RHEL ( ABI compatible ) but not “identical” per se.
This is what Red Hat wanted to happen. Hopefully this puts an end to all the Red Hat has gone proprietary nonsense as well. Pretty much everything Red Hat does will go into Alma as well. Alma will be built on the back of Red Hat’s efforts but it will be in a proper community way and they will share code and not just directly rip off the whole distro.
I literally love this take, because it can't be more wrong.
Most of the CentOS, Alma and Rocky Linux users used these distributions, because they were able to find their way and fix their problems without asking RedHat's help, and RedHat was explicitly saying that they're selling the "support", not the distribution.
Most of the people managed these systems had the chops to rebuild everything from bottom up, if required, and report the bugs the correct places to support everyone, including Red Hat.
Red Hat changed its stance overnight and decided to sell the distribution instead of support, closed the SRPMS they provided, flipped the table and screamed "Thieves!".
It's funny and sad at the same time. They started this model, lived in a win-win relationship for two decades, and changed their minds overnight like a drunk sailor, because IBM wanted more monies.
This will be very harmful to everyone involved in this in the medium-long term, because RHEL will lose the mind share which supported them on the enterprise space.
IBM's software products may be locked to RHEL in the short term, and a RHEL license have to be bundled with their sales, but it'll be a niche, dark room distro, not a well known one.
We'll see how Fedora will fare from IBM "Community"'s bright ideas like "opt-out before the first send" telemetry.
...continuing to be fair, Red Hat support has literally been the best I have ever used in my career. Hands down. I know I sound like a shill but I'm not. Just a reg ol' engineer. I've since moved on, like most, to cloud and past RHEL but it has it's uses. Cattle not pets for cloud!
> Finally, I wanted to say something about the term "freeloaders" I've seen many use it. This is a mostly internal term we have at Red Hat, it looks like at some point it slipped out in the public. So what does it mean? A freeloader is when a large enterprise business has 20 RHEL licenses, 150,000 community rebuild systems, and sometimes hundreds of user accounts and hundreds of kbase searches per month. It's not the enthusiasts, it's not the hackers and coders, it's not the academics, and it's not the people that use rebuilders because they can't afford it. We really try not to use the term, but when we do, it's about the large companies that can afford to pay but don't.
Why should RH help bad faith competitors?
Red Hat went as far as they could without breaking the letter of the GPL. They likely would have gone full proprietary if they could have legally done so.
Whether you see a problem with that depends on your values. A new school of thought has arisen, let's call it COSS - capitalist open source software, in which the purpose of open source is for large companies to flood the market with a loss leader to kill off smaller competitors, and for individuals to pad their resumes.
Under COSS, open source is merely another tool to feed the capitalist machinery, so Red Hat's strategy seems only natural. But many of us do not share these values.
But look how are the tables turned if it’s some other guy exercising his rights.
If we're talking about IBM/Red Hat here, they are not exercising their rights.
Can they close the SRPMs? Yes they can.
Can they tell "You can't redistribute this GPL licensed source code", haha, no. They can't even tell you to not redistribute BSD/MIT/Expat/WTFPL[0] code. They can't override these legal texts with EULAs.
GPL is working as intended, and irritating corporations the way it should, and yes, what Red Hat done is treason, not because they closed the SRPMs, but the collective things they did.
Isn't this limiting in a legal and practical sense?
How this is ethical, and allows unimpeded distribution of FOSS code?
Also, I still need to read and see the agreements myself.
See the 'termination' section of the enterprise agreements from the Red Hat licences page.[1] What constitutes a material breach is specified in Appendix I from the Product Appendices section, and it includes the redistribution of software obtained through the subscription (sections 1.2.f and 1.2.g of Appendix I).
> How this is ethical, and allows unimpeded distribution of FOSS code?
It is designed to discourage the distribution of the code while keeping Red Hat in compliance with the GPL. So it cannot prevent the customer from exercising their GPL rights, but a continued business relationship with the software vendor is not something that the GPL protects. PaX/grsecurity had used a similar tactic with their Linux kernel patches.
If they were only doing "what's required" they would have ditched their upstream first approach to fixes and the only ones seeing their code would be paying customers, or people who the paying customers distributed the code to. The fact that they are still one of the primary contributors not only to the upstream linux kernel but countless other open source projects pretty much by definition means that they are still going well above and beyond "what's required". If you don't like their direction, fine, but pretending like they are entirely freeloading and doing nothing for the broader open source community really weakens your position.
I guess the other issue would be that distributions like Alma or Rocky (and, to varying extents, their users) probably value stability and reliability too highly to be comfortable relying on such a janky form of source distribution.
Sorry, I fail to see how this addresses any of my points.
Rocky Linux on the other hand seems to take a more adventurous path by teaming up with SUSE on the new RHEL fork, whenever that comes. In the meantime it's not 100% clear how Rocky will continue working (they referenced some workarounds but not specifically how they'll get the source code from RHEL).
And Oracle will also work on their own fork.
Right...
SUSE says they will “fork” RHEL with it being a bit unclear if this will be a “bug for bug” copy or not.
Alma will base off CentOS Stream to create a distro that is RHEL ABI compatible.
Of all these, I think the Alma path best reflects the community values that everybody has been on about. The others are driven more by commercial interests than “community values”. That is ok, as long as you are honest about it.
Doesn't Oracle ship both the RHCK (a duplicate of the RHEL kernel) and the UEK (their own kernel)?
The first time I read this line, it didn't sit right with me... but after thinking on it, you're not wrong. I don't think Rocky's choice to stay bug-for-bug is necessarily driven by commercial interest, at least not overtly.
One of the big benefits of "bug-for-bug compatibility" with a support-licensed "Enterprise Linux" is the fact that you end up with an incredibly broad user base running the same systems code everywhere, whether or not they're paying for support. This means that non-licensed systems are helping to uncover bugs and incompatibilities, and it also means that issues reported by enterprise customers and fixed by RedHat get filtered out to everyone else as a matter of course.
At the end of the day, yes: commercial interest is the common factor underpinning this mechanism. However in Rocky's case (as with the original CentOS before being EEE'd by RH/IBM) the conscious motivation is to build something that brings a maximum number of people together in pursuit of a stable, production-ready Linux distribution. And even if the result is "we do this because it helps our users make or save money," as you said -- that's okay.
rpm -qp https://yum.oracle.com/repo/OracleLinux/OL9/baseos/latest/x86_64/getPackageSource/glibc-2.34-60.0.2.el9.src.rpm --changelog | head -n 25
There is no meaningful upstream link between Fedora and RHEL after a RHEL release has been forked off, so it doesn't make any sense to say that CentOS Stream sits between Fedora and RHEL.
Basically: Red Hat takes a Fedora release, heavily tweaks it, and makes a new version of CentOS Stream. After hardening, that version of CentOS Stream then becomes the basis for a new major version of RHEL. Thereafter, that version of CentOS Stream is the upstream for "minor" releases of RHEL.
Patches which are destined to land in that release of RHEL, go through the standard testing process, and land in CentOS Stream (aside from embargoed security fixes that hit RHEL first, then Stream). It's the one and only "direct" upstream of RHEL, and it's a very close upstream. Closer than, say, Debian Testing and Debian Stable. It's more like if you had Debian Stable and then there was some additional variant of Debian which just ensured that everything apart from security fixes was held back until fixed-interval point releases.
a.k.a. stable-proposed-updates
Debian Stable point-releases are not truly 'fixed interval' of course, but close enough.
Rocky Linux, as it is, will always be 1:1 with RHEL. We are dedicated to providing 1:1 "bug for bug" Rocky Linux, for both 8 and 9, for the full duration of their life cycles. Broken promises from CentOS are the reason we started Rocky Linux in the first place, we're not going back on anything we've already committed to.
I believe we (Rocky Linux) were pretty specific about how we're getting source in the announcement at <https://rockylinux.org/news/keeping-open-source-open/>. SRPMs from UBIs, cloud instances, etc. The only intentional omission are the additional legal loopholes we've discovered to get source, that we're keeping in our back pocket. We want to keep those backup methods to ourselves so that Red Hat doesn't close off too many at once, should they choose to try to screw over our community again in the future.
For some definition of community.
Adoption by many large organizations / businesses was also quite fast. And may have even caused us some trouble, I suspect the attention garnered by NASA's use of Rocky Linux had something to do with instigating Red Hat's recent antics.
Fascinating. No wonder why Red Hat got a little irritated.
And that I think I am finally understanding the differences between Alma and Rocky.
And what exactly do you mean by "builders' community", do you actually mean "Red Hat"?
By "not participate in" do you mean "not pay your employer"? Which we do, actually. Just wired a good chunk of money to Red Hat the other day, ostensibly earmarked for supporting the Fedora project.
Red Hat's action didn't screw over the builders community. It did complicate things in the short term for part of the users community, but at the same time it showed a clear way forward for builders that would not screw over users. Red Hat's intention is to stick to what they have been saying for three years and coalesce builders around Fedora ELN and CentOS Stream; not to screw over users.
Hence my original remark: if you think all Red Hat wants is to screw over communities again and again, your definition of community is not the same as mine.
> ostensibly earmarked for supporting the Fedora project.
Thanks for that. I never said you don't support the rpm-/Fedora-based ecosystem as a whole, and I appreciate that you also do so monetarily.
> I never said you don't support the rpm-/Fedora-based ecosystem as a whole,
Er. Really?
Maybe this dichotomy prompted the whole "hackers and hobbyists" imagined threat.
In the case of EL, Fedora and CentOS Stream are communities that office both users and builders, but the two groups congregate in different places. For example EPEL is targeted at the user community but it is outside the focus of the builder community. ELN is of interest to builders but it is outside the focus of the user community.
All users can choose to contribute (with bug reports, code, whatever) at either the upstream or the distro level; or they can just be users. Builders are a (very small) subset of the people who consume a distro, and like all users they can choose how much to participate in the community and whether to be contributors or not.
But that ship sailed, the good of the downstream users' contributions are not enough to cancel out the negative net value of the builders' bad behavior.
However: unfortunately part of what you said is true. Red Hat apparently took this decision because of some new facts that caused the negative from the "rebuilders" to increase, and the decision had the potential of hurting the users of the rebuilds.
Fortunately it's clear now that there are multiple ways for the downstreams to adapt[1]. Users will be able to keep their distros and—if they wish—contribute in a number of ways to the community. And builders and Red Hat will be able to collaborate in a shared place which is ELN for major released and Stream for minor releases. In a sense this removes excuses from Red Hat and should make everyone feel safer.
[1] you could say despite Red Hat's change, and I accept that. But it's also thanks to Red Hat going overall beyond the requirements of the open source licenses, with things such as Stream and the UBI source container.
This also decides the Alma vs Rocky question for me. Rocky wins.
Any Fedora/RH devs/maintainers/employees reading this thank you for everything! If any of you are ever in Berlin beers/cola/etc are on me!
P.S. for other commenters. A fork is NOT a distro. If someone forks a project (as the shape of a fork implies), they go their own way. So if Oracle and SUSE fork RHEL, good luck on standardizing "enterprise" Linux.
At least that's my understanding fromwhat I've read.
Weren't they spinning up virtual machines on public clouds, running their scripts, and getting the source from those VMs?
Now, this won't do down well for those who use RHEL-clones for the "free as in beer" aspect. They can choose Rocky Linux instead. At least now there's some identity separation between the two.
Most linux software is extremely portable and runs on probably all POSIX compatible systems already.
I never understood this need to make "bit-fot-bit" clones of one particular distro - RHEL. I've worked with "enterprise" Linux for a decade now, and I'm totally fine if my stuff is debian-esque or ubunt-esque. Sure, there are libcpp and other shared libraries that need to be compatible if trying to run the same binary of some programs on different distros but it's been a really fringe case.
So what's the point of running Alma then?
- Potentially hold back CentOS Stream updates to stay more in sync with RHEL. If RHEL is on X.Y, CentOS Stream is technically on X.(Y+1) - Alma wants to be X.Y compatible.
- Provide ABI compatibility with RHEL which CentOS Stream may not provide.
CentOS Stream by definition is ABI compatible with RHEL, and is required to be as such.
[1] https://www.reddit.com/r/redhat/comments/14yyf0k/the_future_...
Some of the tools I installed and supported are RHEL first and RHEL only software which require exacting ABI and even library versions to run correctly.
Debian is a great server, and has great support, but being one of the best doesn't always cut it.
These people either pay Red Hat good money for license, or pay good money for a sysadmin who knows what they are doing and install this software on a RHEL downstream distribution.
Sometimes you get a package deal. Hardware + RHEL + software package, which serve a group of engineers or researchers.
IBM is after this revenue. Universities and other research institutions won't care on the long term, because they can migrate piecewise, and group by group. They don't have an official favorite. They use whatever they like and works on their computers generally.
However, bigger research communities like national HPC centers, CERN, NASA, etc. used RHEL and alike, because they get unofficial, sometimes free or almost free support from Red Hat themselves, and they have a sizeable stack built on top of that.
They can and some of them will of course migrate, but it'll take time, and create pain down the road. We'll see.
They are still locking things like glibc, gcc, OpenSSL, LLVM, et cetera versions.
Furthermore, Red Hat is the top commercial contributor to the Linux kernel, and contributes more to the Linux kernel than SUSE and Canonical combined.
That's nice... but Redhat's entire business is also built on the open source software contributed by tens of thousands of others, much of it licensed to them under copyleft terms like the GPL. Yes, RedHat absolutely did contribute immensely to that ecosystem, but once they (or corporate-daddy IBM) decided to take a big fat stinky dump in the collective sandbox and stopped sharing their toys, the rest of us are kinda allowed to be pissed at them, no?
"We will give you the SRPM because we legally have to, but if you actually exercise any of the rights afforded to you by the GPL that software was licensed to us under we will immediately terminate you as a customer" *may* (a court will ultimately decide) fit into some legal loophole that exists in the void between contract vs. copyright law, but it certainly does violate the spirit of term#6 of the GPL, i.e. "You may not impose any further restrictions on the recipients' exercise of the rights granted herein".
RedHat pulled what is commonly known as a "dick-move". Ef them and the horse they rode in on.
I'm sorry, but isn't that EXACTLY what CIQ has been doing with Rocky Linux. Taking RHEL, rebranding it and then undercutting Red Hat on support contracts? Sure, it was always completely legal and even to some extend supported by Red Hat, but the support contracts is probably what drove Red Hat to make this move. What the hell would Rocky Linux even do if they where successful enough and Red Hat went out of business?
Just replace Rocky and Alma with Oracle in any of these complaints and all of the sudden people starts having less sympati.
If people are so upset, why didn't they migrate to something like Debian or SuSE when CentOS got axed? I really don't see why people want to stick with a RHEL clone when they're are so upset with the company behind most of the work.
Many people did. The remainder were too tied up in the RHEL ecosystem (including CentOS) to make the move, but red hat's recent shitcanery will push many more away.
As far as I can see Fedora and Stream still exist. You're basically talking about ~6 months of stable branch work that Red Hat would like to keep confined to their paid product.
Sure. The GPL doesn't have a carve out for "but we'd really like to not be open source for a little while". (And if you threaten your customers for actually exercising their rights, I struggle to believe that you're complying with the license.)
To be fair, nobody forced you to use RHEL. Nor is RHEL a prerequisite to using the open source software that RHEL depends on.
> That's nice... but Redhat's entire business is also built on the open source software contributed by tens of thousands of others... "We will give you the SRPM because we legally have to, but if you actually exercise any of the rights afforded to you by the GPL that software was licensed to us under we will immediately terminate you as a customer"
I'm sorry but your reasoning doesn't make much sense. We all should have the right to cease to do business with anyone for any reason, whether it's a good reason or not. I don't think that is a right that we should be so quick to abolish.
Your rights to the source are not being infringed. Yet you want to force RedHat to do business with you on your terms, and in so doing deny them their right to cease to do business with you for any reason.
How does their right to cease to do business affect your ability to distribute/modify the source? It does not, not any more than a pizza shop's refusal to continue to do business with you affect your ability to take home and consume a pizza you bought.
If, by analogy, buying one pizza gives you a perpetual and unconditional right to demand that you be served another pizza, I don't see how such a line of reasoning could be implied as a violation of your rights rather than theirs. Put another way, your rights to the source doesn't translate into a right to perpetual service.
This is a little tricky to define. By line count or commit count, it usually isn't, however the "top" contributors by those metrics are often hardware vendors committing large quantities of auto-generated hardware definitions in the form of header files, such as the AMDGPU driver. Red Hat also employs a lot of subsystem maintainers, who don't contribute much code themselves, but whose work is still important.
So there's no perfect way to decide who is "top", exactly. I think top 3 would be fairly uncontroversial.
I could choose Red Hat if I wanted to.
As I don't like software audits and license key activation, I do not choose it, as these are not a concern with Oracle Linux (unlike some of their other products).
Good news https://access.redhat.com/articles/simple-content-access
Counter offer: nah... how about I just target+deploy on any of the numerous competitors that don't make me jump through ANY licensing hoops whatsoever from here on out.
You went with the distro cloned by the company that perfected the concept?
In about 2007, we bought our first Red Hat to migrate functionality away from HP-UX. We eventually rolled the license that we purchased on a credit card into the corporate account.
In about 2009, "yum update" stopped working, and I called Red Hat, where I learned that corporate had terminated the licensing. Investigating, they advised me to run the Oracle converter, and resume pulling patches.
In 2013, corporate again switched to Red Hat. I did not. Corporate has been audited. I have not.
I'm using some btrfs loopback mounts with the UEK, and sales calls with Red Hat expressed extreme distaste for this.
At this point, why would I go back?
w/r/t btrfs - I recall when it was deprecated from tech preview in RHEL. I know some users really, really wanted it and it keeps popping up - Fedora is now using it, but I am skeptical that it's going to make it into RHEL anytime soon given that it was in RHEL as a tech preview and then pulled. I know SUSE supports it, I have a NAS that uses it, but seems like it's been found to be wanting by some of the folks who decide what gets into RHEL and doesn't.
For anyone still running RHEL based Linux after the last time RH pulled dumb shit, now's probably the time to move elsewhere. ;)
Another is to go with community projects where possible. If they are healthy and well-run, you should be in pretty good shape over the long run that they won't act against your interests. Here I'd look to Debian which has a very strong focus on governance and doing the right thing. (As they define "right thing" and it is well spelled out.)
Finally, do what's convenient and cheap with the realization that you will likely have to come up with a Plan B someday. This is sort of like the undertaker and Don Corleone: Someday, and that day may never come, you're going to have to pay a price and you don't know what that price is going to be.
I don't mean every open source project needs to use GitHub but some of these home grown tools are just really hard to use. The PHP bug tracker's main page doesn't even have a link to view 8.1 or 8.2 bugs but has one for 7.2.
Now I get 10+ emails a day related to some Openshift module I've never heard of, and the one thing I know for sure is that it a CVSS10.0 comes in nginx out I'll completely miss it in the noise.
Regardless, I think the answer is basically that you'd face an "xkcd 927" (i.e. "15 competing standards") problem
Now they will be CentOS which is already free.
I think it is a fair question if the new model adds value or not.
You now have much less supply for a common use case, but additional supply for a "new one".
Cloudlinux too will start patching on their own, so Almalinux might be able to source patches from them.
Rocky seems to have some sort of partnership with SUSE, it's not clear yet what they are planning, it might become the community version of the upcoming SUSE distro but that doesn't make much sense when the openSUSE brand already exists.
Overall red hats decision made the RHEL-Like ecosystem much richer with many more active parties than before, though I think Oracle Linux is the one most enterprise software is going to support, maybe SUSEs fork too if that works out well enough.
I do think that use cases for Alma users are slightly different, so not having bug-for-bug compatibility might not be a big deal if most of the installed software runs smoothly.
One of the original RHEL clones was Scientific Linux that was maintained by those organizations for their use.
They dropped because CentOS better at the time and they were doubling effort for something that was already available.
That’s why I favored it. Alma seemed to have more robust infrastructure and processes in place looking from the outside.
Rocky Linux didn't have secure boot out the gate because we built everything from scratch, which included going through the long process of getting our own secure boot key approved / signed by Microsoft. See https://github.com/rhboot/shim-review/issues/194 (Alma didn't get their own until https://github.com/rhboot/shim-review/issues/235)
> "AlmaLinux will no longer be aiming to provide 1:1 compatibility with upstream RHEL but will maintain a commitment to be ABI compatible with Red Hat Enterprise Linux. "
So, ABI (Application Binary Interface) means that the calls to the libraries and syscalls will remain the same. However they have leeway in the behavior between those points.
RHEL already makes this guarantees so it means they can 'build' at any of the patch points or sub-point releases. This is where they can differentiate their value.
It served as advertising for Apple and Google. All these newest gadget addicts were thrilled about having "Jellybean" or "Icecream Sandwich", while having no idea what it meant at all.
A place I worked for several years used such release names, which did not actually correlate with releases of the sequential numbers, but were used as catch-all bureaucratic keywords which were applied haphazardly to IT developments.
Just stop it.
I'm not sure why Alma does a new one for every minor release though.
Of course in Android's case this broke down when the marketing department started broadcasting the codenames, and even changing them. But I think they've got bored of that now.
says chair of the board.
isn’t that ironic?
Oracle obviously certifies its database on the Red Hat platform, and has licensed access to immediate source.
Until a grand rupture where Red Hat is decertified, Alma can combine updates from Oracle and CentOS stream.