Rakuten Mobile is replacing Red Hat as its Linux OS supplier
telecomtv.com
telecomtv.com
He seems to be pissed off that Red Hat discontinued the free CentOS distribution. And instead of paying for Red Hat, it goes to Rocky Linux, a free fork of Red Hat, just like CentOS. It is fine (at least legally), but the paid Red Hat distribution is no less open-source than Rocky Linux, in fact, the latter exists because Red Hat is open-source.
That's why I think this article particularly ironic, it says that Red Hat is not "true" open-source and then made a move that is only possible because Red Hat is open-source. It didn't even go to something like Debian, which is the most "free/libre/open-source" of the major distros, it went to the clone of the distro they criticize.
So why rakuten should pay Redhat for work not getting paid by Redhat? See the point ? Who is parasiting the ecosystem here ?
If they were using non-copyleft licenses then RHEL packages would be closed.
He doesn’t want to pay for the OS at all, he just wants to pay for support. That’s fine, but he’s still factually incorrect in what he’s said.
RHEL might be open source but it is structured (obviously) to sell the support license. You can't "just" install DRBD like on every other distro, you need to get the storage whatever the fuck pack, or get the packages from EPEL. In Debian DRBD module just comes with kernel, need to install it on RHEL etc.
It's annoying to manage even as paid customer and wholly worse experience than just installing Debian.
All that is mostly solved with CentOS. I really don't get the people so upset about it.
This is why we also migrating to Rocky on such systems.
And that's for my pets. The cattle is blissfully unaware of anything different.
My CentOS boxes are pulling straight from 9 stream. I had an issue with Grafana today I’m still figuring out.
If we decide we can’t afford 100% RHEL licensing, Alma was my choice, but I see so many choosing Rocky, and I’m genuinely curious why people are doing that.
The primary purpose of CentOS Stream is this: to allow a real community to be built around CentOS, where partners and users can contribute. With CentOS if you encountered a bug your only option was to file it against RHEL (not CentOS), wait for someone at Red Hat to get to it, then wait for RHEL to pick it up and release it. You would be very lucky for that whole process to complete within 6 months and if you're not a customer you probably won't be able to test a hotfix on your own systems.
Whereas with Stream: partners and downstream developers can file issues directly against Stream, and then contribute fixes directly to Stream, where they get QA'd in public and are added directly to the future release of RHEL. So Oracle Linux, Rocky, Alma and anyone else can actually participate in the process and contribute back instead of having to just take whatever RH throws over the wall.
However, the messaging and timelines around all of this were handled... not well, and some of the acrimony around that is reasonable even if the changes make sense.
In any case, I get the justification for CentOS stream, but changing CentOS 8's EoL from 2029, to Jan 1st 2020 ... after it shipped is definitely a good way to alienate your customers and community.
Fedora moves fast enough that by the time a RHEL release comes out, Fedora has moved far enough to not be very relevant as an upstream.
Even if Stream had versions, it’s still only tracking about a minor release ahead of RHEL. If you trusted RHEL’s QE process before, there’s little reason to stop trusting it. I really don’t think much has changed in terms of usage outside of some infrequently used software, but you have to account for that when managing them on a RHEL system, too, so I still don’t really understand the problem.
We have a couple thousand Alma 8 in production and it’s been smooth sailing.
> sometimes break things in unexpected ways
I have faith on RH to keep CentOS stream to be quite stable now that it runs on its own brand.
I can't understand how people had super faith on CentOS when it was run by a community which even had someone lock up the organization's fund/asset at some point.
People believe that CentOS was the same RHEL for free. Bit to bit. May be with changed logos.
So people believed that they’ll get rock stable system for free by using CentOS. They believed it so much that it hurted RHEL sales.
None of those beliefs were correct. But lost sales were real.
We’re a supercomputing center. We’re federated. Everybody is using CentOS. So, we also use CentOS on our pets sometimes because it reduces administration load.
We’re using it for years. We can patch it if we need it. We can submit patches if we need it. We use CentOS as a multinational community, which is completely different from “Hey, look! Free RHEL!”.
We’ll see where we gonna evolve. Stream? Rocky? Alma? It’s will be a collective decision. For some pets we migrated to Rocky, and we’re happy, for now.
Being founded by former CentOS founder also helped. MySQL guys found Maria, CentOS founder started Rocky. That’s OK in my book.
You're talking about a community problem (which most community have at some time and I don't believe for a second that companies don't have these kinds of events or worse) when people are talking about what they liked and actually observed to be good for them in centos. Yes, work should be paid. True also that some things can bring more customers without being profitable by themselves. Yes people are allowed to be angry about a bait and switch.
One can be sad and angry that a service that one used to rely on is discontinued or made very expensive.
Besides, do all those people complaining have projects that are very sensitive to stability that CentOS stream is going to ruin? That is quite hard to believe.
Not sure if it was even a bait when people just chose to use it and CentOS never locked you up in using it.
Most of the whining sounds very childish.
But still Rocky Linux is possible only because Red Hat provides sources for its distribution. There’s no Rocky Windows.
Just to clarify, CentOS is not discontinued. Basically all that is discontinued is CentOS "point" releases. There is CentOS stream 9, it gets updates, there will be CentOS stream 10, it will get updates, there will be overlap between CentOS stream 9 and 10 being supported. Sure the CentOS stream won't be exactly equivalent to RHEL point releases, CentOS stream 9 won't be RHEL 9.1 - but RHEL 9.1 will be cut from CentOS stream 9 AFAIU.
"Which are in large parts RH/IBM open source projects, too"
?
https://mobile.twitter.com/passthejoe/status/144693144711837...
- Several bugs have been encountered in Stream, which were not present in RHEL.
- Some user communities had previously expressed concerns on the feasibility of using Stream in a production setting. This feeling has also recently been expressed internally by some IT groups.
- CERN is evaluating our original recommendation [0] of utilising CentOS Stream as the default Linux Operating System.
[0] https://linux.web.cern.ch/#update-on-centos-linux-strategy
CentOS was never RHEL, it was though very close, CentOS Stream is also very close. Whenever I used RHEL or CentOS before CentOS Stream I always updated to the latest versions of packages for the major release version, if I use CentOS stream I will still get updates to the latest versions of packages for the major release version. Maybe I get them a bit earlier than RHEL, but this to me is somewhat irrelevant. If I want reproducibility there are still plenty of ways to get it, including RHEL UBI.
It was close enough, Stream is almost as close and for some people it isn't close enough when they need bit for bit compatibility for the software they are going to run
You seem to believe a community supported distro was as good as a corporate supported distro and put such a high faith in it but now somehow it's all corrupt when RH supports it putting it as an upstream?
If your usage is that critical, why not buy RHEL (or Ubuntu extended support) for 10 years support and stop freaking out like your Linux days have come to an end?
Stream also notably has a much shorter lifecycle, with support for 8 Stream ending in 2024, compared to 2029 for rhel/rocky/alma.
To me the main value of using rhel-based distros is the great reliability and long support life. Stream compromises on both of those compared to what normal centos offered, and rocky and alma currently offer.
Red Hat made some strange decisions with the way they’re handling Stream vs RHEL, plus the communication was abysmal, and they changed models mid-cycle. Be mad at them for that all you want, the rest was overblown.
This company could have just used Stream 9 if he wanted a free OS. Since he’s virtualizing, he could have bought hypervisor licenses for $4k a piece that allow you to run as many VMs as your hypervisor can handle. He could have used OpenShift for virtualization and container orchestration, and just paid for the OpenShift licensing. If you use OpenShift, you can run all the RHEL VMs and UBIs you’d like at no extra cost. What you never do is pay per core. There isn’t a single licensing model from RH that forces you to do that. If you license individually, each license supports one bare metal machine, or two VMs. This guy is full of shit. He’s just mad everything isn’t $0.
That said, I'm running CentOS Stream 9 right now. It's supported until 2027. I generally like to keep my operating systems relatively up to date, so a roughly 5 year support window is adequate for me.
People freaking out because their free candy is no longer as tasty as one could hope, it looks quite funny they have zero willingness to pay for such a long support effort.
Rocky is "down stream" of RHEL, CentOS 8+ is "up stream" of RHEL (since 2020, and what Fedora has always been).
CIQ sell support contracts for Rocky linux, so rather than having a combination of RHEL (with support) and CentOS. They now just have Rocky with an optional support contract.
Linux is open source, they are aloud to do this, thats the point. They are not leaching. Red Hat, by ending CentOS as a "down stream" equivalent, have brought this on themselves.
There will be a lot of other users of RHEL and CentOS who are going through the same process right now as CentOS 7 support ends next year. We will see more companies making the same choice.
Let me quote the article: “I called him and… said ‘listen, I want to give you a shot and I'm willing to work with you’. So we worked with him for 13 months to build our real-time kernel to meet complex workloads with one condition: It has to be open source and it has to derive its characteristics from a large community,”
Rakuten worked with the founder of Rocky Linux for 13 months to develop real-time kernel. I don't know what kind of deal Rakuten promised to Rocky Linux (founder) and its community, but Rakuten contribute by open-sourced the work to community. That's enough for me. That's the spirit of open source.
The people who actually understand and care about freedom is a small niche.
It seems like the classical "Here's our new pricing and for being a long time customer we throw in a bottle of vaseline too". Which technically has nothing to do with open source but Red Hat ( like every company in this space ) created an "ecosystem" and uses the word "open source" VERY liberally when selling you stuff. Some of their opensource software need some paid service or "maintenance" contract to work or have any value.
I'm sure Red Hat/IBM is ( mostly ) covered on the legal side, but the sausage is made in a very different way.
RedHat has consistently been the most aggressive supporter of Open Source out of any for-profit companies. Not only do they contribute to open source projects, they consistently release their own code/projects as open source early on, which could easily be kept proprietary (if they chose) and used as a competitive advantage over others.
This is true though as RH will terminate your contract if you redistribute their binaries. So they use a contractual backdoor to undermine GPL. And before people talk about how GPL covers source but not binaries, it covers binaries. In fact, all of CentOS, Rocky, Alma, etc. is a massive waste of everyone's time, energy, land, and electricity. Something akin to crypto mining. Everyone could have used RHEL, but we need this complicated indirection for no reason.
They were using CentOS for no cost previously and have switched to another no cost RHEL rebuild (Rocky).
Semi-related: now that RedHat is an IBM entity, with the attendant deprecation of CentOS, I fund myself lumping RH into the "fully commercial Unix" mental category.
CentOS is still out there. It's just that it does no longer follow RHEL on every step, but is ahead of it all the time.
Also, you can't use Centos anymore for test environments. You now have to go with Redhat and the entire licensing shebang when you init your test vms.
No it isn't. CentOS Stream will never make a change which would be out of place for RHEL to make, which means ABI and APIs (and version numbers for the most part) remain stable for the entire lifetime of the distro.
That is wholly incomparable with something like Debian Testing or Arch or Fedora Rawhide which "roll" through major updates all the time.
The source is open but so were many of the one commercial unixes. In fact some customers in the early days of Unix used to change their codebase a lot.
I don't think commercial and open source are mutually exclusive. Not the OP though so perhaps they meant something else.
Personally I don't think open source would have to be free as in beer but I don't like it when big tech takes too big a role in it. Eventually it'll just become another cash cow.
Incorrect.
https://developers.redhat.com/articles/faqs-no-cost-red-hat-...
Since they involved CIQ, I'm pretty sure they are now paying support for Rocky, which, if you read the article, was clear they didn't mind paying for. It was the per-cpu subscription that was not realistic in their business model.
They're in the "Backed By" section of https://rockylinux.org/about/ , so I'm going with "yes".
Giving back to the community could be both more or less than paying cash.
This seems simply another case of using something as the backbone of a business and demanding it to be free (as in beer).
Edit: This reminded me of another story. I once had a long fight with Gitlab's support due to an issue with their billing. They failed to fix it so now we run on the community edition which just works.
My preferens for free is due to there often being bugs in the billing/license checking.
Red Hat's business model was to both support and profit from open source. That is not an easy path, but they were mostly successful, at least until they were swallowed by IBM. If IBM manages to destroy Red Hat's value proposition, this opens a business opportunity for a company that takes a less antagonistic approach to its customers.
Is Debian-Testing any less "true Open Source" than Debian-Stable?
I wonder why is that, if Rocky is repackaging RHEL source performance should be identical, no?
Article mentions realtime kernel. It seems they were working with RH on custom kernel (that's the support contract he mentioned), but started to be asked to pay for RHEL license to use it, which understandably pisses him off
Would you be mostly guessing for things like build flags and optimizations? Could something as simple as package build order affect performance?
1) I am not sure Rocky were actually re-packaging the RT stuff originally, they've got it now though.
2) RT is pretty finicky to tune correctly for your use case at the best times, and especially for use cases as exacting as the 5G RAN. I assume there is a consulting aspect to this that they are paying the CIQ folks for, but probably based on man hours or a flat fee rather than the per socket/node subscription they might have paid Red Hat.
I mean, companies should. I have a couple ;-)
But they are loud and spend most of their time powered down.
The answer is simple, Debian. True freedom.
It sounds like Rakuten is paying CIQ instead of Red Hat on basis of a price increase from Red Hat.
Sounds like they worked with the developer of Rocky Linux for 13 months to get the realtime kernel where it needed to be, and I would assume the author did not work for free and was compensated by Rakuten. "We decided to move to fully open source software and paid the author to customize it for us and released our customizations as open source" doesn't sounds very leachy to me.
I mean, I guess he could have been working for them for free for thirteen months, but that seems rather unlikely.
I very much doubt Rocky is a sustainable business model. Hell even RHEL was not... This is the aftermath of the enterprise/FOSS (pay for support/whatever model that never really worked) war on prices and we are basically going back to the era for paying for software and some people and businesses dont like it.
Is your argument solely that Rocky Linux didn't extract the maximum amount of money they could from their client, or is there something else I'm missing here?
> I shouldn't be forced to pay subscription fees… I have no problem to pay support [fees]
We are breaking apart the traditional monolith deployments in radio base stations, that are deployed at a very large distributed scale. For example we are deploying 30,000 edge nodes to do real time processing of radio baseband signalling, 23,000 of them in the the next 10 months. Now these workloads have more in common with high performance computing than they do with running standard business application systems, so we are aware we have to modify the kernel to make it cope with the real time requirements. We are removing the monolithic model because we know we can reduce the costs and increase the deployment optionality. And the existing enterprise OS licensing model was prohibitively expensive at this scale. And then our open source choices were changed. We are happy to contribute our real time learnings back to a community since that helps us help the community to help us. What we have done is not for everybody, and others need to make their own choices for their own reasons. But we believe it is having those choices is what creates a healthy industry. And we no longer had the choices we wanted, so we want to contribute to making them for others.
I don't recall having problems with Dell server-related packages on Debian, even if their own crapware (OpenManage Essentials and then Enterprise, legitimately one of the worst pieces of software I've ever had the displeasure of using) for managing iDRACs and firmware came as a CentOS-based appliance.
It's still a frighteningly low number of distros considering we used to have an abundance of distros to hop around on for desktop use. Distrowatch.com for example, but for all those distros, when it comes to servers we narrow ourselves down to less than a handful.
Support you how and with what? What does your storage vendor have to do with your server's distros? Even the distro running on the server itself doesn't matter, really, because it's not like Dell/HP are writing distro-specific userspace drivers.
Operating systems have a hardware compatibility list (HCL). If your product is not on the HCL for the operating system vendor and you have a problem--at least one somehow plausibly related to the interaction between your product and the server--neither vendor is obligated to support you. (They may well try but there's no guarantee.)
I don't care if DistroWatch shows bunch of random newbie friendly distro in the upper ranking, they're not corporate supported.
Debian doesn't do clean 2 years LTS release cycle nor has 10 years extended support.
I have seen Gentoo used as a base to deploy binary packages veted by the org in the past. Gentoo provides tooling to do that and it worked fine.
Oracle also still sell a distribution.
ArchLinux, something I would never use professionally but enjoy on my machine. Packages are mostly vanilla and are updated quickly.
The last software company I worked for used CentOS.
Debian patches a lot for completely bonker reasons. They do it to support architectures nobody uses. They do it to remove part of packages for purely ideological motives. They do it to split packages into multiple parts making work harder for upstream when bugs are found. They do it to integrate with their own subpar tools. Sometimes they introduce major bugs along the way like the infamous OpenSSL one.
I understand people liking the fact that Debian is community maintained but the security track record is simply not there.
>On May 13th, 2008 the Debian project announced that Luciano Bello found an interesting vulnerability in the OpenSSL package they were distributing. The bug in question was caused by the removal of the following line of code from md_rand.c
> MD_Update(&m,buf,j); > [ .. ] > MD_Update(&m,buf,j); /* purify complains */ >These lines were removed because they caused the Valgrind and Purify tools to produce warnings about the use of uninitialized data in any code that was linked to OpenSSL. You can see one such report to the OpenSSL team here. Removing this code has the side effect of crippling the seeding process for the OpenSSL PRNG. Instead of mixing in random data for the initial seed, the only “random” value that was used was the current process ID. On the Linux platform, the default maximum process ID is 32,768, resulting in a very small number of seed values being used for all PRNG operations.
The most security focused uh? the distro where a maintainer modifies code he doesn't understand because valgrind complains about it? In one of the most important packages of a linux distribution? No. Anyone who has had memories of this event would treat anything stamped with debian's name as nuclear waste.
Debian has had many conflicts with upstream projects for their relentlessly idiotic patching. Firefox once prohibited Debian from using the FF trademarked name and debian in turn renamed firefox "Iceweasel" in their distribution to continue shipping their tainted version.
I'd trust any of these : RHEL/Fedora, openSUSE, Gentoo and Arch over debian any day.
To be fair, the maintenir had asked about the change on the project mailing list and was given the go ahead.
I too don’t like the general attitude of Debian towards patching but it’s an issue of the whole project. There is no need to single out someone.
AFTER CentOS came out they changed it from 2029 to Jan 1st 2020. Now CentOS stream is not a recompile, but has less tested changes BEFORE RHEL, and is now a rolling release with a 5 year EoL.
So for those that need compatibility with RHEL, that's broken.
Those commits are mostly from Red Hat developers, because all RHEL devel now happens in Stream first, but iiuc technically anyone could submit a PR, and maybe CentOS Stream community members do that from time to time.
Fedora is entirely separate packages from CentOS/EL. Fedora is much closer to upstream projects. Fedora has shorter lifecycle, no API/ABI guarantee across releases, moves fasts and breaks more things, etc.