Hacker News Reader

Comment by verdverm | Hacker News Reader

Not sure these "options" are going to satisfy business risk analysis. We are looking into moving away from all things CentOS, RHEL, or projects like Rocky. It was going to be Rocky, but no more. Red Hat got blue washed and is now ruining their reputation

View on HN

My impression from being somewhat involved in the (early) communities of Rocky and Alma, is that the Rocky people are quite a bit looser with violating EULAs, ToS, etc. Sometimes that is the superior approach, but with something like RHEL rebuilt it makes me nervous. Alma by contrast are very professional. Rocky also has an antagonistic view toward Red Hat, while Alma views Red Hat as partners. People can hate on Red Hat, but without them none of the rebuilds would even be possible. There's still a lot that Red Hat could do to kill them if they wanted to.

Anyway my main point, I'm not worried about Alma getting wiped out, or about them making a bad decision that ends them in legal hot water and jeopardizes the project.

Agree, Alma's response has been way less hacky.

Interesting... Thanks for the info, both of you, will have to take a look

I don't understand all these CentOS forks: you're still dealing with Red Hat and reinforcing their ecosystem for free. What if those forks poured the effort behind Debian? A truly democratic distro that cannot pull rugs from under your feet.

If you want Debian, just use Debian. The idea behind CentOS and its successors is to have a free distro that is bug-for-bug compatible with RHEL, so that you can support users of your software who are on RHEL, or use software supported only on RHEL for free.

That's the idea, but Red Hat is making that ever more difficult. And I can understand why. Why would you build your world on such shifting sands, when all those forks could pitch in on Debian long term support instead?

Or just shell out for RHEL?

Because it hasn't hurt Apple to have their proprietary garden and Red Hat has learned from that. Once you're big enough, people are going to do the work to support your nonsense (eg Asahi Linux) and build their world on your shifting sand. So why would you, as Red Hat, put in extra effort and not be capricious and hard to work with? It just costs you money that you apparently don't have to spend. Call it the Elon Musk is an Asshole (EMA) era of business.

The gratuitous anti-Apple remark. Walled gardens are at least as old as video game consoles and I can run whatever I want on macOS even if I can’t on my phone.

Somehow, whenever anyone spots a flaw in the FOSS model, it’s Apple’s fault. This is what people wanted, this is libre. Red Hat can sell for money. Remember, kids, CC violates a freedom!

This is what people wanted. Red Hat wants to be paid for their modifications to GPL code, and they've gotten their wish. They can't override the conditions of the license, but they knew that when they started making contributions. There's no world in which Red Hat has their hands tied by a license they opt-in to using. Red Hat can sell their own OS and block redistribution if they want, just not with GPL code.

You can only characterize it as "a flaw in the FOSS model" if you consider these patches to be Red Hat's property. They can sell it for as much as they want, just never "own" it.

> Walled gardens are at least as old as video game consoles

Walled gardens have existed much longer than that. But, so to speak, shooting someone at the country club doesn't give you the right to deny state police an investigation.

> Why would you build your world on such shifting sands, when all those forks could pitch in on Debian long term support instead?

Because RHEL is considered a standard in the enterprise circles, and software may not support distros that aren’t RHEL.

> Or just shell out for RHEL?

Why would you do that for smaller projects? Why would you do that for ephemeral containers? Why would you do that if you don’t care about RHEL support but want RHEL’s stability or are tied to RHEL for some reason?

Debian LTS only has 5 years of support, without guarantees, on a volunteer basis.

RHEL has 10 years of support, with the option of 13 years.

I like debian, but sometimes you have to have long term support. And I fully understand why you wouldn't support 13 year old software without getting paid for it.

> Debian LTS only has 5 years of support, without guarantees, on a volunteer basis.

Seems like there's plenty of volunteering in the CentOS fork community. Why not put that wood behind Debian LTS arrow? Seems like a much better idea than continue trying to build on an unreliable partner like Red Hat.

But the work is mostly to repackage RHEL patches. To support Debian LTS for longer you have to do all the backports which is obviously much more work.

Seems like a great time for Debian to step up, and offer 'sponsored' 10 year LTS support for their releases..

> RHEL has 10 years of support, with the option of 13 years.

The problem here is that the people who need 13 years of support are perfectly OK with paying Red Hat for that. No one is saying that doesn't have great value if you need it... problem is you don't need it for an awful lot use cases.

Red Hat didn't get to a billion dollar business on the back of not "needing it for an awful lot of use cases". We may not see it since we're developers reading HN, where it's all about the latest tech stack, but those 13 years if support are a huge money maker for Red Hat. They'd do 15 and 20 if their engineers wouldn't revolt. Outside of sexy tech is companies that want really really really stable, don't ever want to upgrade, and can afford to pay not to, until the very end. Hence that whole Y2K rush to update software.

There's a ton of Python 2 code still out there that won't get updated to 3 if it can be helped.

It doesn't look big, but it's like an iceberg. 9/10ths of it is invisible, and just sits there, raking in money.

> Debian LTS only has 5 years of support, without guarantees, on a volunteer basis.

Less than that if you care about security patches. Debian's Security team stop after ~18-24 months, at which point it transitions off to, quote, "Debian LTS is not handled by the Debian Security team, but by a separate group of volunteers and companies interested in making it a success.".

In practice I don't think this is a huge problem. It seems the Debian LTS team only gives up on maintaining packages once it becomes nearly completely impractical to do so.

For example, here are the unsupported packages in Debian 10. These are the sorts of packages that don't get included with RHEL or SLES in the first place. https://salsa.debian.org/debian/debian-security-support/-/bl...

and RHEL has paid, by-the-vendors-themselves support options.

unless you're going to have an entire org full of Linux SMEs -- and most businesses aren't going to do that, they're not software companies -- then you need a support option.

Debian ain't got that, and Canonical is something of a basketcase.

The idea was that Red Hat was doing things right, no need to punish them. But if they became a problem, there were other ecosystems.

I'll admit it's a big source of schadenfreude for me and I hope compliance reaches similar decisions at $DAYJOB.

> Red Hat got blue washed

I'm not sure I understand the term. Wikipedia says, "Bluewashing is term used to describe deceptive marketing that overstates a company's commitment to responsible social practices. It can be used interchangeably with the term greenwashing but has a greater focus on economic and community factors."

If I follow your usage, you're saying that Red Hat states they are committed to the ethics and social responsibilities of FOSS and the GPL but the actual practices violate those factors. If they are using GPL code but not releasing their code, then I would tend to agree, but IANAL.

Can you clarify a bit?

IBM = big blue. Red Hat were bought by IBM a few years back.

Ah, I hadn't thought of that angle. IBM has become the CA Technologies (formerly Computer Associates International) or MicroFocus of the 21st century.

I read messages like this and it always seems like folk think Red Hat are missing out because people aren't using a free competitor.

What I think this change will make happen, 80-95% of the companies using CentOS or Rocky stop using RHEL based linux distros. But the rest start paying Red Hat. Those who moved away were never going to buy anyways so why should Red Hat care if they don't use a free competitor that used their work?

There's two ways to increase business income, extract more $$$ from a captured user base or create value to attract revenue. One is a sign of a healthy company looking at future growth, the other the sign of a zombie company that has to put a tighter squeeze on you quarter after quarter to keep their numbers up. The people making purchasing/vendor decisions take into consideration the risk of going with a 'zombie' company, so those that do pay are also likely to consider moving away as well.

"Zombie company" is a weird way to describe a profitable company. How many people are going to say "You know what, this solves my problem really well but I am scared they're not going to solve future problems?" Especially in the sector that Red Hat are in, a world where there is a famous quote of "No one gets fired for buying IBM".

IBM/Red Hat are not in the cycle where they create new products and create value that way, they're in the business of buying things and increasing their market share that way. And have been for quite some time.

This has definitely spooked us as well. We have numerous images based on our rocky base image, so we're looking at what moving away from all things RHEL, Rocky, etc. as well.

And now with the release of Debian 12 ("bookworm"), Debian is now sufficiently up-to-date for most users.

Has RHEL ever been about being up to date? I thought their selling point was maintaining ancient ass software.

They backport a lot so you can run your ancient-ass software knowing you have maintained security patches.

You are mixing that up with Debian

> sufficiently up-to-date for most users

Always has been, compared to RHEL

The "problem" with Debian is the lifecycle. You get about a 12-24 months at most of security patches, and that's effectively it (the patching model after that is unreliable). That means you've got to allocate resources for upgrading/validating far more regularly than you would with other distributions. Depending on the size of your business, that could get really expensive and disruptive, for negligible benefit.

Canonical Ubuntu has a longer support on their LTS releases, and may be preferable. If you can get past annoyance with things like snaps.

It should be noted, as many businesses are finding out, that you can't redistribute Ubuntu binaries or sources as-is, since they contain registered trademarks of Canonical.

So, if you want to ship Ubuntu-based systems, you actually have to maintain your own version of all of their software stripped of the trademarks and re-compiled, or pay them. It seems Canonical is getting more interested in actually enforcing this, I believe they mostly ignored it for a long time now.

Debian seems like a much simpler alternative than doing all this.

Source: https://ubuntu.com/legal/intellectual-property-policy

> Any redistribution of modified versions of Ubuntu must be approved, certified or provided by Canonical if you are going to associate it with the Trademarks. Otherwise you must remove and replace the Trademarks and will need to recompile the source code to create your own binaries [emphasis mine]. This does not affect your rights under any open source licence applicable to any of the components of Ubuntu. If you need us to approve, certify or provide modified versions for redistribution you will require a licence agreement from Canonical, for which you may be required to pay. For further information, please contact us (as set out below).

Do you have any other references on this?

It seems that it all hinges on what a "modified version" of Ubuntu is. Is redistributing their packages outside of a full disk image a modified version?

Don't have more sources , but my understanding is that Canonical considers that anything other than downloading an Ubuntu disk image from Canonical and hosting that on your own site constitutes a modification.

So, for example, if you take an Ubuntu image, change the default username and password, and re-export it as a new ISO, you have modified the Ubuntu image and can't redistribute it with the *buntu trademarks unless you make an agreement with Canonical. IANAL so don't take my word for it, but this is my honest understanding of what Canonical claims at least.

This does seem to be in agreement with the next item in the FAQ I linked - where they say that using an image that doesn't conform to the IPRights policy from someone else is not recommended since they can't guarantee that it will work with future updates or such - which any modification even of default settings could provoke.

Thanks! I agree with your reading that changing the default username and password and re-exporting would count as modification.

However, it seems there are plenty of Ubuntu .deb mirrors out there... and there are even instructions at https://help.ubuntu.com/community/Debmirror.

Your previous post said "you can't redistribute Ubuntu binaries or sources as-is, since they contain registered trademarks of Canonical" (emphasis mine), which I think isn't quite true - there has to be some modification involved to fall foul of Ubuntu's IPR.

Oops, you're right, I misspoke - I was thinking about the modification step as a given, but that's not what I said. Apologies for the confusion.

> Do you have any other references on this?

Not a reference per-se, but an interesting post on the Nirokey blog "NextBox: Why we Decided for and Against Ubuntu Core"

[1]https://www.nitrokey.com/news/2021/nextbox-why-we-decided-an...

learning from their new Masters in Redmond perhaps?

That is basically the same as what Red Hat used to do. Centos had to be very careful to avoid improper trademark usage in their rebuild.

I tend to agree, if Debian embraced a longer term support model it would attract a lot of enterprise deployments.

RHEL is something like 10 years while Debian is more like 2, 3 if you push it.

Stock debian is 5 years with their LTS project, but they have a paid "ELTS" project that adds an additional 5 years. So 5 years for free, 10 total years as a paid support option. https://wiki.debian.org/DebianReleases

No, all Debian releases are completely supported 3 years (1 full year after the release of the new stable release) and are LTS for 5 years.

Yes 2 years is the realistic lifespan in service. 3 is pushing it as part of the last year is spent on planning/upgrading. LTS is sub optimal.

That's probably because people pay to use RHEL, which funds supporting it for 10 years.

And that is why RHEL is so valuable in enterprise.

When you see people still running PHP 5, or Python 2, and not for tiny little nonprofits either... there are large sums of money being thrown over the wall to support that.

> You get about a 12-24 months at most of security patches, and that's effectively it

What if all that effort on CentOS forks was spent on this?

CentOS (and it's forks) never backported security fixes for old software versions; it was always RedHat that took on this grunt work.

Even within that 12-24 month timeline, security fixes are commonly only backported where there is a significant enough security risk in a significant enough package. More resource on this mundane but important task is sorely needed

That's simply not true. All Debian releases have been LTS since 2014, for the following architectures : i386, amd64, armhf and arm64.

All Debian releases get 1 year of full updates after the new release, and at least 5 years of updates in total from their initial publication.

After the first couple of years, Debian Security stops patching it. "Debian LTS is not handled by the Debian Security team, but by a separate group of volunteers and companies interested in making it a success.",from https://wiki.debian.org/LTS. It is not the same thing, and the time scales and patching consistency is not on the same scale.

I havent been following the saga, just learned about it last week.

Was Red Hat losing money? Or they wanted more money?

"It's not the pile, it's piling it higher."

They sold out to IBM which was already dying, now they are taking Red Hat down with them... sad times

I worked at a different office that got blue washed, it was depressing af, feel bad for the long-time redhatters

IBM thought they could extract more than the already nice margins Red Hat was generating.

IBM sees economic utility as just making money. They don't see the utility of name branding and ease of transition.

For example, I was using CentOS for prototype development. No way would I be able to get RHEL for this. CentOS allowed for not having to hassle with development license management with RHEL. Meaning more time spent on trying to get something work vs handling business to business logistics. This way, if the prototype was deemed acceptable, I could hand off OS management to IT and they could handle B2B logistics.

Now I no longer prototype with an easily transferable RHEL style OS. Means IBM just reduced their probability of receiving future support contracts. This is where IBM lacks understanding of economic utility.

> I was using CentOS for prototype development. No way would I be able to get RHEL for this.

Why doesn't the free RHEL developer subscription work for this use case? (Honestly curious, as I use it for similar prototype development and want to make sure I'm not missing something).

If you spin up a bunch of cattle, your automation and testing is going to want to be for your specific use case to get the meat from them - the moment you need to waste even 10 minutes dealing with (even uncertainties around) subscription management to name and enumerate each of your "pets", then you know the vendor doesn't care about your real-life problems and will persist in generating roadblocks to fulfill their business requirements while ignoring yours. Where an option exists to do the CI/CD/testing/prototyping with an airgap or where no communication with the mothership is required, and no requests for licenses/subscriptions are needed for you to do your job, that option begins to look very appealing. Sure, you may get a bunch of free dev licenses - but I believe the integration cost of using them costs more engineering effort than option #2: not bothering with any engineering integration effort to fulfil another business's requirements while you should instead be working on your own.

I see where you are coming from, but I've found it beneficial to my business to trade the minor inconvenience of free subscription management for the stability and long-term support for RHEL.

I get that others may not like that trade-off, but I was mostly curious if there were any specific reasons that would tip my personal scales. Sounds like there aren't.

Time is an economic utility. Managing developer account(s) and licensing is a time sink to bad utility. Also dealing with the B2B sales department is also within that time sink. More friction to use a product for a prototype is also a bad utility.

Just look at the developer Subscription FAQ and there are a bunch of possible issues that one must spend time to work around. Loading CentOS is near frictionless.

The better example is that small companies could build on CentOS and when those small companies become big or get bought by someone big and get infested with useless middle management, RedHat could then sell support contracts into those businesses. CentOS was basically a big loss-leader to get lock-in and just wait until middle management went looking to throw money at vendor support to have someone else to blame.

IBM is now going to be cutting that channel off. They might juice a few more contracts in the short term, but mostly they're going to push small businesses away from the RHEL ecosystem entirely and cut off their pipeline of new leads.

I'm not sure that the old model did either, tbh. The clones were never a great choice if you needed to satisfy business concerns.

Can you share the top contenders for replacements?

debian most likely, still have to do a proper evaluation, it's a big lift to go from yum to apt though...

Yeah, the saddest bit about RHEL dying is that they really were the best distro by a wide margin. Redhat beat every piece of software into the RHEL way and what came out was an incredibly cohesive system. SELinux for all the years everyone cursed its name ended up working great in the end. All while being the most adventurous and ambitious distro in terms of adopting new userspace primitives.

RedHat was the first linux distro I ever installed on one of my own machines, so it has some sentimental value for me, but it's been a long time since I used it either personally or at work.

I started doing this yesterday, and with chatgpt 4’s help it hasn’t been as bad as I feared - I’ve been able to dump Ansible playbooks into it and ask for Debian package names/alternatives - obviously it depends on what you have, and my needs are definitely not nearly as demanding as they will be for a lot of companies, but I have around 100 servers that I will be switching over, and what I feared would be a mammoth task is now looking like something I can knock out in a couple weeks

Not really because people who are using RHEL in the first place are $BigCorp which run enterprise software (for example Oracle) for which you can only get support if you run on a supported enterprise OS.

So we'd just run RHEL for the clients who needed these support contracts and used CentOS for everyone else to still have a homogeneous environment.

With Debian I can't get this kind of support. So the options are reduced to Oracle Linux and SLES/openSUSE.

SUSE or Ubuntu

Reply on news.ycombinator.com