Hacker News Reader

Comment by sysk | Hacker News Reader

I have a very limited understanding of how distributions work so this is probably a stupid question but why don't RHEL/CentOS update the Linux kernel they ship with their 6.x OS? Isn't the kernel supposed to never break user space apps?

View on HN

RHEL focuses on stability over bleeding edge features. What this typically means is that you will have an older kernel / user land packages installed, then Redhat backports security patches into it's maintained ports. You can read about it here [1]. The end result is a fairly stable kernel and system OS. But, you have to live with an older feature set (think 2-3+ year old package with just security patches applied).

I've maintained lots of RHEL boxes as a sysadmin -- here's my take. This type of release cycle works pretty well, in that you can almost always apply patches without hosing your system due to "new features". One example of something this protects you against, is a package changing the expected config file format, or maybe the expected command line arguments. Another good example, would be a change to a kernel driver for some storage array, or network interface, that could quickly break lots of systems!

This type of release cycle also gives commercial software vendors a known target to develop against and sell support for. Say for example, that you are using an enterprise database like Oracle/Sybase/DB2, they will verify their product works against RHEL 5.x/6.x/7.x and give you commercial support, as they have a pretty good understanding of what you are running. But, if the kernel/packages are always moving targets, it would be near impossible to support your end platform, so this also adds to the backporting of security patches without adding features thought process.

Although, it is also a pain, in that you are typically way behind on cool things happening out there on other distros. So, there are pros and cons to it. But, if you are running a large mission critical DB cluster, you typically want things to be extremely stable, so for the target market (the enterprise) this works pretty well.

[1] https://access.redhat.com/security/updates/backporting

There are definitely upsides and downsides. The vast majority of our systems are RHEL6, because that's what a vendor of ours supports. Since we have the infrastructure around to support RHEL6 (puppet modules, vm templates etc), most of our supporting systems are also RHEL6.

If we need newer features or packages, we spin up a CentOS7 or RHEL7 image which still isn't bleeding edge, but is significantly newer than RHEL6.

It's nice to have a bit of stability for apps and services that require it - but there's always the option of bringing newer versions online for the services that need it.

As much as RHEL is super conservative, Fedora is the polar opposite. They'll wantonly break things, throw things out, completely overhaul how things integrate.

For example, in previous releases they've removed ifconfig, and lately, even yum got chopped.

It's like living a few years in the future. systemctl was there years ago.

The issue of bringing anything like Docker (app virtualization) to RHEL/CentOS/SLC/CERN/Rocks 6.x is supporting an old kernel, adding custom-compiled kernel modules or compiling a custom one. I had this exact conversation with some clients whom I did some hypervisors work with and whom recently sold their startup to one of (VMware, Citrix, Microsoft). In fact, about 1-2 years ago we joked they should've bought dotCloud (now Docker).

PS: Docker is a prime example of how a side feature can be instrumental to a pivot to something much larger.

The kernel isn't just about user space apps though - it's about interaction with hardware. Updating the kernel can have large (or subtle) impacts on the performance and characteristics of hardware, which in a production environment can be dangerous, hard to track down, and impact your application anyway.

Linux does introduce bugs, which change the interface, but also adding new features can change the way programs behave, eg if the program detects features at run time and uses them.

They don't update whole kernel, but they backport features and bug fixes from newer kernels to the ones they ship.

Reply on news.ycombinator.com