Six Great Features with the Upcoming Linux 6.6 Kernel
phoronix.com
phoronix.com
Wouldn't it be cleaner, easier and simpler to keep those separate and add them later as "modules"?
Isn't this how we're supposed to write other types of software? Why is this case different?
That is exactly what Linux does. Part of building a Linux distribution is determining which modules to ship by default to your users. To the best of my knowledge, the overwhelming majority of drivers are not installed by default, and the average user will never need them.
The situation of GPL'd drivers in a kernel with an unstable ABI wasn't really planned, but for users it's probably the best outcome.
If you combine that with a stable set of C function names and parameters, then you could define an ABI for kernel device drivers if you (and the Linux developers) wanted. While it sounds like a good idea, the outcome probably wouldn't be great for users.
Stable ABIs work in other places. For nbdkit we defined a stable ABI for plugins and have maintained it for over 10 years, so a binary plugin written for 10 years old nbdkit can be loaded by modern nbdkit just fine (and we test this too). This was done to encourage proprietary modules, because that helps with our wider goal to make all block devices and formats visible through an open, interoperable protocol (ie. NBD). Implementing this wasn't very easy, since we also want to allow the plugin interface to evolve. It involves a bunch of C macros and careful expansion of C structs so earlier fields remain compatible while later fields add the new features. https://libguestfs.org/nbdkit-plugin.3.html#API-and-ABI-guar... https://gitlab.com/nbdkit/nbdkit/-/blob/9f96d53dbbbf67ae5ed0...
If you're asking why so many drivers are included in Linux's mainline repository and not maintained off-tree - the major advantage of being upstreamed is that changes to the kernel have to take your driver into account as well. e.g. anyone who changes any APIs will have to update your driver as well.
Moreover, being upstreamed certifies that your driver has gone through a non-trivial review process to be merged into the mainline and is an indication of quality/stability.
The modules included in the kernel are updated whenever the API changes.
For the external modules, every time when a new kernel version is released, they may become broken and there is a lot of work to identify how to update the modules, unless one watches the kernel mail lists every day, because there is also no documentation about how to migrate the old modules to the new kernel.
The reasons for module breakage may be just the movement of some definitions from one header to another, which break compiling, but most frequently some structures gain new members or lose old members, or some functions gain new arguments or lose old arguments.
When there are new members or arguments it is hard to discover what values should be put in them, while when members or arguments are deleted it is hard to discover whether their absence must be compensated somehow, e.g. by inserting invocations to other functions.
In the worst case everything can be solved by reading the kernel source, but that takes a lot of time and those who maintain out-of-kernel modules usually do not do this as a full-time job, so they do not have time to scan every day the kernel mail lists, to see if anyone has plans to make changes that will break their modules.
Those who cannot do that because their modules are not open source, like NVIDIA, have a lot of work to do for module maintenance.
Nevertheless, there are also open-source modules that cannot be pushed into the kernel because they are for uncommon hardware with few users, and those also require a lot of work for maintenance, so they are frequently abandoned and no longer brought up-to-date, to be compatible with the latest kernels.
> Those who cannot do that because their modules are not open source, like NVIDIA, have a lot of work to do for module maintenance.
Say someone gets a driver for a WiFi USB stick into the kernel source and leaves it for others to maintain. Wouldn't the maintainers need the hardware device for testing changes and an understanding of the hardware's quirks to keep the driver updated? Obviously the system works but I don't get how it doesn't suffer from software rot, especially when physical hardware is required for robust testing including testing the hardware against different motherboards.
They can't test what they don't have, and I haven't seen evidence than any open source kernel has a testinfg time with large fleet of hardware for tests. There's more or less an incidental distributed test fleet as users have various testing setups, etc; but driver mistakes in obscure hardware tends to get found late.
It's not unusual for drivers to be found to have been broken for years, and then unless the reporter can fix it, or the fix is obvious from the report, the driver gets removed.
Another path towards driver removal is some interfaces get updated in multiple steps: a new interface is added, drivers are updated to the new interface, the old interface and any drivers that haven't been updated are removed. If nobody has stepped up to update the driver before removal, that's an indication of disuse; sometimes they get added back after removal though.
For the record, Windows isn't necessarily better. I had a printer where there was a working driver, but Windows would prefer the non-working driver from Windows Update, and it was impossible to get that fixed.
"Okay" to "very poorly", depending on your view. And for the record, many times the drivers go untested, frankly. I've absolutely found completely broken drivers in the kernel (as in just obvious lock imbalances where unlock() can't possibly ever get called, null pointer derefs; simple things you can spot in a code review) in the kernel where the break was introduced as some part of a refactoring and the person doing the work made a mistake. Because they also refactored 30 other drivers in the same patch series to match an API change. (The filesystem developers have said many times in patch reviews that certain things are massive chores because they have to go fix 40+ filesystem drivers every time some API break happens.)
Linux actually has tons of hardware regressions because of this whole design choice. When you totally rewrite a piece of code that is shared among multiple components, that can be OK, as long as you preserve the external behavior that the previous interface exposed. An easy way to do that is to establish a contract between the implementation and call sites, which is often reflected as a stable API. The stable API makes some contracts explicit, by construction. But many times API refactorings will happen at the same time, and that in practice typically introduces new behaviors into the downstream code that previously didn't exist before. These new behaviors, when introduced into an existing driver that was not developed for them, tends to cause bugs when not tested fully.
A big reason Linux chooses not to have stable internal APIs is for agility, more or less. But nothing is free and this is the price that is paid as the project grows.
My personal poster child for this stuff is the amdgpu driver. I use a Navi workstation card (WX5500) in my server and whether or not the amdgpu driver functions correctly on updates is a crap shoot. A while back it went completely headless; when I upgraded to 6.5 like 2 weeks ago, and I had to attach a monitor to look at the kernel logs (network config snafu), my dmesg had 7 kernel faults in its log from amdgpu. Seven! For a 3.5 year old card! With no desktop environment! Despite the fact that the card isn't changing, the driver is changing; new hardware support, expanded interfaces, new features, and those cause regressions. Many subsystems get overhauls and behavioral changes every release, so these things are bound to happen. Testing is not uniform; many parts of the kernel are far more well tested than others. Peripherial drivers are another example of easily broken code (Xilinx code upstream is frankly broken garbage half the time.)
The Linux Kernel is a pretty amazing project in many respects but I'm frankly astonished half the time it actually boots to a working desktop successfully.
Our module is GPL FWIW but I doubt it would be accepted into the kernel tree as it is. We need to have a driver developer on staff anyway to support it so it works out okay for us. But if we ever stop maintaining it, it will bitrot quickly and stop working on new kernels.
Also, loading a non-GPL module "taints" your kernel (this cannot be undone except through reboot) and tells everyone on the support mailinglists that you've loaded proprietary code.
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux...
And there is much hating on Nvidia, for using a "workaround". Including, famously:
https://www.youtube.com/watch?v=tQIdxbWhHSM
Also famously, this is the reason ZFS is not in the linux kernel.
Ironically, this is a form of DRM.
You just can't distribute kernels with proprietary modules outside your organisation, and you have to upgrade and support the modules yourself. Which, I'd bet, effectively means Google, Meta, Samsung & Oracle and other behemoths can and do use proprietary modules.
The main complaint is that, say, any kind of small hardware vendor effectively cannot have proprietary modules in the kernel.
And this is generally regarded as a very good thing, a nice way to deal with a very real threat to open source.
Except that the Linux kernel falsely taints itself with the "proprietary" bit when a module with a non-proprietary, non-GPL compatible license is loaded.
But of course, not being GPL compatible does not necessarily imply that it's proprietary, e.g. the CDDL (as used by ZFS) is a non-GPL compatible, free software license (according to the FSF).
And furthermore, this is documented and intentional.
But has there ever been any attempt to create an abstraction layer in the kernel that does provide a stable ABI? Something that could be used for certain classes of driver?
How would you possibly implement this as a technical restriction?
When only the ABI changes, a recompilation of all sources is enough.
When the API changes, then programmers must manually edit the sources and change function invocations and data structures, to match the new API.