A Linux kernel without struct buffer_head

LWN.net is one site that has such consistently good writing that I even subscribed to it even though I wasn't really in the linux development community because it is just that good. I don't have time to read it now which is the only reason I don't subscribe currently. I highly recommend it to anyone.

I no longer work in a very kernel-adjacent area but I keep my LWN subscription because of the quality of the writing, and its general sense of the wider FLOSS ecosystem. It really is a great resource. https://lwn.net/subscribe/

It will certainly take a couple of years before it's completely removed. This is just a first step.

Microsoft is hiring. In the past, Microsoft had gone as far as emulating undocumented/undefined API behavior for misbehaving/buggy applications, to keep them working under new OS releases.

Nobody in GNU/Linux tests old binaries; only whatever they recompiled for their current distro, and screw the rest.

The kernel is pretty good about maintaining syscall compatibility.

All that the proposed new CONFIG_BUFFER_HEAD option does (https://lwn.net/ml/linux-kernel/20230424054926.26927-18-hch@...) is to enable/disable the struct code, and for each filesystem that still requires it, add:

   config EXFAT_FS
    tristate "exFAT filesystem support"
  + select BUFFER_HEAD
    select NLS
    select LEGACY_DIRECT_IO
    help

which means that if CONFIG_BUFFER_HEAD is disabled then all those filesystems get disabled too (or if you pick a filesystem to enable that still needs struct buffer_head then the CONFIG_BUFFER_HEAD option is enabled).

So basically it's only to show the kernel devs what is left. You already got the performance benefit for XFS etc as it just isn't used by those filesystems.

The filesystem doesn't just read from one part of the disk and write to another part of the disk, it reads from the disk and stores the result in memory, or it takes data from memory and writes it to the disk.

The other part of the kernel that deals with data in memory is the cache.

Rather than have two separate representations of "data in memory" and have to translate between them all the time, it makes sense to have one representation of "data in memory" that all the affected systems can use to talk to each other.

Even though the kernel now has multiple representations of "data in memory" due to advances in other parts of the kernel, it's still desirable to have just one representation shared between the cache and the filesystem, which is why people are talking about removing the old ones, and porting old code to use the new ones.

I think it's a question of how you design the OS kernel. Consider for example an application that has opened a file, and now wants to read data from the file. So it calls the read() syscall (https://pubs.opengroup.org/onlinepubs/9699919799/functions/r... ) either directly or via fread(), iostream or whatever. How should the kernel handle that?

- Since a file is something filesystems do, it seems sensible to quite directly plumb the read() syscall into the filesystem, and the filesystem can then handle reading blocks from the actual disk, or caching previously read blocks. If an OS wants to support multiple filesystem types, it might even provide some common data structures and code for doing this caching of disk blocks. In Linux this being the "struct buffer_head".

- The other option being that managing memory is something kernels do, and it makes sense to centralize this management in a common code that can then make sensible decisions what to do with this memory. Such as giving it to applications that ask for memory, using it for caching file contents (page cache), and the crucial part, balancing these different usecases and making good choices what to evict. If you make this design choice, the read() syscall will not immediately be handed down to the filesystem, but rather it first asks the memory management subsystem whether the data happens to be in the page cache, and only if it isn't, it calls down to the filesystem to read that data from the physical disk, copy it to the page cache (potentially evicting something else to make room), and then copy it back to the application asking for it.

Now it seems that historically quite a few OS'es, Linux included, started off with the first approach. However later on it turned out that the second way of designing the OS is better, and thus Linux was slowly over time modified to mostly use the page cache. Though the struct buffer_head is still in use here and there.

The sibling comment is one reason, but I'd like to add another. ext2 was introduced into Linux in Jan 1993, which is less than 18 months after Linux was first released. ext3 was made by copying the code from ext2 and renaming the identifiers s/ext2/ext3/g, and ext4 the same way from ext3.

So these filesystems are old. Even if it was a good idea to structure caching and filesystem block access separately (which as the sibling comment says, it isn't), the Linux developers were just getting started back then and it was much more important to get something working than to think about good abstractions.

Think of the internal kernel interfaces like a library. The older filesystems were coded against the library as it existed at the time of their writing. Some of the APIs got superseded by newer interfaces which are more recommended for newer code. But a lot of code written against the old interface is still out there. For good measure, let's say the new interface isn't a trivial port, sometimes it requires rethinking larger parts of the caller's code.

I think it's because ext2 and ext4 both use block allocation in favor of paging thru bio and uses buffer tricks to write data in a way that reduces fragmentation