Hacker News Reader

Comment by winter_blue | Hacker News Reader

Wow, I'm surprised by how small the file systems are.

On the smaller side, Ext4 at just 21,515 lines, and on the larger side, Btrfs at 55,758 lines.

It almost makes me think that writing a new file system in Rust, might actually be a tractable project.

I'm writing a hobby kernel/operating system and one of the things that surprised me was how simple ext2 is. It really isn't much different from fat32 in many ways, just a bit more flexible and designed in a more future-proof way.

Obviously ext2 is a very outdated filesystem by today's standards, but considering ext3 and ext4 are essentially just some extensions slapped onto ext2, I still didn't expect it.

Another thing I found really impressive was just how resilient e2fsck is. During early development of my ext2 code I frequently mangled the file system in a myriad of weird ways, but no matter how much I broke things, e2fsck was almost always able to restore the file system into a reasonable state.

(For the curious, there is some excellent documentation on the ext2 file system at https://www.nongnu.org/ext2-doc/ext2.html that I mostly used for my implementation of it. Sadly no such thing seems to really exist for ext3/ext4).

The difficulty in filesystems is generally getting to the point of high enough reliability that people trust then with their data. This is why development and adoption of new ones tends to be slow, not the quantity or complexity of implementation. So for a hobby it's quite accessible because that expectation of reliability isn't there.

I also thought that'd be a fun project. BTRFS seems to have the largest feature set although many features are experimental but maybe there would be a place for something smaller that scales well and maybe has some of the new features like deduplication

Filesystems tend not to be all that complex to implement.

Simple filesystems - those oriented at speed, without integrity checking, compression, encryption and raid-like features. Which are extremely useful and desirable in a filesystem.

When people tried to do a full-featured filesystem, with a remarkable exception (ZFS) it's either a disappointing unfinished mess (btrfs) or a promising, but a long-term project taking years (bcachefs).

Making a filesystem with strong support of important features is an extremely demanding project.

Don't forget: buggy filesystem code can easily lead to catastrophic data loss, which ruins your filesystem's reputation.

Lines of code isn't the only important metric. A project with a smaller number of LoC can take a long time to fully test and debug and validate.

Hammer2 comes to mind as well..

Does it run on Linux or ordinary BSD? Does any company use it?

I'm sorry, as I keep hearing these criticisms around BTRFS, I still don't know why. What is so messy about it? Honest question

Have you tried reading the code? It's twice as large (107KLOC) as ext4 (45KLOC), where ext4 is viewed as a natural evolution of ext2 (7KLOC).

45KLOC of C code already sounds really scary to me, although yeah, journaling is really nice to have.

Yes, that's scary. Understandable, but it does come with a lot of functionality that's nice to have no?

It's "perpetually half-finished". Convoluted admin cli tooling, bad track record with reliability and loss of data, lots of stories on the internet. It perhaps got better, but RAID5/RAID6 for metadata (standard use of disks) still not oficially recommended and thus standard RAID5/RAID6 is not reliable. Also, look at some FS benchmarks - BTRFS is still much slower than EXT/XFS, and slower than ZFS.

About the only class of scenarios to use BTRFS instead of ZFS is if you want the features, do not need RAID5/6, require support by Linux distributor/kernel developers or for some other (hard to think of) reason you can't use ZFS.

I do use BTRFS on a single disk where I don't touch it with cli tools, and there it's fine.

Thanks, that makes sense. I should look more into it

No problem. For more, this looks like a good intro

https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu...

Filesystems aren't complex to implement -- even complex filesystems.

What's harder is making them as robust as they need to be for practical daily use. Doing that is a matter of comprehensive testing, though. With filesystems, the hard part is QA, not implementation.

Oh it's word games with "implement". OK then, let's agree then developing a modern filesystem into a robust software for practical daily use is hard.

Reply on news.ycombinator.com