Hacker News Reader

Comment by zogomoox | Hacker News Reader

Wouldn't a file system in a slightly higher level language solve some of this pain? (edit: in terms of being able to understand the system)

View on HN

You're at odds with the desire (necessity, actually) to have precise low-level control, not just of the in-memory layout of the data structures, but also of the performance characteristics of your code (e.g. no unnecessary pointer-chasing). Higher-level languages tend to make it easier to pile up abstractions; we want the orthogonal property of making it harder to shoot yourself in the foot.

You can look at ZFS for inspiration. It is more complex than say ext4, but it's less complex than the full stack of mdadm+LVM+ext4 - the latter piles up the abstractions, where ZFS is able to "reach around" and e.g. directly track free physical blocks across multiple devices. So when you need to resilver the pool, you don't need to copy the unused blocks, and reduce the load on the pool (and thus the chance of double failure).

Are there programming languages that have such properties? At the risk of perpetuating the meme, I'd say Rust (with no_std) fits that description. I can't tell how would it help in this specific instance (I'm very far from an expert in FS implementation), but it does tend to prevent data races in general.

I like Rust, but you are going to be writing a lot of unsafe Rust for a filesystem implementation. Multiple processes are writing to the filesystem simultaneously. "Ownership" is fuzzy and is moving around.

At that point, is Rust buying you anything for how much it's going to get in your way?

I really don't see an advantage to Rust when operating at these kinds of low levels.

We have concrete data on this at this point, and it’s just not true that, even in these sorts of low level programs, everything ends up unsafe.

And beyond that, rust has many features that are useful separate from memory safety.

https://asahilinux.org/2022/11/tales-of-the-m1-gpu/ being just one example.

That said I have no opinion if they should write this driver in Rust or not, I simply do not know about the details. But in general, “it’s too low level and so tons of unsafe and so therefore Rust is useless” is at least arguable, if not just fully incorrect, as a general point.

Have you ever written a file system? Most of the work is conforming to the semantics of the interface to the kernel/programs calling it. Very little of that requires unsafe code, and the bulk of the provably unsafe stuff (physically writing to memory/disk) is very simple.

The complex stuff can and should be written at a higher level than C.

> Have you ever written a file system? Most of the work is conforming to the semantics of the interface to the kernel/programs calling it.

Yes, I have. And I have implemented partial file locking semantics. And I can painfully remember what I went through to validate it.

Quite a few of those pointers are write pointers which are simultaneously active with a lot of read pointers and they have different owners. That is a task which is screaming "Rust is going to make your life miserable."

There are entire kernels written in Rust with 10% or less unsafe code.

"Just rewrite it in Rust" is a bit meme-y, but what I wanted to highlight is how you can look at ZFS itself for inspiration. ZFS has checksums to detect bit rot; maybe you want a language with better support for contracts / static analysis. ZFS has zvols to expose virtual block devices for use with other filesystems; maybe you want a language with safe C interop; etc. Any improvement in safety over plain old C would be desirable, and even unsafe Rust is safer.

For the bug in the commit linked up? I don't see how: the issue is a logic error causing a data race which depends on a whole bunch of factors a higher level language can't model directly.

The next step up for that is formal verification which is staggeringly time consuming (read: expensive) and hard to understand in the first place.

Maybe something like Ada with SPARK, or at least partially transactional memory to make some types "atomic" in behaviour.

But that would make it way harder to add to systems it's already running on.

Reply on news.ycombinator.com