Hacker News Reader

Comment by atombender | Hacker News Reader

Out of interest, why can't file system implementations be portable? A shared core that exists as a library that can be tested in userland, and which can then be used by a lightweight shim that implements the kernel-facing interfaces. But I haven't seen any file systems implemented this way.

View on HN

The irony is that most of the newer ones in Linux are written that way, just nobody writes a FUSE binding to them. For example, both Btrfs and XFS contain fully functional pure-userspace implementations of the filesystem inside their userspace code (libbtrfs and libxfs both contain the complete filesystem code), but nobody has written a FUSE binding to either.

Btrfs' implementation is even used to do most of the functionality in the tools (like send/receive) rather than using the kernel to do it (in order to minimize context switches that reduce I/O performance).

FUSE is neat for one-off access to obscure file systems, and FUSE is usable for things like sshfs for those who use that, but it is my impression from having used FUSE myself that FUSE does come with a noticeable performance penalty. Is my perception wrong on this? Because if it is not then I’d be hesitant to use a system where the main storage was relying on FUSE.

Stuff like gocryptfs seem to push FUSE to quite high performance. There is still some latency but streaming read/write seem to go fast.

Do you have any documentation on that?

The benchmarks on their website[1] compare it running on a 4 x 3.20GHz with hardware AES-N instructions, against an in-kernel implementation of a different cryptosystem running on a 2.7GHz cpu that does NOT have AES instructions. I also can't see anything that speaks to speed of the storage (or if this is controlled for), so I'm admittedly quite skeptical.

The numbers they do publish aren't particularly impressive to me either, so we may also have wildly different ideas about what "quite high performance" even means.

[1]: https://nuetzlich.net/gocryptfs/comparison/#performance

FUSE is also great for slower media. Copying files over USB sticks without being restrained by FAT-32 limitations is great. There have been times that I've used NTFS-3g for this because its FUSE implementation meant I could read it on any OS.

This is also how the original BSD UFS/FFS was developed some three decades ago.

While the kernel interfaces aren't exactly wildly disparate, they are perhaps surprisingly diverse (and coupled to kernel implementation details) for something as boring as a filesystem. At least some of this is for performance reasons. Accessing offsets into kernel structures is usually going to be faster than copying the data to use on your own.

On top of that, since there is no single standard interface in the kernel, kernel maintainers are skeptical of adding shims to the mainline kernel. The Linux kernel developers are perhaps the most famous for being vocal about this, but they are not unique.

That being said, there are portable file system implementations that use FUSE. See e.g. NTFS-3g[1]

1: https://en.wikipedia.org/wiki/NTFS-3G

Shims in the kernel seem a bit anti-social, since all module developers are expected to contribute to the shared core code. Of course it can be done, and it has been done. But I think FUSE is better if all you care about is portability, and you don't mind the heavy performance cost. My module could be ported for FUSE of course, but it's a lot more work than it may seem, the kernel interfaces are too alien compared to a userland library.

ZFS is pretty portable because of spl ;)

Reply on news.ycombinator.com