Hacker News Reader

Comment by cayleyh | Hacker News Reader

"because of the monolithic nature of ZFS that created problems with the Redox microkernel design"

Anyone have an idea what this actually means and what problems they were having in practice?

I can only speculate, but maybe they're referring to the same thing Andrew Morton meant when he described ZFS as a rampant layering violation.

ie ZFS isn't just a file system. It's a volume manager, raid and file system rolled into one holistic system vs for example LVM + MD + ext4.

And (again I'm only speculating) in their micro kernel design want to have individual components running separately to layer together a complete solution.

It's only a rampant layering violation if you mandate the use of external layers like Linux device mapper as the only allowed way... Or you haven't actually read through the code and assume based on external user interface.

No, ZFS is not "monolithic".

It's just that on the outside you have a well integrated user interface that does not expose you to SPA (block layer), ZIO (IO layer, that one is a bit intersectional but still a component others call), DMU (object storage), and finally ZVOL (block storage emulated over DMU) and ZPL (POSIX-compatible filesystem on top of DMU) or Lustre-ZFS (Lustre metadata and object stores implented on top of DMU). There are also a few utility components that are effectively libraries (AVL trees, key-value data serialization library, etc)

In the Linux world you need to be hard to use in order to prove how pure you are. Anything that is actually easy to use is always considered unpure and bad.

Not sure why you're getting downvoted, considering how people torture themselves with calculating SSD cache sector offsets by hand so they can imitate 1% of ZFS's feature set with LVM2.

> when he described ZFS as a rampant layering violation

I read some blog posts back in the day about why they did this and it sounded a lot like those layers were more historical accidents or something.

You can turn it around and say that ZFS is a full stack filesystem (or vertically integrated if you will) and it should be pretty obvious that a rethink on that level can have big advantages.

Good question. I don't know about other microkernels, but NetBSD is a small kernel that supports ZFS. The support has been there since the 4.0.5 and 5.3[0], possibly earlier too. I'm not adept at navigating the mailing lists here, but I imagine a good place to learn about the challenges of porting ZFS to a smaller kernel would be the NetBSD and ZFS lists from that era (2008-2009). What NetBSD does today is use a 'zfs' modlue that depends on a 'solaris' kernel modile. The dependency of Solaris primitives is probably one of the major challenges with porting ZFS to any kernel. FWIW, somehow a ZFS port for the "hybrid" kernel in Windows also exists[1].

[0] https://vermaden.wordpress.com/2022/03/25/zfs-compatibility/

[1] https://github.com/openzfsonwindows/openzfs

NetBSD isn’t a microkernel.

Who is calling it a microkernel? The post youre replying to calls it a “small kernel” - that does not imply it’s a microkernel tho, right? I didn’t think size has anything to do with it.

I'm not sure if it originally said small kernel, though I know for sure the italics weren't originally there. The wording is unclear in a couple ways.

I came back to maybe delete my comments as I felt I might have came off harsh, esp before I saw the dead comment chain. No ill will, was confused as well I think.

I don’t think it’s microkernels in general but their microkernel design which wants as much as possible in userspace. They want each component to have its own memory space. ZFS blurs the layers between filesystem and the volume management. This kinda bothers layers of abstraction model folks. And I assume combined with their posix like model it just sorta clashes with what they want to do. Not impossible to integrate, but they want something a little different.

That seems odd to me too. It seems like they could have put all of ZFS (and SPL) in a single system service.

I particularly don't buy it because ZFS used to have a FUSE build, and I'm pretty sure there's at least one company still running it in userspace in some form (something for k8s, IIRC?)

Maybe they meant that they couldn't separate out the various layers of ZFS into their own services?

I guess? I'm not personally convinced that even 1 ZFS service per zpool would be a serious problem, but I can see why it would be considered unideal to a microkernel designer.

The main selling point of ZFS is it being monolithic. Because of that it can optimize many things that are impossible in a layered approach.

If I had to guess, it is because ZFS likes to insert itself into things beyond just being a filesystem. It is one of the reasons ZFS notoriously works poorly with database engines, which have a tendency to bypass or ignore the filesystem (for good reason). It is a design choice on the part of ZFS.

Oracle specifically came back to Sun because they had ZFS based servers. So that seems a bit strange to me.

Reply on news.ycombinator.com