Comment by hnlmorg | Hacker News Reader

Comment by hnlmorg | Hacker News Reader

Parent Full thread

> From content, parent reply is correct. ZoL was far ahead FreeBSD's native ZFS port, that FreeBSD devs found that is better using ZoL instead.

The FreeBSD fork also had a bunch of ZFS features that hadn't yet made it to ZoL as well as their own bug fixes too. I know this first hand from porting volumes between ZoL and FreeBSD.

FreeBSD didn't switch their code base and dump everything they had in their own fork. They switched upstream so that everyone worked on the same common code base and merged what was practical and kept any of their own OS dependent code separate. Just as I said in my previous post and just as the presentation you linked to also confirms. And for what it's worth, it's the OS dependant stuff that makes ZFS's experience better on FreeBSD than it is on Linux.

I also don't understand why so many people want to assume a negative spin on the story. Projects fork and change upstreams all the time when it's identified that there's a commonality they can share resources on. This isn't any different.

I don't see it as negative relying to an upstream with larger developer base, and as you said this allows better collaboration. But this dependence makes it hard for me to see how, though once it was, this is FreeBSD's killer feature. How is ZFS, now based on ZFS-on-Linux, more native on FreeBSD than Linux?

Today that's easy to answer: Try setting up ZFS boot with a mirrored boot drive and encryption on FreeBSD, then try it on Redhat, or SLES, or Ubuntu Server.

One of the experiences is very much "native" - the other is possible, but a GIANT pain to accomplish.

At some point I'm sure, Ubuntu at least, will make installing to ZFS on a server OS more straightforward. I've got several systems doing it today, but I can tell you it's not for the faint-hearted and it took a TON of trial and error to get it to do what I wanted consistently. I also cringe every time I apply an update because I know it's not something that is getting any significant testing by Canonical or the community at large.

My production Linux+ZFS server is still running a LVM mirror with ext4 on root because of exactly the fear you mentioned. All the FreeBSD systems have had mirrored ZFS root since v10.x (released 20 January 2014 according to Wikipedia). 7+ years later and I'm still not confident enough to run the same set up in production using Linux.

Do i smell Proxmox here? Fantastic system!!

Not here but I have used Proxmox before (v1 -- back when containerisation in Linux required a custom kernel) and it was pretty nice.

I'd already answered that -- both in my OP and in the comment you've just replied to. It's the OS specific code. Using ZFS on FreeBSD is as natural as using ext4 on Linux. Sure, ZoL exists and some distro's ship pre-compiled binaries for ZFS, but if your server drops into an emergency shell or even fails to boot entirely, if you need to boot from a live image (CD/USB) or your boot volume gets cocked up... fixing all of these requires extra hurdles on Linux if root is ZFS. Whereas that isn't the case on FreeBSD.

There's other OS specific features that FreeBSD does better with regards to ZFS such as ACLs too. But like with trashed volumes, they're edge cases.

FreeBSD also has better bundled tooling for working with ZFS. By this I don't mean `zfs` and `zpool` but little shell scripts et al that are FreeBSD specific.

Then there is the documentation. If you're running Linux then all the documentation regarding server management is about standard server management (ie with an ext4 or xfs root). But FreeBSD documentation will talk about how to do management with the assumption you're running ZFS. The FreeBSD docs (which are excellent, by the way) has pages and pages on ZFS outside of the stuff published by ZoL (which is pretty poor it has to be said) and OpenZFS. And I say the last points as someone who's had to manually compile kernel modules on Linux to get the latest patch to support a feature that made it to FreeBSD first.

I do have servers running ZFS + Linux and they work really well. So I'm not taking anything away from that architecture. But when things go wrong or you're trying to do something a little more specialised outside of basic file storage, ZFS has always felt like an awesome hack on top of Linux rather than an integral part of it's design. Unlike with FreeBSD where the same people managing the kernel and userland are the same people pushing ZFS as the default file system. If Torvalds, Pottering or others held ZoL in the same esteem then I'm sure the edge cases I've described would go away (Linux certainly has the larger developer community).

I see. It isn't the implementation, as I understood the argument to be about, but rather the environment around it. What you're concluding is basically the core difference between Linux distros and BSDs. Whereas the former are built as an assemble, the later define a base system making them more coherent. The better documentation can be said is result of this. Disappointingly the submitted article mentions nothing of those.

That's a fair summary.

That article is much better:

https://vermaden.wordpress.com/2020/09/07/quare-freebsd/

As always, it's about tools and integration.

ZFS as root is completely normal on FreeBSD, is it on Linux? You have tools like zfs-stats, bectl, zfs or dataset-jails, poudriere (packagebuilder) fully integrated with zfs, the list goes on and on. So even when the base of OpenZFS came from ZoL, it's use on freebsd feels much more integrated...or let's call it native.

On the nativeness, as I understands:

ZFS relies heavily on Solaris VFS API, which is inherently different from both Linux's VFS and FreeBSD's VFS implementation (e.g., behavior of vnode[1]). Both FreeBSD and Linux implements this VFS API with a layer called Solaris Porting Layer (SPL).

In the original Illumos-based FreeBSD ZFS, this SPL layer was implemented as a kernel module (opensolaris.ko). OpenZFS originally maintain this shim in other repo, but later incorporate this shim into its codebase. FreeBSD port of OpenZFS involved importing SPL layer from FreeBSD tree into OpenZFS. On this part, there's not much difference in nativeness of the port.

However, this SPL layer also serve the purpose of adding one degree of separation between GPL code and CDDL code in order to comply with GPL in order to prevent ZFS code from becoming a derivative of the Linux kernel. The shim in OpenZFS is in three licenses; the Linux part of SPL is GPLv2, the FreeBSD part is 2-BSD, and the part that interacts directly with ZFS code is CDDL.

This is the area where FreeBSD can be more native with regarding to ZFS. Since CDDL is compatible with BSD license, FreeBSD kernel and its userland can integrate directly with ZFS without any licensing issue, e.g., FreeBSD bootloader can simply linked against ZFS, whereas GRUB2 must reimplement a part of ZFS in order to do probing.

[1]: https://wiki.freebsd.org/AndriyGapon/AvgVfsSolarisVsFreeBSD

For example, on Solaris and FreeBSD, it is possible to force-unmount a filesystem. on Linux it is not.

Integrated with Kernel, vs not. Other tools in the OS that can take advantage of ZFS, vs not.

Reply on news.ycombinator.com