Hacker News Reader

Comment by zfsthroway1 | Hacker News Reader

The FreeBSD project doesn't have the manpower to maintain ZFS. So FreeBSD is now relying on zfsonlinux.org, the Linux port of ZFS. In general, FreeBSD is increasingly having to rely on ports of Linux things, or emulating Linux things, because without enough volunteers or paid professionals to write native BSD solutions that is the only way to stay in the game. (Linus, too, complains of lack of maintainers for Linux.)

EDIT: Adding reference: "FreeBSD moving to ZFS-on-Linux"

https://forums.freebsd.org/threads/freebsd-moving-to-zfs-on-...

View on HN

FreeBSD is a first-class citizen in the OpenZFS project. It's not a hack or compatibility layer on top of the linux codebase.

It makes a great deal of sense for the Linux and FreeBSD communities to collaborate on ZFS rather than both maintaining their own separate forks and merging back and forth. It means you much more easily maintain feature & bugfix parity, and makes migrating filesystems between OSes easier.

It's a win for everyone involved, not a negative.

True, but Linux is also a first-class citizen in the OpenZFS project. The argument that was made earlier is that FreeBSD is so nice, because of its superior ZFS integration. This has been true for a long time and as such I agree with OP.

However, since the ZFS code basis is now the same for Linux and FreeBSD (which I also agree is a good thing) I don't think we will have the same superior FreeBSD ZFS argument anymore. And people who are in favor of FreeBSD over Linux have to provide other pro arguments than ZFS in order to make a case for FreeBSD.

Sure, the ZFS features are the same, but the integration is still not all there on Linux, and is on FreeBSD.

FreeBSD has features like 'bootonce', to switch ZFS boot environments on the next boot only. Better integration with the boot loader, more boot environment tooling, and other things that can only come when the filesystem is part of the OS.

I agree there are still some FreeBSD only features for ZFS, but the gap is getting smaller and smaller. So unless you are really depended on some of those features, there is no particular reason of why to prefer FreeBSD over Linux, unlike a few years ago.

You've literally just read the headline and not the content. What's happening is that FreeBSD are now collaborating with ZoL rather than the two teams working in parallel to create the same features -- thus needlessly duplicating effort.

To any sane person this news should be viewed as a victory not a loss.

    [W]orking through the git history of ZoL I have also discovered that
    many races and locking bugs have been fixed in ZoL and never made it
    back to Illumos and thus FreeBSD. This state of affairs has led to a
    general agreement among the stakeholders that I have spoken to that it
    makes sense to rebase FreeBSD's ZFS on ZoL.

    This port will provide FreeBSD users with multi modifier protection,
    project quotas, encrypted datasets, allocation classes, vectorized
    raidz, vectorized checksums, and various command line improvements.

From content, parent reply is correct. ZoL was far ahead FreeBSD's native ZFS port, that FreeBSD devs found that is better using ZoL instead.

Edit: Also related presentation by A. Jude at https://2019.eurobsdcon.org/slides/The%20Future%20of%20OpenZ...

> From content, parent reply is correct. ZoL was far ahead FreeBSD's native ZFS port, that FreeBSD devs found that is better using ZoL instead.

The FreeBSD fork also had a bunch of ZFS features that hadn't yet made it to ZoL as well as their own bug fixes too. I know this first hand from porting volumes between ZoL and FreeBSD.

FreeBSD didn't switch their code base and dump everything they had in their own fork. They switched upstream so that everyone worked on the same common code base and merged what was practical and kept any of their own OS dependent code separate. Just as I said in my previous post and just as the presentation you linked to also confirms. And for what it's worth, it's the OS dependant stuff that makes ZFS's experience better on FreeBSD than it is on Linux.

I also don't understand why so many people want to assume a negative spin on the story. Projects fork and change upstreams all the time when it's identified that there's a commonality they can share resources on. This isn't any different.

I don't see it as negative relying to an upstream with larger developer base, and as you said this allows better collaboration. But this dependence makes it hard for me to see how, though once it was, this is FreeBSD's killer feature. How is ZFS, now based on ZFS-on-Linux, more native on FreeBSD than Linux?

Today that's easy to answer: Try setting up ZFS boot with a mirrored boot drive and encryption on FreeBSD, then try it on Redhat, or SLES, or Ubuntu Server.

One of the experiences is very much "native" - the other is possible, but a GIANT pain to accomplish.

At some point I'm sure, Ubuntu at least, will make installing to ZFS on a server OS more straightforward. I've got several systems doing it today, but I can tell you it's not for the faint-hearted and it took a TON of trial and error to get it to do what I wanted consistently. I also cringe every time I apply an update because I know it's not something that is getting any significant testing by Canonical or the community at large.

My production Linux+ZFS server is still running a LVM mirror with ext4 on root because of exactly the fear you mentioned. All the FreeBSD systems have had mirrored ZFS root since v10.x (released 20 January 2014 according to Wikipedia). 7+ years later and I'm still not confident enough to run the same set up in production using Linux.

Do i smell Proxmox here? Fantastic system!!

Not here but I have used Proxmox before (v1 -- back when containerisation in Linux required a custom kernel) and it was pretty nice.

I'd already answered that -- both in my OP and in the comment you've just replied to. It's the OS specific code. Using ZFS on FreeBSD is as natural as using ext4 on Linux. Sure, ZoL exists and some distro's ship pre-compiled binaries for ZFS, but if your server drops into an emergency shell or even fails to boot entirely, if you need to boot from a live image (CD/USB) or your boot volume gets cocked up... fixing all of these requires extra hurdles on Linux if root is ZFS. Whereas that isn't the case on FreeBSD.

There's other OS specific features that FreeBSD does better with regards to ZFS such as ACLs too. But like with trashed volumes, they're edge cases.

FreeBSD also has better bundled tooling for working with ZFS. By this I don't mean `zfs` and `zpool` but little shell scripts et al that are FreeBSD specific.

Then there is the documentation. If you're running Linux then all the documentation regarding server management is about standard server management (ie with an ext4 or xfs root). But FreeBSD documentation will talk about how to do management with the assumption you're running ZFS. The FreeBSD docs (which are excellent, by the way) has pages and pages on ZFS outside of the stuff published by ZoL (which is pretty poor it has to be said) and OpenZFS. And I say the last points as someone who's had to manually compile kernel modules on Linux to get the latest patch to support a feature that made it to FreeBSD first.

I do have servers running ZFS + Linux and they work really well. So I'm not taking anything away from that architecture. But when things go wrong or you're trying to do something a little more specialised outside of basic file storage, ZFS has always felt like an awesome hack on top of Linux rather than an integral part of it's design. Unlike with FreeBSD where the same people managing the kernel and userland are the same people pushing ZFS as the default file system. If Torvalds, Pottering or others held ZoL in the same esteem then I'm sure the edge cases I've described would go away (Linux certainly has the larger developer community).

I see. It isn't the implementation, as I understood the argument to be about, but rather the environment around it. What you're concluding is basically the core difference between Linux distros and BSDs. Whereas the former are built as an assemble, the later define a base system making them more coherent. The better documentation can be said is result of this. Disappointingly the submitted article mentions nothing of those.

That's a fair summary.

That article is much better:

https://vermaden.wordpress.com/2020/09/07/quare-freebsd/

As always, it's about tools and integration.

ZFS as root is completely normal on FreeBSD, is it on Linux? You have tools like zfs-stats, bectl, zfs or dataset-jails, poudriere (packagebuilder) fully integrated with zfs, the list goes on and on. So even when the base of OpenZFS came from ZoL, it's use on freebsd feels much more integrated...or let's call it native.

On the nativeness, as I understands:

ZFS relies heavily on Solaris VFS API, which is inherently different from both Linux's VFS and FreeBSD's VFS implementation (e.g., behavior of vnode[1]). Both FreeBSD and Linux implements this VFS API with a layer called Solaris Porting Layer (SPL).

In the original Illumos-based FreeBSD ZFS, this SPL layer was implemented as a kernel module (opensolaris.ko). OpenZFS originally maintain this shim in other repo, but later incorporate this shim into its codebase. FreeBSD port of OpenZFS involved importing SPL layer from FreeBSD tree into OpenZFS. On this part, there's not much difference in nativeness of the port.

However, this SPL layer also serve the purpose of adding one degree of separation between GPL code and CDDL code in order to comply with GPL in order to prevent ZFS code from becoming a derivative of the Linux kernel. The shim in OpenZFS is in three licenses; the Linux part of SPL is GPLv2, the FreeBSD part is 2-BSD, and the part that interacts directly with ZFS code is CDDL.

This is the area where FreeBSD can be more native with regarding to ZFS. Since CDDL is compatible with BSD license, FreeBSD kernel and its userland can integrate directly with ZFS without any licensing issue, e.g., FreeBSD bootloader can simply linked against ZFS, whereas GRUB2 must reimplement a part of ZFS in order to do probing.

[1]: https://wiki.freebsd.org/AndriyGapon/AvgVfsSolarisVsFreeBSD

For example, on Solaris and FreeBSD, it is possible to force-unmount a filesystem. on Linux it is not.

Integrated with Kernel, vs not. Other tools in the OS that can take advantage of ZFS, vs not.

That's BS and you know it, it's OpenZFS and the code from zfsonlinux was brought in, why the hell should someone do work on a Filesystem but just for one OS?

FreeBSD.org forums: "FreeBSD moving to ZFS-on-Linux"

https://forums.freebsd.org/threads/freebsd-moving-to-zfs-on-...

They took the zfsonlinux-code into OpenZFS, so OpenZFS is based on the zfsonLinux features and NOT the illumos (solaris) one:

https://openzfs.org/wiki/Main_Page

>The OpenZFS project brings together developers from the Linux, FreeBSD, illumos, MacOS, and Windows platforms. OpenZFS is supported by a wide range of companies.

Filesystem implementations are normally OS-specific because each kernel has different infrastructure for filesystems. e.g. Linux has the unified page cache, dentry cache etc, and ZFS just ignores all that and ships its own ARC and dnode caches, that then have to compete for shared system resources.

That's why SPL is such a nice feature, very little OS specific code. ZFS was ported in ~10 Days over to FreeBSD. And is (hopefully) in the near future THE open and "real" portable filesystem for Mac/Linux/BSD/Windows.

Reply on news.ycombinator.com