The Future of ZFS on Ubuntu Desktop Is Not Looking Good
omgubuntu.co.uk
omgubuntu.co.uk
* zsys has both implementation and design flaws. They utilize two zpools - one with limited feature flags that GRUB can use and one general pool, and then try to tightly couple datasets between those two pools while coordinating snapshots. zsys routinely runs a system out of space due to not cleaning up snapshots. It's been known to destroy user data as well.
https://github.com/ubuntu/zsys/issues/196
https://github.com/ubuntu/zsys/issues/218
* Their packaging was and is still very sloppy. It's quite common to catch people running one version of kernel modules and another version of userland tools - often multiple releases apart. This has led to some serious problems.
(no direct source for this, typically noted in libera.chat#zfsonlinux)
* Ubuntu has created and applied bad patches to ZFS, resulting in destroyed pools.
https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/190...
If your exposure to ZFS is through Ubuntu and zsys, you'd naturally have a very poor opinion of it. ZFS, of course, isn't with out flaws. Ubuntu is not helping.
As a long time ZFS user, I was excited when Canonical promised first class ZFS integration. But it was clear from the first release that Ubuntu's ZFS project was a mess. Clearly some folks got excited about ZFS, but didn't absorb much of the existing knowledge or wisdom in the ecosystem. Issues like dataset/pool layout, how snapshots were handled on update, were handled in completely unusual and boneheaded ways.
The ecosystem already had the concept of boot environments, which more or less standardized what a rool-on-zpool layout ought to look like, and how to handle graceful, recoverable system upgrades. What Ubuntu could have brought to the table was a really high quality beadm replacement for Linux, moving Grub's ZFS support forward with support for more feature flags and boot environments, or investing in ZFSBootMenu.
Instead they went ahead with zsys, and it's still not entirely clear what they want or expect it to be. That they needed to write a daemon to enable "complex zfs dataset layouts" should've been an early clue that they were going down the wrong path. The dataset layout for a root filesystem should never be so complex, that you need to write a whole daemon to manage it.
The packaging issues and bad patches are almost besides the point.
Edit: handcuffing yourself to grub seems pointless. Not to mention, grub is controlled by oracle devs who want nothing to do with zfs support and have actively ignored bringing the grub zfs in line with openzfs mainline.
Disappointing, because Zsys is quite good. I believe it would have gained prominence among competing ZFS boot environment managers if it hadn't been so tied to Ubuntu.
Ubuntu was the first Linux distro that I could actually boot, back in 2007. Steam and ZFS were reasons for me to continue with it in 2020, but now, everything I want to do can be done better on NixOS.
All RAID1, backups to object storage.
https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu...
but for desktop use, it's probably fine. It's been the default for new Fedora installations for over a year now, and I don't see too many Fedora users complaining about their filesystem being hosed.
My current installation is still legacy EXT4 since I upgrade, but at some point I might blow it away and try a fresh install.
When the file system performance is important, XFS and EXT4 are the fastest. When the extra features provided by ZFS are important, it becomes the appropriate choice. For BTRFS, I am not aware of any application for which it would be the right choice.
But for the second one: when a file is striped across different physical disks with possibly different characteristics, how would you defrag it without knowing the read access pattern?
Looking at a single disk at a time, do your best to make sure all chunks of the same file are in order and adjacent.
You could be smarter than that, but just that would do fine.
The problem is that ZFS basically can't move things.
The one killer feature for me with btrfs is offline deduping (and by extension not having de-duping require a super computer to run). My data has enough duplication that it'd be nice to have, but not worth requiring hundreds of gigs of ram. Being able to fire off a job after importing a large set of data (or maybe yearly-ish) would be awesome and give me a few extra years without having to update my hardware.
In that case you want to know if you optimize for hardware failure as opposed to for performance.
This could be fixed for specific usecases but not all block patterns in general. Just why I posted this.
Also a lot of other fancy features would be broken by pagecache design, so it's a no-go, and instead project focused on reducing duplication between the two.
On the other hand, I have not had much need for defragmenting.
Only in very limited instances. I think ZFS duplicates the inodes and dentries cache, and only file pages when you do something like mmap a file.
The ability to add capacity to a pool by adding a single drive at a time, or the fact that it is readily available on any Linux box.
zfs supports that now
There's a reason the other major non-debian distros have switched to it by default.
I'm still waiting for ubuntu to allow installing using btrfs + fulldisk encryption by default, which is currently a massive issue.
Not sure where you got this from. OpenZFS on Linux is considered stable by the project. It has been rock solid, free of major issues, and considered more than ready for prime time since at least 0.8, and probably much earlier. It’s now at 2.1.
I’ve been using it across multiple environments, including my personal laptop and workstation which maintained the same root-on-ZFS install for 6+ years, including 2 full migrations to new hardware using zfs send. I’ve deployed much larger pools in professional settings and they’ve been rock solid.
Btrfs has had numerous data loss issues in that time. I can only recall one data loss issue in a ZFS on Linux release, several years ago, which was caught very quickly.
Btrfs hasn't had any major data loss issues for the last 5 years (excluding the raid issues) btrfs is much newer than ZFS, so obviously it took longer to be stable, if you're talking about 6+ years ago, Btrfs was obviously not ready yet, but now? It's been the default on OpenSUSE for 5 years and Fedora for 2.
“I can’t believe Nixon won. I don’t know anyone who voted for him.” -- Pauline Kael
I use ZFS on root. Know lots of people who do. Might be your bubble?
That says more about the distributions than about ZFS’s fitness or maturity as a file system.
It’s understandable that many distributions don’t bundle ZFS, since the licensing is still an open debate. Although that’s not so cut-and-dried [1], I wouldn’t expect broad support from distributions unless and until these issues have been tested and proved out one way or the other. No organization wants to insert themselves into a legal skirmish.
The best (and probably only good thing) to come out of Ubuntu’s support for ZFS is the vote of confidence in favor of ZFS’s legality in-base from a major corporate entity. We can hope more distributions pick up on this, but I’m not holding my breath.
But legal issues and technicalities say nothing at all about the relative technical merit of ZFS (or any other file system).
> how many people use it as a filesystem for the linux root? Never seen anyone putting the main OS on zfs
Quite a lot of users, actually. There’s a significant community around ZFS, including an ecosystem of some robust tooling. ZFSBootMenu is evidence enough that people want to run root on ZFS, and that root on ZFS is both feasible and highly desirable. Compare its feature list to what you can achieve with root-on-Btrfs, and ZFS pretty handily comes out ahead as the more powerful file system in terms of real world administrative capabilities.
[1] https://blog.hansenpartnership.com/are-gplv2-and-cddl-incomp...
Although I'd still suggest single-block-device-btrfs on mdadm instead. The per-file RAID in btrfs (/zfs) can skip over unallocated sectors, but mdadm gets to do larger sequential writes with less random seeking. On spinning disks this can very quickly overtake the performance at even lightly filled volumes.
Maybe its just to much work?
But then other distros have implemented many of the features with BTRFS so it can't be that much effort.
What would this look like? In what ways do existing distros fail to live up to that vision?
That is not given if you have one-off custom integration of these parts. For a while, I had a CentOS box with openzfs kABI modules; having to tinker after updates to mount the pool was not fun.
ZFSBootMenu is developed natively on Void, and their wiki has bootstrap instructions for a Void install with root on ZFS. Everything's well supported, including native zpool encryption. Plus gives you a multitude of tools right in the bootloader -- snapshot rollback/management, boot environment creation/management, booting older kernels, and a full recovery command line with zfs available. You can tweak the bundled dracut config to include other tools as well.
Even supports generating a unified, SecureBoot-signed EFI binary.
The experience is leagues ahead of the Ubuntu mess.
https://docs.zfsbootmenu.org/en/latest/index.html
https://docs.zfsbootmenu.org/en/latest/guides/debian.html
https://docs.zfsbootmenu.org/en/latest/guides/fedora.html
Void remains our primary development platform, but we're working on improving documentation and support for other distributions.
NixOS currently has this! Since upgrades are atomic, if you have some other version constraint that pushes you towards a kernel version that ZoL doesn't target or support*, your system just doesn't finish building the upgrade and never switches over. If the system upgrades, that means the kernel was built and ZFS was added to your initial ramdisk successfully.
In case that succeeds but there are still ZFS issues (which I've never seen or heard of), NixOS will still let you boot the old, working system until you figure out what messed you up in the upgrade. Just select it in GRUB or whatever you're using.
Maybe NixOS overall is more DIY than what you're looking for, but the ZFS support feels very first class! I personally recommend giving it a try, if you're up for giving a declarative distro a chance.
--
*: I've never heard of this happening except to NVIDIA users, since NVIDIA's proprietary kernel module imposes restrictions on how new of a kernel you can use. On AMD, I've never had a ZFS-related update issue.
It is both a technical and legal problem.