Hacker News Reader

I can talk to the opposite side about why I am very happy going all in with NixOS. Prior to that I was running dual-boot Windows (for games) and Ubuntu (for coding). Despite all of the progress made on gaming on Linux, I was still having problems with the Nvidia drivers in Ubuntu, sway refused to work, hyprland glitched out, and I was stuck on i3 (because I am a full convert to tiling windows managers). I used home-manager with Nix to manage my dotfiles and had been quite happy with that workflow.

As the story goes, Ubuntu had just released 24.04 and I pushed to upgrade. My linux partition was on btrfs. I had been bitten by the snapshots-locking-my-system problem before so I cleared out all my snapshots (and backed up my drive to local NAS). During the update Ubuntu did ANOTHER SNAPSHOT, it took up all the space on device (256GB, yeah I should have had more space) and crashed the system. At that point I was finally done with all of the shenanigans that Ubuntu pulls with apt (not updating everything unless you have their subscription, forcing some installs to go through snapd, separate driver management from apt…), and this last update crashing my system’s boot, even though I could have recovered with my backup, was the last straw.

I considered my experience with Ubuntu, and MSFT’s actions to integrate Copilot into the OS, and decided to wipe the entire 1TB drive and go all in with NixOS. I was able to move my home-manager config right over, get my configs all converted to flakes, and get hyprland set up all in less than an hour. Smoothest transition in a long time. I even was able to jump to the 6.8 kernel and things just started working again. I was finally able to play steam games well on my system again. I may even have a path to get my VR to work in Linux again.

Are there still problems? Sure. Hyprland is a compromise because sway still won’t cooperate with Nvidia GPUs. But, when I have made changes that get me to a non-bootable state, I can boot to a previous generation directly from GRUB, see what went wrong, fix it, and get back to a good state. Having that ability to recover from a bad boot situation without hacking into a safe mode or GRUB console or rip the drive out and put it into another system was incredibly refreshing. I am not going back anytime soon.

TBH, I mostly find this to be a (very valid) story about why Ubuntu is problematic. Most of my team around me uses Ubuntu and periodically has to wrestle with some issue, and have to wipe and reinstall every year or so.

This Fedora installation is 5 years old now. I have done a distro upgrade at least every year. It has worked extremely stabley and predictably all the way though, including the Fedora transition from X to Wayland.

I recently helped my friend install his first Linux distro, Ubuntu. He told me he could not log in, and he kept getting back to the login page. I realized that he might be having problems related to his display manager installed by default. I made him go to virtual console and replace the default display manager so that he can start his desktop environment session without getting stuck in a display manager loop.

I did not like my own Ubuntu experience when I was using few years before as well. I don't think its stable enough for newcomers as well.

> As the sibling comment describes, Nix can get a bit gnarly once you're off the beaten path, and having your whole OS subject to that is a pain.

For me, years and years ago, that 'pain' (having my whole 9S subject to Nix's constraints) is what pushed me to really learn it instead of just retreating to the traditional base system. It transformed learning Nix (and even sometimes writing packages or extending NixOS) into the path of least resistance to run the software I wanted or needed to use. And that was helpful for me.

For others (and for me at other times in my life) I understand that this is not always the right approach. But it can be really good.

I actually would go the other direction: I actually moved from Arch to NixOS just to get fully-supported out of the box ZFS support, including ZFS root. I had even intended to use NixOS as a thin layer that provided my "base" OS but run all the real stuff in distrobox (likely even with Arch in the container, effectively returning to my old setup but with ZFS underneath), but then when I actually used it I found it easier to just do most things natively.

Depends on what support means. If support means root on ZFS can be installed effortlessly, then yes. But if support means if things failed and you can get support to fix it, then no.

> But if support means if things failed and you can get support to fix it, then no.

Well, yes actually, you can get help in IRC/Matrix/etc. It's best effort and might not help, but it's not like commercial vendors promise fixes either.

In any event, Arch is in the same position so that's not really an argument in favor of either one.

Right. I'd like to hear your experience with running OpenZFS on nixOS. E.g. how long have you tried it and your update cadence and robustness etc.

I'm in my quest to find the "best supported" way to run ZFS on Linux and I've tried Ubuntu, Arch, RHEL, OpenSUSE. (I also tried FreeNAS (i.e. FreeBSD) in the past.)

Ubuntu has knowingly made a version of OpenZFS with (with their own patch causing) data corruption bug in the stable release. So since then I lost trust in them.

Arch Linux never officially supports it, and Arch's rolling release model with OpenZFS as kernel modules are not a good match for UX.

RHEL with kABI tracking kmod maintained by upstream seems to be a great, robust support combination, but later I found out that the kABI is not fully covered so it working relies on a bit of luck. AFAIK, it is still not completely resolved.

OpenSUSE with the "filesystem" project seems to be a semi-official support of seamless integration of OpenZFS with OpenSUSE. So far so good but I'm still evaluating.

In principle, nix and nixOS should be quite robust from its design (practically reproducible build). But I am not trusting nix that much yet and prefer something more battle tested. Coincidentally, I like the documentation of all the previous OSes I mentioned above (where Ubuntu falls a bit shorter among the crowd.) And from my limited experience, nix and friends are quite poorly documented.

> I'd like to hear your experience with running OpenZFS on nixOS. E.g. how long have you tried it and your update cadence and robustness etc.

My main laptop, and the main laptop before it, have run NixOS with ZFS root since November 2022. Commit history seems to show that I started on version 22.11 and I think I've stepped forward through every version since then (might have skipped one in the middle, not 100% sure; I didn't switch to flakes until partway through and channels aren't in the commit history). In that time, the only problem I can recall ever hitting is accidentally filling up the /boot partition because I wasn't cleaning up old generations. If I was doing it over I would either make /boot like 8GB, or just use zfsbootmenu so the kernel+initramfs live in the root filesystem and it's irrelevant. Other than that, it's been smooth sailing.

> I'm in my quest to find the "best supported" way to run ZFS on Linux and I've tried Ubuntu, Arch, RHEL, OpenSUSE. (I also tried FreeNAS (i.e. FreeBSD) in the past.)

Ah yeah, if the rest of your software+hardware+preferences support it, I would easily believe that FreeBSD is better at ZFS than any Linux distro. IIRC they even ship boot environments in the base system which is great.

> Arch Linux never officially supports it, and Arch's rolling release model with OpenZFS as kernel modules are not a good match for UX.

IIRC there's some unofficial repo that provides both the ZFS packages and kernels together? In any event, in NixOS just use

    boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;

to pin your kernel to the latest compatible version and you're set.

> In principle, nix and nixOS should be quite robust from its design (practically reproducible build). But I am not trusting nix that much yet and prefer something more battle tested.

One advantage of ZFS is that you can compliment nix's versioning (seriously I do recommend using flakes and keeping your system flake in version control) with regular old ZFS snapshots. Just run `zfs snapshot -r rpool@todaysdate` before any change and you can roll back just about anything. Also you can zfs send those snapshots to some external storage and have full system backups with easy incrementals as desired.

> Coincidentally, I like the documentation of all the previous OSes I mentioned above (where Ubuntu falls a bit shorter among the crowd.) And from my limited experience, nix and friends are quite poorly documented.

Yeah, that's fair. It's improving with time, but anything off the beaten path tends to devolve into random blog posts. OTOH, if you keep the NixOS system to a simple base system and run most of your stuff in distrobox or some other container, you can have the best of both worlds.

Thanks. Interesting to know!

The thing I worried most is not the kind of problems fixable by rolling back, but (potentially silent) data corruption. Do you know how exactly is OpenZFS on nixOS supported? For example, currently both the Arch Linux and nixOS guide in the OpenZFS documentation said people should mention @ne9z in filing issues on those platforms, but @ne9z no longer exists.

I have seen how maintainers can screw up and causes data corruption (Ubuntu specifically) so I worry about that aspect as well.

Hm, so I've not really touched that side of things; mostly I just interact as a happy user. I think the relevant sources are https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specifi... and https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specifi... , with the former listing a couple of maintainers and the latter I think showing that NixOS is taking upstream OpenZFS with zero source code patches (because mkDerivation is passed `patches = extraPatches;` and I'm reasonably sure `extraPatches ? []` defines an empty list). I'm not sure why those two files list one maintainer in one and two in the other.