Hacker News Reader

Comment by macksd | Hacker News Reader

I'm a big fan of BSD and it's philosophy, but I think it's relative lack of adoption is now a vicious cycle and it's falling behind. It is better in some specialties, IMO: if I was setting up a NAS server, some serious firewalls, or traditional web servers, I would give BSDs a serious look first. The documentation is great, and there's a lot to be said for it's simpler, cleaner implementation. However, I think Linux is winning still. I need CUDA for work, and frequently use software that supports Mac, Windows, or Linux. So for my daily driver or other day to day usage (like cloud instances), I wish I could use BSD but I just don't see it as being a viable choice anymore.

View on HN

I think the case for running FreeBSD in some of those cases is dwindling a bit.

There are a few advantages freebsd has historically had, one being ZFS. Recently, though, freebsd switched their ZFS sources from the upstream Illumos to linux's version of ZFS. Not to say this negates freebsd's integration advantage, but I think it shows how linux's ZFS has grown really good over time.

Another one I've heard is FreeBSD handles really heavy TCP/IP traffic better than Linux. Not sure about this one since I've never had heavy enough workloads to test and don't know the theoretical basis of it, although Netflix seems very happy with FreeBSD for this reason.

Also, as you say, BSDs are just nice to use sometimes. And sometimes all you need is something just good enough to run x/y/z. Maybe the pleasure of using it can offset the performance losses for some people.

As long as linux has to hack zfs into the kernel with all their shims to avoid breaking license terms, I'm going to consider zfs a FreeBSD advantage. I've heard too many people complaining about issues with zfs on linux, so as someone who likes to configure once and then forget about it for a few years, that's not for me. I don't want to update my system one day and then find I can't mount my filesystem anymore (or worse).

For networking performance, I doubt it makes a noticeable difference in personal use. Firewalls are pretty nice on FreeBSD though, at least for my uses.

Of course, it's not like FreeBSD doesn't have problems of its own - I've had multiple times where I boot into my machine and find that I don't have graphics until I build the drivers from ports because their binary packages were compiled against an earlier release, leading to kernel incompatibilities (also can't figure out how to get hdmi audio out of my current card, not a deal breaker but wasn't what I was hoping for either).

Pick your poison I guess, but until linux gets a comparably featured and stable filesystem I have a tough time seeing myself move away from FreeBSD. I've definitely considered various linux distros due to certain applications missing in FreeBSD or just general pain points though. Just very hard to leave zfs, and also most linux distros either seem to be stable with ancient packages, or always changing and often breaking things in the process. Stable base with latest packages is very nice.

I haven't used ZFS myself, but the people I know who have have never mentioned problems with it. This is on pretty exotic configurations, too, like POWER hardware.

Obviously distros trying to be especially kosher like Debian and Fedora will never include it by default, and Canonical is maybe stepping on thin licensing ice by including it with Ubuntu, but I get the impression it's pretty stable, especially if Canonical put it in a LTS release.

Something else I learned that might be relevant, iXsystems of FreeNAS/TrueNAS fame appear to be moving their development to Linux with "TrueNAS SCALE". Make of that what you will, I guess.

Due to the licensing issues FreeBSD is never going to be a first-class citizen in the kernel source tree, and that's the most optimistic possible view of the situation.

Linus and Greg K-H have both made statements and actions that make it clear that they are philosophically opposed to ZFS as an implementation ("more of a buzzword than anything else"), regardless of licensing issues. They want you to use BTRFS instead, and are going to throw up arbitrary barriers to make ZFS usage and development as painful as possible (like re-licensing non-GPL symbols that they notice ZFS using out from underneath it into GPL).

https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-s...

Like, people make a huge racket about how NVIDIA's driver isn't in the kernel and how that means it will never be suitable for day to day use in a desktop environment, because it's out-of-tree and thus will always be in a place of reacting to changes in the kernel as an external team rather than being supported by the kernel team itself. And that's exactly the same situation ZFS is in and will always be in on Linux. And in fact the situation is actually worse than that, while CDDL is open-source it is incompatible with copyleft licenses, by design and by selection when Sun licensed the code. And then you've got the fact that the kernel team is stacked with BTRFS fanboys who are thinking and acting with their loyalties.

I get the desire to improve ZFS on Linux support but the FreeBSD version probably should have remained the repository of record so that ZFS could have been a first-class-citizen in at least one distro. The Linux team is legally and personally opposed to ZFS, it's ZFS choosing to place their focus on improving an abusive relationship.

Just out of curiosity, wouldn't it be able to setup thousands of ZFS test in a CI environment for every new linux development build, to see if there are any regressions, e.g. 1 failed, 999 passed.

Second thing, if more and more people start to use ZFS then it will automatically become a center of attention for linux kernel developers, because they don't want to break their users systems.

And one last note: not everything that is out-of-tree is automatically unstable, otherwise the whole userspace code would be unstable.

ZFS never came from FreeBSD -- its origin is Solaris, and specifically from the tediously open-sourced and then thrown under a bus by Oracle OpenSolaris, forked into the Illumos project.

So FreeBSD's version has never been the "master copy", so to speak. In some ways the most sophisticated version has always been the (now very aggressively close sourced) Solaris 11 ZFS, which is very incompatible with the open source forks because of new features added.

FreeBSD switched from upstream Illumos to the OpenZFS/Linux sources because Illumos development has slowed to a crawl, while ZFS on Linux is ironically very active and has more and more corporate backing, while essentially all corporate backers of Illumos except maybe Joyent (?) are dead, as mentioned on the FreeBSD mailing list. This means in practice development on that fork of ZFS ceased as bugs piled up, since relatively few hobbyists are going to deep dive into something as complex as ZFS. [0]

FreeBSD gained several new features from Linux's ZFS that it simply didn't have before, like native encryption support in ZFS itself without needing Geli/GEOM. I've heard several FreeBSD users get excited about this, so I presume the wrapper wasn't well-loved. Other features mentioned in the mailing list are "multi modifier protection, project quotas, encrypted datasets, allocation classes, vectorized raidz, vectorized checksums, and various command line improvements". It's simply superior to the Illumos upstream, and will only improve over time while Illumos sadly stagnates into oblivion.

The kernel guys are never going to like it, and I fall closer to their camp. I'd rather use Linux's native layering mechanisms instead of using an omnibus package. But I also think ZFS on Linux is in practice very widely used and very mature. Same as the nvidia drivers, too -- people endlessly complain about them, but even Fedora adds a button in the GUI for installing 3rd party repos with the drivers, and my nvidia blobs have survived for 4 OS updates without breaking. There are several OSes like Pop_OS! that just include them OOTB.

[0] https://lists.freebsd.org/pipermail/freebsd-current/2018-Dec...

> I've heard too many people complaining about issues with zfs on linux

I think this is mostly related to Root on ZFS, which is pretty standard in the FreeBSD world, but not very stable in the Linux world. If you want to use ZFS on linux I would recommend you to put your linux root on ext4 (or something similar battle tested) and create a ZFS pool on the remaining disks.

Used ZFS on Linux on NixOS as root for 2.5 years without any issues. NixOS does have good integration though, including automatically-run OS tests that verify that there are no regressions in ZFS support.

Idk Linux on desktop has regressed lately: OOM crashes/OOM killer kicking in (which maybe wasn't so great an idea to begin with), gnome3, snaps and other crap (that doesn't work well IME) littering mount points and other places with info used for internal purposes, systemd creating complexity for it's own sake (or rather monopolizing know-how at RedHat), wayland, bash/getline with vi editing mode broken more than it used to be ...

All the while I'm not doing things differently from a decade or two ago ): While Docker is working well for me, I'm not sure last decade's push towards containers and dockerizing everything is actual progress. Or rather, declaring defeat in front of self-inflicted problems with shared libs (which btw seems also to a large degree a problem with glibc's insistence on shared libs), language/platform ecosystems, and a false separation of concerns story mixing permission and lib versioning issues.

Yeah I'd love to see an update on this benchmark re: network syscall performance: http://bulk.fefe.de/scalability/. FreeBSD has very good results in some of those, but even then Linux was showing some truly impressive strengths of its own. And which kernel has gotten even more attention since then, and become even more (that is to say, completely) dominant in the Top 500? (happens to matter for me, even though it's a strong data point anyway).

I will say that performance aside, the UX of BSD firewalls is SO nice. Clear configuration files, features exposed simply, just wonderful.

RE: CUDA, I wouldn't say running a VM is an especially enjoyable solution, but the built-in hypervisor does have VT-d support https://wiki.freebsd.org/bhyve/pci_passthru

PCI pass thru, eh? That's pretty cool. I'll hand it to the BSD community (the community is another thing I love about BSD btw - attended MeetBSD a few years ago and it was such a fun crowd) - they create some killer compatibility layers. Like in this thread, that simultaneously illustrates this, and also why I'm afraid to put it on my primary work machine: https://forums.freebsd.org/threads/would-it-be-possible-to-p...

It's on linux too. https://www.kernel.org/doc/html/latest/driver-api/vfio.html

And on Oracle's VirtualBox, which is available on both OSes: https://www.freshports.org/emulators/virtualbox-ose

hopefully GVT-d soon[1] too!

[1]: https://reviews.freebsd.org/D26209

Reply on news.ycombinator.com