Hacker News Reader

Comment by aeonik | Hacker News Reader

Everyone who uses BSD says the same thing, but it's still unclear to me what you mean. What inconsistencies does Linux have that BSD does not?

I've looked for an explanation, but I only get high level explanations, or specific features like jails.

You wouldn't happen to have more information, would you?

View on HN

firewalls. is it iptables, nftables? what is iptables-nft and iptables-legacy doing in this mix? or was I supposed to manage them with firewallctl or ufw?

network settings. if-up scripts? NetworkManager? Are we already supposed to use that systemd-network thingy or is it still not ready ? I just need to add an IP address in addition to the one given by DHCP...

who is managing /etc/resolv.conf today?

(my regular frustrations when dealing with both Ubuntu and Rocky Linux hosts..)

> firewalls

This seems like not the best example since FreeBSD has 3 implementations that may be in use depending on the whims of the sysadmin.

FreeBSD has 3 different firewalls, not 3 different interfaces to the same firewall. Each firewall has its own purpose. IPF is lightweight, pf has a nice UI/UX, ipfw is very integrated into the system.

More importantly, doing a simple kldstat would tell you which firewall is running. On Ubuntu (as an example) I have no idea if I should be using nftables, iptables or if ufw is working or not.

> both Ubuntu and Rocky Linux hosts

This is a little like comparing FreeBSD with macOS. Their userlands share similarities but they are different operating systems.

That's the main problem with Linux these days: Experience with distro A rarely transfers to distro B.

Also, at least with Ubuntu, switching to a new LTS means that most administration tools have been replaced with different (usually buggier) ones, so knowledge of the old release doesn't necessarily transfer either.

It wasn't this way in the early days, but the community focus stopped aligning with end user interest about a decade go. At that point fragmentation + complexity exploded.

I say this as a big time BSD friend, the same can be said about the BSDs. OpenBSD and FreeBSD are very different , I’ve never used NetBSD, but I can only imagine it’s not the same as the other two.

Yah, it's a bit wrong that people compare an operating system like FreeBSD (or Solaris or AIX etc) to "Linux" which is just a kernel. The distribution IS the operating system, and of course there will be differences.

SystemD is changing things up a bit and packaging up all the "boilerplate" and making things more consistent across distros, which is convenient sure. I joke that the old adage "GNU/Linux" should be updated to "SystemD/Linux".

I agree with you, FreeBSD should be judged by its own right against every other operating system out there, including the 100s of GNU/systemd/Linux-distributions, and every obscure operating system out there. How deep you dig depends on you.

My preferences have fallen on a combination of FreeBSD, OpenBSD, Manjaro Linux, with FreeBSD my main operating system.

The main draw backs are

1) poorer wifi support *

2) non-existing bluetooth support

But the main advantages of FreeBSD

1) FreeBSD is a source distribution first, always has been, always will be. 2) The most permissive software licenses are prefered, which I think is really cool 3) By far the best package managers. both ports and pkg are simpler to use than anything I have tried from any other distribution. I know some people swear by Slackware, Gentoo and Arch, but in general their package management do not appeal to me. Plus it always seems like the linux distributions are either source or binary. Sure, you can usually do both(except for the source first distributions) on most linux distributions, it's usually inferior to ports/pkg.

4) first class ZFS support

5) I get to run the same system on my desktop as I do on my production systems which I consider a big advantage.

* https://man.freebsd.org/cgi/man.cgi?query=wifibox

I have resolved the WIFI support by running wifibox, a tiny virtualized Linux vm running on bhyve. It gives me a 20-fold increase in speed! Coincidentally, it’s based on Aline Linux, which the blog post is all about!

When I want to play games, I reboot to Windows or Manjaro, which takes about 60 seconds... Both fairly stable and easy to maintain operating systems. I like MacOS as well, but I don't have any apple computers anymore.

The first 3 points are pretty much covered by nixos too. 1. It's source compilation based, but you download the cached result if it exists. 2. Unfree option has to be explicitly set if you want that for specific packages. 3. Depends on the tastes, but it's pretty easy.

It's been a while since the first-class ZFS support had any advantage for the user beyond an initial install. Maintenance on it was so limited that they ended up rebasing on ZFS on Linux anyway, making it literally less first-class than on Linux.

Today you can get ZFS packages from contrib in Debian and run it for several years with no problems. I know because I did that from Debian 9 (2017) through Debian 12 (2023) and still going. Ironically, Debian 9 took over that ZFS pool from a FreeBSD server, and there is not one part of that migration that I regretted.

The distribution IS the os, until you complain about ports packages not compiling, then suddenly it's not their problem, contact the ports maintainer.

Let's see with 24.04.

Afair, for switching from 20.04 to 22.04 I had to ensure network configs are under netplan and that's all.

What's imimportant as well, there is no rush to switch to newer LTS, no problem to plan and test migration over 1 year be needed as old LTS is still supported.

The question was why people may find Linux inconsistent. The distributions are very much part of that, and even within a distribution, just a 2 year later LTS might have wild differences because of the kernel promoting new mechanisms

It may not be that much of a problem in practice, I deal with multiple distributions because for new servers, we pick 'm based on expected future support, and they're only a bootstrap to docker/podman which is the great 'equalizer'. So the inconsistencies are only a problem until our Ansible scripts have learned the difference and when we need to debug an issue. Not that often fortunately, once the configurations are in place things are generally stable.

As a greybeard GNUlinux sysadmin: nftables raw ripping out iptables (newer gui/tui firewall interfaces support nftables) rip out NetworkManager, and use systemd-resolved to manage DNS. (On non-systemd systems like Devuan then this changes.) Use systemd units for powerful program and service control, including systemd-nspawn for containerization.

systemd-nspawn is like a secret weapon. Very few resources about containers mention it. I use it all over the place.

The fact that you have to rip out that much software to make it reasonable is a fantastic argument for the BSDs.

? FreeBSD ships with three different firewalls in base.

Yes, however, each has a clear set of tools, and it's clear which one are you using. There are no shims to use IPFW tooling with PF and vice versa, while on linux they are all mixed.

iptables has been with us for more than 20 years and is only now being replaced (pretty slowly I might add). The old rules are still supported through iptables-nft, you can just import them and forget nft exists.

Distributions I prefer have never used NetworkManager and haven't changed network configuration in a long time. RHEL and its rebuilds have used NM for what feels like an eternity. Ubuntu is the odd one out here with its constant churn, afaik.

Same with firewall wrappers like ufw and firewalld. Either your distribution uses one and you just use whatever has been chosen for you, or it doesn't and you go with nftables (or iptables-nft if you prefer).

This is all only really a problem if your organization uses a bunch of distributions instead of standardizing on one, but then you probably have a lot more other serious problems than learning how to configure your firewall...

As a counterpoint, I evaluated FreeBSD for a project about a year ago and was really put off by its primitive service management (compared to systemd which I know pretty well and use its features extensively, they really do help in your daily work), and the three firewalls which all seem to be approximately equally supported and you never really know where to put your time. (Unfortunately, I had to pass the OS for other reasons which have no relation to its technical merit.)

Sorry, for such inconvenience, we will stop writing software we want so that we won't risk filling BSDers brains

I really don't get these criticisms, you have choice, having choices doesn't make a system bad, makes you have to make your choices, which can also be going towards systems where stuff is standard

See paradox of choice.

Not having any choice isn't great either. See Soviet grocery shops.

Choice should only be offered after you have a stable foundation/base. Suppose you have a store that sells frozen food only, an incredible amount of choices, but no base ingredients like flour, grains and meat.

Software is utilitarian in nature, the goal is the task, but, how do you accomplish a task with an infinite amount of tools? and not only that, but how can you be sure that the tool is secure and stable?

In the context, you accomplish task with the defaulty installed iptables

What is the BSD idempotent cattle deployment pipeline? It's all just configs selecting packages in the first place is it not?

Or, and hear me out... Don't install it?

Or just to avoid a systemd distro.

I've had nothing but issues with systemd-resolved.

Networkmanager seems to be what things are standardizing on these days. Which, while for some reason I've always avoided networkmanager and used various combinations as alternatives, I'm all for having one most common standard networking utility for Linux.

Same here. However, from what I've seen, touching any systemd component causes cascading issues.

I usually settle on networkmanager, since there's not a great alternative for dealing with wifi. However, it often delegates to a giant broken pile of fail.

Things can be much simpler on machines that connect via ethernet (including VMs).

You might want to have a look at IWD, but read the docs to see how to configure it to work nicely with networkmanager or systemd-networkd

NetworkManager and systemd-resolved are not really interchangeable. The latter is a local caching multiprotocol name resolver and NetworkManager supports its use for name resolution.

In practice, do you interact with many different distributions ?

I only interact with Debian, sometimes Ubuntu, and that description of the Linux situation is fair and accurate. I love Linux, but it’s also a chaotic mess, just as described.

Ubuntu LTS 18 & 20, and decided to move to Rocky 9 for the future.

Firewalls is not where I would argue that BSDs are more consistent. FreeBSD comes with three different choices in base: ipfw, pf, and ipf.

Linux user since 1994, tiny kernel developments eons ago. My whole home depends on Linux and Home Assistant, Vaultwarden and a few others.

I love Linux but the total mess with networking and sound is disheartening. It is a pile of crap.

I do not care if this is this or another solution, but for fucks sake - let's have one system and not five that step on each other. This is infuriating.

As well as what others have said: With FreeBSD things change over time, this is a a given. But I can always use the release notes and the FreeBSD handbook to resolve any issues.

With GNU/Linux things change and the lack of authoritative documentation is tiresome. I end up on Stack-overflow triaging legacy posts from sources I just cannot trust.

Is FreeBSD perfect? No of course not. Is Linux a complete waste of time? No of course not! BUT my time is better spent in FreeBSD (and yes even windows) than Linux.

The big problem I have as BSD person who occasionally needs linux for something is that the google search results are terrible. Even when specifying the OS, I often get incorrect results that show how one would do what I want to do 8 years ago, when the tools were different.

Its a userbase problem, nothing anybody can do about it.

Many Linux distros have handbooks and release notes, too.

https://www.debian.org/doc/user-manuals

(How deep do you want to go?)

I have a job with Linux again, so I set up a dev vm, and I've got prod hosts on gcp that run our application in a container...

On prod, I have to use ifconfig, because ip isn't installed. On dev, I have to use ip, because ifconfig is 'obsolete'. Same deal with netstat vs ss. Those are the big ones for me.

I don't particularly care about the progression of firewalls on Linux, it seems like one day it will be back to one, but FreeBSD has always had three and I use two of them simultaneously, so how can I complain?

It seems like the real problem is that you've got a ton of drift between your Prod and Dev environments. How old is your Prod compared to your Dev?

Since you're running something in a container, you should be able to upgrade the host OS (more accurately, recreate the hosts with current images) very easily. I wouldn't expect there to be more than a few weeks of drift between Prod and Dev.

Our GCP prod is running google's container optimized os [1], something relatively current? But the container we run our app in is debian something, and my dev environment matches that... But when I go to prod, I don't go into the container, because a) I don't really understand how? b) I don't need to anyway; I can do almost all the stuff that needs doing without getting a shell into the container.

The real problem is that ifconfig can do the job, but nobody wanted to modify it, so they built a new tool, but both tools still work, and nobody is going to cleanse the world of ifconfig. Same deal for netstat vs ss. ss does the same job as netstat, but supposedly faster or better? but they could have made netstat better and didn't. In the meantime, causing churn and leaving a wake of broken documentation and frustrated users.

I didn't pick the environment, it was there, and I don't have enough authority to change the environment that much, I'm just working part time, and I want to get in, do my work, and get out. It sucks having to use two different tools for the same thing, all over the place. If I had an open offer to come work part time with a former boss at a place running FreeBSD and Erlang, I'd have taken it, but I got Linux and Rust instead, so I'm dealing with it :P

Of course, FreeBSD isn't perfect either. I've just updated a machine to FreeBSD 14, to find that I can no longer use the command

    ifconfig pfsync vlandev main vlan 4 192.168.4.11/24

because I get an error

    ifconfig: both vlan and vlandev must be specified

Instead, I've got to put the vlan number first. So I've got that to chase down, and freebsd-update was also very slow, so I've got that to chase down too.

[1] https://cloud.google.com/container-optimized-os/docs/

This is actually really interesting!

We run a lot of containerized stuff too, but in AWS EKS. For us, access to the container host isn't really a thing, and I'm not even sure if it's possible to be honest. The container hosts are an implementation detail, any direct access to the containers is through `kubectl exec ...`, so the tools available there come from the container image and match from dev through to prod.

I do agree with you though. It really feels like the change from netstat to ss, ifconfig to ip, etc, is churn for the sake of churn. FreeBSD is nice because it's a comprehensive operating system. Linux is a kernel used by a whole bunch of different, but fairly similar, operating systems.

My Debian server has a mix of systemd and /etc/init.d startup scripts. That’s the sort of thing where a BSD would be likely to say, ok, as of version N due in 3 months, we’re migrating everything and going from 100% old way to 100% new way.

> My Debian server has a mix of systemd and /etc/init.d startup scripts. [...] ok, as of version N due in 3 months, we’re migrating everything and going from 100% old way to 100% new way.

You'll be happy to know that this is going to happen. Quoting https://github.com/systemd/systemd/blob/v255/NEWS

> Support for System V service scripts is now deprecated and will be removed in a future release. Please make sure to update your software now to include a native systemd unit file instead of a legacy System V script to retain compatibility with future systemd releases.

Eh, I don’t really care either way. The ones I really care about are in my shell history. I meant that more as an example of where the BSD way would be different. Since they manage all of the software together, it’s much easier for them to do wholesale migrations like that.

Who can keep up with that? OpenBSD is a placid pond by comparison.

Long ago, it was that a linux system was "linux kernel, big pile of random GNU utilities for userland" while a BSD was "every utility is written by the same team."

This ends up with the documentation "working" and very consistent vs a linux (jeez, no man pages, what the heck is this "info" crap?).

Things are better now, especially in the documentation front, but it remains that the "gnu" ecosystem is still a hodgepodge of different utilities written by different teams at different times and there are still inconsistencies.

I've spent decades in the linux ecosystem and away from "unix" so the memories are hazy and the brain damage (from linux) permanent at this point; the fish no longer notices the water.

Yes, this has been my experience as well, even at the kernel level. I actually enjoy looking at and hacking on BSD kernel code. Linux kernel code is ... another story.

But I tend to think the difficulty with Linux/GNU is more a result of the enormously larger community and the huge diversity of use cases. For example if you stick with a complete vendor (Red Hat being the best example) among the system tools is a fair amount of consistency and documentation (man pages). As you accumulate more applications and extra tools, that's where the community fragmentation really hits. This is most intensely felt when I try to set up a workstation (laptop or desktop) with a BSD. Even discounting hardware support, I run into so many things where BSD is consistent because in part because it doesn't exist there.

I still have a dream though that one day BSD will start becoming the GOTO for various places. Though, I think how Apple took it and made it common but also locked it down, I have my suspicions that the permissive licensing (which as a developer I really love) does seem to end up being taken by big tech to get huge profits and used without giving (much) back.

As others have alluded, greater Linux is awash with too many choices for every component, like a walmart supermarket, not to mention CADT-driven development.

https://www.jwz.org/doc/cadt.html (cut/paste link)

BSD is closer to "omakase" in that sense. Pick your cliche... people "rowing in the same direction," "more wood behind fewer arrows," etc.

Reply on news.ycombinator.com