rc.d belongs in libexec, not etc
jmmv.dev
jmmv.dev
Unexpected to say the least.
Good read though, mostly agree.
> My main gripe is that the files under /etc/rc.d/ are immutable scripts.
Are they though? A lot of, well at least old software, lacked configuration files, their command line flags and unit scripts were for all intensive purposes, their config files.
Whether this actually makes upgrade harder (the next point made), is kind of debatable no?
Arch does fine with pacsave and pacnew files.
Making changes or tracking changes to these files is going to be required regardless from a systems upgrade point of view - because people want control over their system units.
And for what its worth, I'd rather find out about a breaking update via pacsave/new than figure it out months later when my copied but now outdated and untracked unit starts failing.
https://old.reddit.com/r/BoneAppleTea/
My favorite being a friend called a cul-de-sac a "culture sack". Years later we still laugh about this one.
Normal form isn't always the best answer, sometimes a flat cludge is the better solution.
All files that the distributor ships as configured ought to be put into /lib or /usr. Any system specific configuration belongs to /etc, but ONLY that configuration.
On a fresh install of BSD/Linux I'd love to see an empty /etc directory sans any necessary overrides to make it boot.
Makes backing up that directory a lot easier too.
And I rather have a full set of configuration files that specifies the defaults at the time of the installation so my system even boots after an update in which some program changed its hardcoded defaults.
> Makes backing up that directory a lot easier too.
I don't really understand what the problem with backing up 0.1 or 1.0MB is.
The issue is not how much is backed up but what.
If /etc only represents changes to system defaults, then the backup represents a list of my changes and it's always easily visible from each full snapshot what changes were active in the system at that time.
If /etc contains all configuration, then from a single backup snapshot, you cannot determine what changes where made to the system.
This would be an issue if you're trying to recover the backup to a freshly installed system, for example, because you cannot tell what changes you have made vs what changes occured in the backup due to updates or upstream due to updates since the backup.
It's a semantic difference that can make restoring backups easier and replicable across different systems.
b) Not all settings done in defaults can be simply undone later on (e.g.: apache's httpd.conf, IIRC the same for nginx - good luck getting this default listener down again)
c) If you want to infer which changes were made by you or by the system's update procedures, explicitly creating snapshots before every manual or automatic modifications would be the way to go. Preferably one after each as well, so you have something to verify the integrity of a system against in case of security issues.
c) As mentioned above, inferring changes requires two snapshots, if /etc only contains changes a single snapshot allows me to infer changes. And I can verify the integrity just as well using an "after" snapshot. Just means one less snapshot is involved, which is reduced complexity.
I hope that ELI5 was sufficient for you.
Most of the article is dedicated to explaining how modern BSDs changed from these origin and how system owners are very much expected not to edit these scripts so… yes, they are?
> Whether this actually makes upgrade harder (the next point made), is kind of debatable no?
Not it’s not? The article explains exactly how: BSD upgrade process diffs and merged every config file, which includes core rc scripts which the system owner would or should not have edited. So said system owner finds themselves spammed with every upstream update to scripts they never touched, none of which would be relevant to them.
If you argue that this is a good thing, then surely they should also be spammed with upstream updates to every binary, utility or script these use?
> Arch does fine with pacsave and pacnew files.
The article is not about arch?
> Making changes or tracking changes to these files is going to be required
No.
> And for what its worth, I'd rather find out about a breaking update via pacsave/new than figure it out months later when my copied but now outdated and untracked unit starts failing.
Does pacsave/pacnew force you to review every built-in systemd unit?
If the BSDs' upgrade procedure cannot distinguish between changes to /etc where the local sysadmin has also modified the file and changes to /etc where the file is unmodified, and they prompt you to review every single change to /etc, that seems like an easily fixable missing feature.
Do the BSDs really prompt you to accept every change to /etc on upgrade for files you didn't modify, such as sshd.conf or services?
I'd also note that the first time I had to debug a Tru64 networking problem it took me quite some time to find the bloody things.
I mean, once I did my first thought was "honestly if anything that makes more sense than /etc would".
But argh.
Then in the end it turned out the network configuration had been fine all along and one of the developers had stolen the switch port it was using for something else.
(being a sysadmin at a software house tends to involve the developers getting to do wtf they want since their work is revenue critical and yours isn't perceived to be so much, though I still mostly enjoyed the job because my director was understanding and let me yell at developers for doing stupid shit so long as he didn't get too many complaints ;)
Plenty of files that properly belong in /etc get edited rarely by the local sysadmin, perhaps more rarely than rc.d. (When was the last time you edited /etc/pam.d/ or /etc/gai.conf or /etc/protocols?) Yet they can be edited, and so they properly belong in /etc. If the local sysadmin doesn't actually touch them, an upgrade procedure should just upgrade them without prompting. (I assume the BSDs are clever enough for this.)
So, if you're getting prompted, it's not just because there's a change to review, it's because there's a merge to review, and therefore it's because there was a local modification, and therefore clearly either you or some other sysadmin did in fact edit that file. It is of course worth making it no longer necessary to edit it, such as by moving configuration to /etc/rc.conf for /etc/default/ or a service-specific /etc/foobard.conf, but that change needs to actually happen before you can call the rc script immutable, and once you do make that change, you no longer care if the rc script moves to libexec because you're no longer getting merge prompts.
That's the way it should be, but while upgrading Debian I have very often seen prompts for files that were never touched locally.
When you run etcupdate on NetBSD, the tool will diff your current /etc contents against the new versions shipped in the source tree or in the etc.txz distribution file -- and it will ask you to manually review every difference.
It doesn't matter if you have edited a file /etc by hand or not. If the currently-installed file differs from the new version in any way, you'll be asked about it.
I don't want to review diffs to pam.d or sshd_config or (as another user pointed out) bashrc and profile, either.
>My main gripe is that the files under /etc/rc.d/ are immutable scripts.
If the files/config (I know the author argues they aren't config files) are truely immutable why would the system upgrade command modify them? and how would it as if they are immutable in the same way that running chattr +i somefile on Linux will make a file immutable even to root then I don't understand how the upgrade command is modifying them.
Systemd-d is growing on me, partly because it is being forced upon us but also because once you get over the initial carpet pull from beneath you, you don't need to be exposed to the extra bullshit it brings.
If you want to experience pain with System-d, install a fresh copy of Ubuntu and try to setup /etc/resolv.conf, for an extra challenge try setup /etc/resolv.conf using unbound as a stub resolver.
I was amazed at how much effort I had to spend fighting the OS to just say "Stop managing this file, do not load this file, I am going to manage it"
I spend most of my time with Linux systems so I admire the simplicity of the System V approach. It is interesting to see other people discuss this from a BSD perspective as I have very little exposure to BSD like systems.
BSD seems great, I love reading the security.html page of the OpenBSD project, it has so many great ideas + implementations.
Updating these files is not in any way shape or form a hassle anymore if you do not update them. etcupdate has that solved. Even mergemaster has specific options to handle unchanged /etc that only diffs in svn-id tags and similar.
But in my opinion the basic premise of the article is false. mergemaster and etcupdate with their diff and 3-way merge capabilities are there because these files are assumed to have local edits.
The startup procedure, scripts, options are expected to be customized; thus the update procedure has handling to preserve local edits.
You are confusing the two different scenarios that you need to distinguish.
The first scenario is when a config file is simply a part of a package, managed by the package manager. In any reasonable package manager, if you ever edited a config file manually under /etc, the package manager will never attempt to update it automatically. For example, apt-get will ask you whether you need to preserve the old file, use the new file, or diff-and-edit it manually. This is the reason why the author suggests to move rc.d from /etc to /libexec, because every time you upgrade your system, the package manager will ask you to diff-and-update some random runscripts that you've never written.
The author's point here is that the rc files do not behave like config files, they are closer to programs or executables, the package manager should be able to update them when a new version in installed without annoying the user, just like any other binaries.
---
The second scenario is when a config file is managed by the init system, a script, or a daemon in the system. For example, /boot/grub.cfg is managed by update-grub & os-prober, and ideally one should never edit the file manually.
In this scenario, it's completely off-topic and irrelevant to the original topic of whether rc.d should be treated as libexec, but here's my opinion anyway...
> I was amazed at how much effort I had to spend fighting the OS to just say "Stop managing this file, do not load this file, I am going to manage it"
Never attempt to fight with the system in a brute force manner - edit this file, later the file is edited back, edit again, edit back, and finally "chattr +i" the file, I never understand this mentality. Using "chattr +i" is almost never the correct solution.
All you need to do is politely tell your intention to the daemon in charge, spending 30 minutes (or each time after a system upgrade) to fight with it, can easily save your 5 minutes to read the documentation.
> for an extra challenge try setup /etc/resolv.conf using unbound as a stub resolver.
If you read the documentation, you'll sometimes find your needs are already supported, and managing it manually is not needed in the first place. For example, NetworkManager already comes with builtin, automatic support of a stub resolver. Not only that, it supports both dnsmasq and unbound, and even has extra logic to help you updating the DNS when you're connected to a VPN. All of these features are here to make one's work easier.
Yet one still chooses to fight with the system in the most brute force way such as "chattr +i", at the very least you can turn it off smoothly and it'll never bother you again.
For example,
$ man NetworkManager.conf
> dns
> Set the DNS processing mode.
> default: NetworkManager will update /etc/resolv.conf to reflect the nameservers provided by currently active connections.
> dnsmasq: NetworkManager will run dnsmasq as a local caching nameserver, using "Conditional Forwarding" if you are connected to a VPN, and then update resolv.conf to point to the local nameserver. It is possible to pass custom options to the dnsmasq instance by adding them to files in the "/etc/NetworkManager/dnsmasq.d/" directory. Note that when multiple upstream servers are available, dnsmasq will initially contact them in parallel and then use the fastest to respond, probing again other servers after some time. This behavior can be modified passing the 'all-servers' or 'strict-order' options to dnsmasq (see the manual page for more details).
> systemd-resolved: NetworkManager will push the DNS configuration to systemd-resolved
> unbound: NetworkManager will talk to unbound and dnssec-triggerd, using "Conditional Forwarding" with DNSSEC support. /etc/resolv.conf will be managed by dnssec-trigger daemon.
> none: NetworkManager will not modify resolv.conf. This implies rc-manager unmanaged.
$ man systemd-networked
> by one of the .network files will be ignored. It is also possible to
> explicitly tell systemd-networkd to ignore a link by using
> Unmanaged=yes option, see systemd.network(5).
I guess one reason that everyone's fighting with the system is not realizing the official way of telling one's intention to the system. Perhaps everyone should learn the way GRUB warns its user.
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#
Perhaps someone should contribute a patch to NetworkManager to add a similar warning. # DO NOT EDIT THIS FILE. This file is currently under the management
# of NetworkManager. It's automatically generated according to the
# DNS configuration of the current network connection in NetworkManager.
# Any change to this file will be overwritten.
#
# If you are trying to change your DNS resolver per connection, use "nmtui", "nmcli",
# or alternatively, edit "/etc/NetworkManager/system-connections/".
#
# To stop NetworkManager from managing it altogether, edit
# /etc/NetworkManager.conf, set the following and
# restart NetworkManager, before you edit /etc/resolve.conf.
#
# [main]
# dns=none
#
# If you are trying to set up a stub DNS resolver, it's recommended to
# NetworkManager's builtin support. Set "dns=dnsmasq" or "dns=unbound"
#
# See "man 5 NetworkManager.conf" for more details.Worth noting that NetworkManager does indeed add a shorter and milder version of such a warning (at least on my machine):
# Generated by NetworkManager
search blah.blah.blah
nameserver x.x.x.x
nameserver y.y.y.y
nameserver z.z.z.zsudo apt install -d -y unbound sudo systemctl stop systemd-resolved sudo systemctl disable systemd-resolved sudo unlink /etc/resolv.conf sudo sh -c 'echo "nameserver ::1 127.0.0.1" >/etc/resolve.conf sudo apt install -y unbound
The distinction between configuration and code isn't what's important here. The purpose of these files are to describe a system wide default state. It is not to mark them as devoid of programmable logic.
The latter doesn't even make sense if you look at what else is there. /etc/profile for example is clearly code.
Yup sysmerge(8) does this on OpenBSD. Not sure if other BSDs have similar mechanisms, from the article I would guess not.
What Bell would have done in /etc is not what BSD did.
This distinction is nonsensical. They are files that you can change to suit your needs. I often comment a few lines on them. Code is a particular case of data.
It's possible I did modify the conf file, in a late-night attempt to fix some issue long ago, but that was all forgotten by now.
The changes to the conf file were extensive, and I had no idea what were _my_ changes, assuming I had indeed done any changes, vs what was just changes due to the new version.
Why couldn't my change in etc just be a one-liner? Then the OS could upgrade the base conf file safely, and simply inform me that I need to review my conf override in etc to see if it was still valid or needed.
This way the upgrade tool could potentially be even smarter as it can clearly distinguish what has been changed, so it can warn even more loudly when a configuration variable has been removed or drastically changed for example.
And that's why I also mentioned en passing that sourcing local files from a potential /usr/pkg/libexec/rc.d/ directory would be problematic.
In fact, even today, making rc.d files automatically work from installed packages is a problem precisely because of this: those files currently end up in /usr/pkg/etc/rc.d/, and /etc/rc cannot use them by default because /usr/pkg/ might be in a different partition. There have been endless and repeated discussions about this issue on the NetBSD mailing lists.
If your point is that rc should not be modified for <whatever> because of BSDs… rc is not a centralised and unified tool, how BSDs handle their initialisation process has no bearing on any other system unless those others decide to follow suit.
LSB is worth reading, because it had a precise definition of /etc that is not "important system stuff."
1. Here is the list:
https://tracker.debian.org/pkg/apper
https://tracker.debian.org/pkg/arctica-greeter
https://tracker.debian.org/pkg/arctica-greeter-guest-session
https://tracker.debian.org/pkg/at-spi2-core
https://tracker.debian.org/pkg/bacula-common
https://tracker.debian.org/pkg/dillo
https://tracker.debian.org/pkg/drkonqi
https://tracker.debian.org/pkg/eximon4
https://tracker.debian.org/pkg/fgetty
https://tracker.debian.org/pkg/flatpak
https://tracker.debian.org/pkg/flatpak-builder-tests
https://tracker.debian.org/pkg/flatpak-tests
https://tracker.debian.org/pkg/flatpak-xdg-utils
https://tracker.debian.org/pkg/flatpak-xdg-utils-tests
https://tracker.debian.org/pkg/font-viewer
https://tracker.debian.org/pkg/frameworkintegration
https://tracker.debian.org/pkg/gamemode-daemon
https://tracker.debian.org/pkg/gstreamer-qapt
https://tracker.debian.org/pkg/gtk-3-examples
https://tracker.debian.org/pkg/hotspot
https://tracker.debian.org/pkg/iwd
https://tracker.debian.org/pkg/kde-cli-tools
https://tracker.debian.org/pkg/kde-config-gtk-style
https://tracker.debian.org/pkg/kde-config-gtk-style-preview
https://tracker.debian.org/pkg/kde-config-systemd
https://tracker.debian.org/pkg/kdeconnect
https://tracker.debian.org/pkg/kde-telepathy-auth-handler
https://tracker.debian.org/pkg/kde-telepathy-call-ui
https://tracker.debian.org/pkg/kde-telepathy-filetransfer-handler
https://tracker.debian.org/pkg/kde-telepathy-text-ui
https://tracker.debian.org/pkg/khelpcenter
https://tracker.debian.org/pkg/kinit
https://tracker.debian.org/pkg/kio
https://tracker.debian.org/pkg/kwin-common
https://tracker.debian.org/pkg/lambda-align2
https://tracker.debian.org/pkg/libexecline2.5
https://tracker.debian.org/pkg/libexecline-dev
https://tracker.debian.org/pkg/libexecs0
https://tracker.debian.org/pkg/libexecs-dev
https://tracker.debian.org/pkg/libexecs-embedded0
https://tracker.debian.org/pkg/libkf5config-bin
https://tracker.debian.org/pkg/libkf5kdelibs4support5-bin
https://tracker.debian.org/pkg/libkf5purpose-bin
https://tracker.debian.org/pkg/libkf5screen-bin
https://tracker.debian.org/pkg/libkf5su-bin
https://tracker.debian.org/pkg/libkf5wayland-dev
https://tracker.debian.org/pkg/libkf5xmlgui-bin
https://tracker.debian.org/pkg/libkim-api-dev
https://tracker.debian.org/pkg/libkscreenlocker5
https://tracker.debian.org/pkg/libktpotr9
https://tracker.debian.org/pkg/libmbim-proxy
https://tracker.debian.org/pkg/libmodsecurity3
https://tracker.debian.org/pkg/libqmi-proxy
https://tracker.debian.org/pkg/mlucas
https://tracker.debian.org/pkg/ntpsec
https://tracker.debian.org/pkg/openconnect
https://tracker.debian.org/pkg/opensmtpd
https://tracker.debian.org/pkg/opensmtpd-extras
https://tracker.debian.org/pkg/os-autoinst
https://tracker.debian.org/pkg/ostree-boot
https://tracker.debian.org/pkg/ostree-tests
https://tracker.debian.org/pkg/plasma-desktop
https://tracker.debian.org/pkg/plasma-discover
https://tracker.debian.org/pkg/plasma-discover-backend-snap
https://tracker.debian.org/pkg/plasma-workspace
https://tracker.debian.org/pkg/plasma-workspace-wayland
https://tracker.debian.org/pkg/polkit-kde-agent-1
https://tracker.debian.org/pkg/powerdevil
https://tracker.debian.org/pkg/pulseaudio-module-gsettings
https://tracker.debian.org/pkg/remote-logon-service
https://tracker.debian.org/pkg/rkward
https://tracker.debian.org/pkg/seahorse
https://tracker.debian.org/pkg/sysprof
https://tracker.debian.org/pkg/tomcat9
https://tracker.debian.org/pkg/tomcat9-common
https://tracker.debian.org/pkg/xdg-dbus-proxy-tests
https://tracker.debian.org/pkg/xdg-desktop-portal
https://tracker.debian.org/pkg/xdg-desktop-portal-gtk
https://tracker.debian.org/pkg/xdg-desktop-portal-kde
https://tracker.debian.org/pkg/xdg-desktop-portal-tests
2. https://www.debian.org/doc/debian-policy/3. https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#us...
This seems like a very useful read indeed.
However, I also mentioned that the same problem applies to pre-systemd Linux systems because of their /etc/init.d/ directory. systemd has actually done the right thing here (which is also mentioned in the article) by putting unit logic under /lib/.
The problem there is that distributions, be it linux or bsd, lack a proper way to distinguish between a "static" resolv.conf and a (always-)generated one.
An "easy" fix would be to mandate that if /etc/resolv.conf contains a "dynamic" key then /run/resolve.conf (or /var/run/resolv.conf or whatver) is read as well (or instead).
However, tons of stuff tries to parse resolv.conf, so it would be better to have a user-editable /etc/resolv.conf.conf and a /run/dynamic-resolve.conf and dump the result or into /etc/resolv.conf to keep compatible.
First of all, there was a system before Mewburn rc. FreeBSD had an /etc/rc.local.d/ mechanism for drop-in scripts in the middle 1990s, added by Jordan Hubbard and Garrett Wollman.
* http://jdebp.uk./FGA/rc.local-is-history.html
Second, /etc/rc.conf is not the only service configuration file in FreeBSD's version of Mewburn rc. The primary configuration file is in fact /etc/defaults/rc.conf , and that file lists, in a variable, the name of other secondary configuration files, including /etc/rc.conf and /etc/rc.conf.local . On PC-BSD of old, and then TrueOS, it also named things like /etc/rc.conf.pcbsd for example.
% sysrc -f /etc/defaults/rc.conf rc_conf_files
rc_conf_files: /etc/rc.conf.pcbsd /etc/rc.conf /etc/rc.conf.local
%
Third, if you go back to Usenet of the 1980s, and indeed contemporary books about system administration, you'll find that the distinction between /etc/rc and /etc/rc.local was problematic and not what people tell you it is supposed to be. People then too thought that /etc/rc was for the operating system manufacturer, and /etc/rc.local was for the system administrator. But in practice operating system resellers ended up putting their stuff in /etc/rc.local making it also something that system administrators touched at their peril, lest they break operating system upgrades.Fourth, the article says "You might even want to support a separate location for user-created services". That already exists. It's /usr/local/etc/rc.d/ . And to highlight the point about resellers invading "local" stuff, note that PC-BSD and TrueOS fill it with additional PC-BSD services like pcdm and pc-samba and the problem of "reseller"-provided add-ons being "local" continues to this day.
% sysrc -f /etc/defaults/rc.conf local_startup
local_startup: /usr/local/etc/rc.d /usr/pbi/rc.d
%
Fifth, systemd unit files are not scripts. This is one of their defining features, and one of the things widely touted about them, in fact. As the systemd developers will themselves tell you, and have, they are configuration files. Hence the systemd developers thinking that it was a mistake to provide an EnvironmentFile setting. systemd isn't dividing things up into "configuration files" and "code", as this article talks about. It has multiple sets of things all of which are configuration files.* https://unix.stackexchange.com/a/419061/5132
Sixth, the systemd search order for unit files isn't that simple, and has a history of being improperly documented. It searches 17 directories nowadays, including subdirectories of /usr/local .
* http://jdebp.uk./FGA/systemd-documentation-errata.html#Missi...
For completeness, note that Debian's 2014 improvements to van Smoorenburg rc put new things like init-d-script into /lib/init/ .
* https://manpages.debian.org/sid/sysvinit-utils/init-d-script...