I love not having to track down pid files. I love not having to check if the pid file contents match a valid instance of the expected binary.
The RC system was simple and also full of exceptions and corner cases.
I love not having to track down pid files. I love not having to check if the pid file contents match a valid instance of the expected binary.
The RC system was simple and also full of exceptions and corner cases.
This is important, because this pid file is used to determine which process receives the kill signal. If you get it wrong, and have the right permissions, you can accidentally kill something that you did not intend to kill.
This is further complicated if you want to run multiple instances of Tomcat because now you need to have a unique path for this pid file per tomcat instance.
If the thing that you're trying to run doesn't fork, you then have to execute it in the background and then store the result of $! somewhere so that you know how to kill the process later on.
It's all very error prone and the process for each daemon is often different.
https://tomcat.apache.org/tomcat-8.5-doc/windows-service-how...
The commandline argument is --PidFile and --LogPath. Most, if not all, programs allow you to customise this. It should never be a guessing game, especially when you are the one creating the init file, therefore you are the one in control of the running program.
It's still a guessing game, though, even with CATALINA_PID. It is entirely possible for Java to crash (something which RC scripts do not handle, at all) and another java process starting up which happens to be assigned the same process id as the dead java process. This can not happen with systemd units because each service unit is its own Linux cgroup and it can tell which processes belong to the service.
With respect to servers that crash, approaches differ, but you could use an off the shelf, single purpose daemon respawner, or you could change the whole system, or you could endeavor to not have crashing servers, such that if it crashes, it's worth a human taking a look.
Systemd has heuristics to detect the main process to send the kill signal or detect a crash, but it can guess wrong. Telling it the pid file location makes things reliable.
Plus creation of a pid file serves as a synchronization point that systemd uses to know that the process is ready. The latter is useful when the service does not support systemd native API for state transition notifications.
And the best thing is that even if systemd has to be told about pid files, one never needs to deal with stalled files or pid reuse. Systemd automatically removes them on process shutdown.
I can't stress this enough. Pid files are really bad. The fact that we used to have to use them is a flaw in the OS design. Using them for startup notification is also a hack in and of itself. The kernel has enough information to fully track lifecycle of these processes without writing information into files that are inherently prone to race conditions, and we now have better APIs to expose this information, so we shouldn't need to use these hacks anymore. I don't think there is any Unix-like system left that considers the old forking daemon style to be a good idea, and systemd's handling of pidfiles is really just a legacy compatibility thing.
This still doesn't handle restarting crashed services, and it still is true if you need to make init scripts for your own services outside of the ports tree.
It took me a long time to come to terms with systemd, but I am very glad that I did. For me it makes defining services both easy and reliable.
I looked at one of my FreeBSD servers to see how it was handled in the case of ZNC, an IRC bouncer that I use. ZNC doesn't produce a pid file on startup, so the FreeBSD RC framework tries to find a matching process in the output of 'ps'. [1] As soon as you attempt to run multiple instances of a given service, this falls over completely. [1]
daemon(8) helps -- it handles restarting of processes if they crash, and it can manage pid files and prevent them from getting stale. Nothing on my FreeBSD system uses it. The unbound (dns cache) port uses the pidfile option for rc.subr(8). Looking at how check_pidfile is implemented, it attempts to verify that the process represented by the pidfile matches the process name. Pids also wrap on FreeBSD, so you have a chance of a false positive match if you run multiple instances of a given daemon. I could, of course, change unbound's rc scripts to use daemon, but that feels like a lot of thinking about pidfiles for something that was taken care of a decade ago.
I do like FreeBSD, don't get me wrong, and I use it in my every day life. systemd solves problems for me, and I really like the way it manages process groups by utilizing kernel features.
1. https://docs.freebsd.org/en/articles/rc-scripting/#rcng-daemon
2. https://unix.stackexchange.com/questions/503150/rc-scripts-for-multiple-zope-instances-in-freebsdOkay so you have a PID file somewhere /var/myservice/myservice.pid. The contents of that file is a number which is supposed to correspond to a process you can find in proc.
But PIDs are recycled or more likely your PID files didn't get cleaned up on reboot. So you look at your file and it says 2567, you look up 2567 and see a running process! Done right? Well it just so happenes that a random other process was assigned that PID and your service isn't actually running.
pidfd's are the real real solution to this but the short and long tail of software uses pidfiles.
systemd uses linux cgroups so that it knows exactly which pids belong to the group.
The defaults have surely changed over the years, but pid_max used to be ~32K by default. On the system I'm typing this comment on, /proc/sys/kernel/pid_max is set to 4194304.
The commit which changed the defaults is this one: https://github.com/systemd/systemd/commit/45497f4d3b21230756...
I was focusing on the "surprise startup" scenario where it would all get wiped and reset.
If it blindly reads the pid and sends a kill signal, your odds are pretty good with a limit of 32K pids on a busy system. If it confirms that the process name matches an expected value, you have less of a chance.. but if your process name is bash, java, or python, maybe not as good as you would hope.
I don't have things crashing a lot, but it's naive to pretend that it never happens. It could result in two things: the rc system telling me that everything is fine or the rc system sending signals to some unrelated, poor, unsuspecting, processes.
I don't need a process monitor just for that. I have systemd. :-)
If you're unlucky, though, pid 2567 might match another myservice instance. This can easily happen if you're running many instances of the same service. Even checking /proc/$PID/exe could give you a false positive.
There might still be race conditions but this should cut down the chance dramatically.
Your nice shell-scripts in /etc/rc.d will start EVERYTHING and ANYTHING just in, case, even if you don't always need it (systemd does support socket activation)
Your nice shell-scripts can't handle parallelism (systemd can)
Your nice shell-scripts can't reorder stuff at boot, you have to specify it by hand (systemd can via Needs/RequiredBy etc)
Your nice shell-scripts are worthless if you need to run more than one instance of a service per server (with systemd having many instances of the same service is a feature, not an after-thought)
Your nice shell-scripts won't help you troubleshoot a failing service (systemd will, via systemctl status AND journalctl).
I must say that now it has more features and the configuration format, while could be simpler, is way better than sysvrc magic comments. I'd suggest it should be XML with a simple config file equivalent for simple cases, as the subtree update in YAML-like is a hell anyway.
Different UNIX systems have gone on to replace inetd to address specific shortcomings of the inetd model with launchd, service management facility, systemd, SAM, but the service restart is not an innovation that systemd has brought along. It is a further, fine-grained, improvement over a decade old concept and a tool + ecosystem that has implemented the concept (inetd + rc scripts).
OpenBSD does this stuff right. You keep pushing faulty and buggy crap over and over.
The *BSD rc systems are robust and rely on a framework that abstracts stuff out, you're never going to be digging in there for a pid file's location.
You don't need to dig for the pid file's location, I'll give you that, but it's also possible for this robust rc subsystem to kill the wrong process under the right circumstances.
My system runs unbound (dns cache), so I looked for its pidfile. It's not in /var/run, but rather wherever it happens to be specified in the config. The default is /usr/local/etc/unbound/unbound.pid.
In mainstream Linux distributions, things in /var/run are likely sane if you live in the packaged world. My tomcat example was about adding software outside of the packaged world / ports tree, which is why you would have to track down the pid file to make a service. Given that unbound places its pid file outside of /var/run, which is more chaotic?
I like FreeBSD, and I use it daily! I also very much like the features and organization that systemd brings to the table.
1. https://cgit.freebsd.org/ports/tree/dns/unbound/files/unbound.in?id=6eb036d0d6656a72d27b34557ed4bf1feb1cd4f0A lot of the staying power of Unix is that it's based on a very small set of simple, yet powerful, concepts. RC is one of them.