Linux greybeards release beta of systemd-free Debian fork
theregister.co.uk
theregister.co.uk
This has the problem that I have to vacuum or nuke the journal if I want to remove knowledge of past crashes because coredumpctl tool has no delete command. And coredumpctl hard-codes GDB as the debugger.
To fix all of this I've overwritten the systctl.d file and made it write somewhere I know and can find without the use of an unfinished tool.
I have also regularly had it happen that bootup would intermittently hang up or shutdown as well. There are also unfixed warnings and errors during shutdown on multiple machines.
My conclusion is that systemd uses the broader population for beta testing, even if you don't follow Fedora Rawhide or Arch rolling.
1. https://www.freedesktop.org/software/systemd/man/coredump.co...
If I wanted to be cynical, I'd believe anything coming out of systemd, regardless of quality, is accepted as the new way while discarding existing ways as obsolete.
I should add to my startup hangup remark that I never before had a startup problem that was intermittent with plain old stupid init from 1994 up until systemd became the distro default.
That seems to be the case for you as well. You have my sympathies, it's most frustrating when it occurs.
Recently there have been major regressions in the kernel due to DRM atomic modesetting with bug reports with many confirmations and zero action from the maintainers while not being able to go past kernel 4.1 which is fortunately one of the long-term branches. 4.2 is also okay but there's no 4.2 lts branch.
I never used Ubuntu as my workstation OS, so I cannot talk about their QA standards or how well it would have worked for me, but I've observed sloppiness in their packaging and integration work on others' systems. I mean, it's also important to remember that the advertised number of Ubuntu packages is not truthful if you're honest, because it's a snapshot of Debian's repositories and thus anything in Universe is probably unmaintained when compared to the repository in Debian. That means, the curated set of packages is quite small in comparison. I hope the situation improved in the last years.
Peace :-)
It probably doesn't help that they're both heavily developed by Red Hat, who see taking open source software and turning it into something stable and end-user viable as their secret sauce that justifies paying them lots of money.
I wouldn't blame Red Hat per se but the GNOME fall back they ship with RHEL is a joke and in no way resembles the old GNOME desktop Pixar was used to.
Also what's with the stupid versioning of systemd/udev? Having a single version number is simply a dumb idea.
I love bash, I love nitpicking about how to use bash along with the other bash nazis in #bash@freenode. But I would never put myself through sysvinit again. I thoroughly enjoy using distros like CentOS 7, Fedora 23, RHEL7. They're a joy to manage and work with, in part thanks to Systemd.
I never want to debug, troubleshoot, or even look at another sysvinit-script again. :)
I believe in lowest common denominator everytime. Binary anything is bad when alternatives exist. Linux has given itself over to binary blobs in the name of being more attractive to corporate users. Device drivers under Linux are now almost all binary blobs. You will not see this in OpenBSD, for example. The BSD folk reverse engineer the drivers and write their own free drivers -- and they are usually far and away better afterwards.
Take the wireless chipset drivers for Ralink. The BSD guys took a 30,000 line driver and rewrote the entire thing in less than 1/3 of the code. That's just the whole attitude between Linux and BSD. Linux is a series of cobbled-together hacks on top of an ever-growing, ever-complex kernel (that really needs legacy crap removed). BSD, on the other hand, is engineered as a single OS with a small collection of pruned base programs. Big difference. If you've ever used BSD, you will know the joy of /etc/rc.conf vs the nightmare of Linux config files that live in myriad locations depending on the distro.
https://fedoraproject.org/wiki/How_to_debug_Systemd_problems
It's curious that he says "Binary anything is bad when alternatives exist. Linux has given itself over to binary blobs in the name of being more attractive to corporate users. Device drivers under Linux are now almost all binary blobs."
That's not exactly true now, is it? He seems to be implying that there are closed source binaries in the Linux kernel...
The only true competitor to systemd is s6, and you should see the amount of "binary" he has had to develop to get his service manager to run outside of PID 1!
However... it's all a moot point, because that isn't the definition of a "binary blob". In the context of Linux, or "free and open-source software, a binary blob is a closed-source binary-only piece of software without publicly available source code." [1]
Those commands will not help you if you have to find a bug into systemd code to find something like a constant on the maximum size of a core dump and work around it. Then open a bug and hope it does not get ignored for ~3 years. But who needs core dumps anyway, right?
There's complaints to be made about systemd, but I don't find this complaint compelling, at all. I think moving from procedural programs that are sometimes hundreds of lines long to declarative config files that are never more than a couple dozen is an overwhelmingly positive thing.
On Debian they're specifically not bash scripts, they're posix shell scripts and /bin/sh is provided by dash which is much lighter than bash.
I like the declarative nature of service files I'm just not a fan of the hoops it makes us jump through to achieve the same results (see my other comment about changing daemon args)
It's not like new services and startup programs aren't constantly being introduced - so whilst the old scripts were pretty steady, new ones weren't always foolproof.
I'm not adverse to SysV and rc scripts, but they were never the panacea that some grey-beards make out. Dependency orders and parallelisation are particularly difficult, but it could be argued the scripts give control during the boot process.
I think in practice, it's fairly rate that the scripts were really needed - certainly systemd's more declarative nature for a problem that naturally lends itself to a dependency structure that has a directed acyclic graph seems like it's quite an effective an elegant approach!
Aside from the reasonable daemon-arguments issue you raise in the other comment (I just read this, interesting observation!) I really feel that systemd largely eliminates huge quantities of shell scripting in the startup process!
I've been wondering what scenarios there are that would require scripting. I thought that perhaps a more complex setup might be that a service needs to wait on two or three processes before starting, in which case a child can have multiple parents? Can systemd handle that?
I'm still very curious as to what binary blobs you are referring to though.
You mean all those init systems you relied upon for the decades before systemd...or did you just start using linux recently?
I rarely make use of the BSD scripts provided but I like them as simple examples of how various services are intended to be run.
I find there is a certain amount of uniformity and consistency and arguably coherence to the rc.d system.
But that could easily be explained by the average complexity of services increasing along with my career path into Linux and Devops.
I don't remember any clear differences between sysvinit and rc, the scripting is still left up to the user which means code style can diverge from whatever the maintainers had in mind.
Systemd, albeit a binary blob, at least eliminates that from the equation and breaks down services into a limited number of types that it handles equally every time.
I actually liked upstart too. :) I might sound mad for saying that but as a user of Linux upstart gave me just what I wanted.
Bottom line is that I think discussions about scripting style have no place in the context of service management. Services should be dealt with in a simple way like upstart or systemd. If you want to add more complexity you can write a start script for the service but that should not be the norm.
This is commonly provided by the port maintainer - I haven't conducted a survey but I'm pretty confident that service scripts in FreeBSD ports rarely make it into rc.d without a deliberate freebsdization.
van Smoorenburg rc and Mewburn rc are noticably different.
* https://news.ycombinator.com/item?id=11550802
They are both rc, by the way. Don't conflate init and rc.
> If you want to add more complexity you can write a start script for the service but that should not be the norm.
In some fields, it appears that it is.
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/sy...
The biggest problem with java is knowing when the service is started. And that's actually a problem with other clustered services like wsrep in mariadb for example. You need to get an OK from the service, by checking logs or APIs, before you know that the restart worked.
So services like that will always require an added layer of abstraction. But that's not the norm, far from it.
Ironically, MariaDB is not one of those fields, and your chosen example is actually a counterexample. The push in MariaDB is to use systemd's "notify" readiness notification protocol and actually get rid of the wrapper shell scripts completely.
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/un...
* https://mariadb.atlassian.net/browse/MDEV-5713
* https://github.com/MariaDB/server/pull/26
* https://github.com/MariaDB/server/pull/83
* https://github.com/MariaDB/server/commit/20c2ae39db3dd0ec4c3...
As in MySQL and PostgreSQL.
* http://bugs.mysql.com/bug.php?id=65809
* http://www.postgresql.org/message-id/E1aQnE5-0003Cj-CN@gemul...
* http://postgresql.nabble.com/PATCH-better-systemd-integratio...
If you mean to figure out what was actually executed for a unit "systemctl cat [service name]" will tell you the full path of each file systemd processed for that service and what content was pulled from it.
OpenRC fixes a lot of that, without adding the giant pile of dependencies that systemd imposes. As someone who is already comfortable with bash, OpenRC scripts should be easy.
> I can't believe anyone would like to keep using sysvinit-scripts or RC-scripts.
There are more options than sticking with Debian's ancient scripts.
The larger problem is that init is only a tiny part of systemd. Systemd's mandatory link-time dependencies and tightly-coupled (monolithic) features are trying to be an entire wrapper around important parts of the kernel. Even if you like the init features of systemd, you should be looking at all the changes; do you really want to depend on all of systemd's features in the future? Do you enjoy depending on a moving target that changes at the whim of the systemd cabal and their reputation for WONTFIXing anything that they don't understand or haven't considered as a use case?
What parts of the kernel are you referring to? Genuinely interested...
That's because you can't reasonably fix the many problems with sysvinit (or OpenRC) without also providing a proper process manager, and dependency system that does far more than just trigger scripts in the right order.
The biggest problem with init is that it's just init.
> Do you enjoy depending on a moving target that changes at the whim of the systemd cabal and their reputation for WONTFIXing anything that they don't understand or haven't considered as a use case?
I don't enjoy that, but the alternative of relying on sysvinit or OpenRC is so many times worse.
Give me a better alternative, and I'll throw Systemd out of the window immediately. But to be better it will need to at a minimum:
- Be able to sanely handle service restarts, including configurable behaviour on failure.
- Be able to ensure service output is sanely logged.
- Be able to reliably keep track of processes belonging to a service even if they double fork or the prent process dies.
- Have a dependency system that allows starting and stopping services automatically depending on the status of other services, the availability of a directory or file.
- Be able to schedule execution (like cron) but still subject them to the process management and dependency rules.
.. to mention some.
Why does init have to service supervision? What's wrong with having init just launch the service supervision process and afterwards just reap dead processes (which allows init to be a very simple program indeed [1]).
As for the requirements to the init+rc system.. mine look a bit different: it should be reliable and predictable first.. performance not so much.
OpenRC works pretty well for me, the only thing I would change right now is to replace that start-stop-daemon rubbish with runit (and there is work under way to allow that if I am not mistaken).
Now you will probably say that anything still using start-stop-daemon is strictly worse than whatever systemd does, because it won't restart the daemon at all if it crashes (and is ugly in some other ways too). I would agree with that in theory, however in my case this happens rarely enough that it is the smaller evil versus the systemd system sometimes not booting properly at all.
And to them systemd is mana from heaven, as it allows them to bring up and shut down any number of nodes in their cattle farm swiftly.
Where before admins fretted about making sure a server stayed up when booted (and if it went down, make damn sure to figure out why so it didn't happen again), devops seems to adopt what i consider uptime by machinegun.
That server won't stay up for some reason? No problem. Lets just throw thousands more just like it at the task, like a machinegun throws bullets at a target.
daemontools
A series of scripts that run the supervise program. It initially runs svscanboot [1], which enables logging and uses svscan [2] to look through a service directory in order to execute the supervise executable for each service. [3] Supervise loads the run command, and unlike runit (which was inspired by daemontools) when run ends, it waits a second and executes run again. As with runit, if it can read a down file, then it does not start the service. [3]
To control the service, you run svc [command] services, which writes to a the named pipe control. This pipe is being managed by supervise, which uses it to send a signal to the process depending on the control value it reads. [4]
When this uses System V init, you must add svscanboot to the inittab as a respawn process.
---
runit
Executes /etc/runit/2 and if this fails, restarts it. /etc/runit/2 generally runs runsvdir on a particular directory (normally /etc/service) [5] and once every 5 seconds checks if the time of last modification, the inode, or the device has changed, in which case it starts a new runsv process on the new directory.
runsv switches to the directory of the service and if the file down doesn't exist it executes the run script. Once the script exits, if there is a finish script it executes this. If there is no finish script, then runsv executes the run script again. [6]
To handle abnormal failure, you must write a script that handles it. When the run command exits, runsv passes the exit code (-1 if abnormal exit) and least significant byte of the exit status as determined by waitpid - useful if a signal is sent. [6]
You start and stop a service by using the sv program. The mechanism used in to create a pipe named control in the service directory, which runsv reads and when a control character is written to the pipe it first checks to see if there is a custom command it needs to run, if so then it runs it. runsv will send a signal to the process it is monitoring depending on what control character is written to the pipe, or unless the custom command returns 0 in which case no signal is sent. [6]
runit is meant to replace init, and run under PID 1. [7] However, it can still use it with the existing init on your system. [8] You can configure it to replace the logging facility via the <service>/log/run script - runit just redirects the services standard output to it as standard input and you just send that input to the logger.
---
OpenRC
Not a replacement for init, but instead is invoked by init. /sbin/openrc reads /etc/runlevels, builds a dependency graph and then starts the run scripts for each of the services in the correct order. Then executable ends. [9]
---
s6
Basically inspired by runit, except the guy who makes it feels that dependency management is important and should be run by PID1. [14]
To do this, he has created a minimal shell language called execline [10] which he has run as the first process by his program execlineb [11], creates a tmpfs filesystem and copies a base image prepared by s6-linux-init into it [12], has the process reload its environment variables then forks and blocks a child process. Next, it executes as PID 1 into the service dependency tool s6-svscan, which scans a service directory and loads some initial minimal services (the logger and an optional getty, etc).
Once this is done the child process unblocks and executes s6-rc-init, which requires a compiled service database and a services directory for it to scan. After this is done, the system will be in a fully operational state.
To shutdown, PID 1 is sent a s6-svscanctl command and a shutdown script is issued.
---
In Summary
Here's the thing I see. With the exception of s6, which makes sense but seems overly complicated to me, all of these init systems have issues. daemontools polls to see if services need to be loaded, and has no dependency order. Runit has the same issue, but as pointed out by the s6 creator, if the runsv process dies then the pipe between the logger and the process means that you can lose the log. [15] s6 also does dependency by forking early, then blocking that child process till the system is usable enough for normal services to start correctly. To do this requires considerable effort, which may not be a big problem but I don't know if it's a big enough concern to be concerned about.
Runit and daemontools run shell scripts, but not in the system V way - they have run scripts you can customize. s6 has something similar, but allows for dependency tracking through listening and waiting for the status of services. You can combine Runit with OpenRC.
systemd uses socket activation for it's dependency tracking. The biggest criticism of it is that it runs in PID 1, and if it crashes it will take out your system. Whilst there are ways around this (as shown by the s6 system), I think in reality it's unlikely to occur. YMMV. But systemd provides unit files and a bunch of configurability, and I personally think it makes everyone's life easier.
The ancilliary utilities are not running in PID 1 though. And as with other systems, you use a utility to start and stop services - which some object to, but when you objectively look at it I really think you'd be hard pressed to be terribly worried about it.
So systemd does do a lot, but I think most of the arguments I've seen so far don't really show how it is a failed system in any way.
1. http://cr.yp.to/daemontools/svscanboot.html
2. http://cr.yp.to/daemontools/svscan.html
3. http://cr.yp.to/daemontools/supervise.html
4. http://cr.yp.to/daemontools/svc.html
5. http://smarden.org/runit/debian/2
6. http://smarden.org/runit/runsv.8.html
7. http://smarden.org/runit/replaceinit.html
8. http://smarden.org/runit/useinit.html
9. https://github.com/OpenRC/openrc/blob/master/guide.md
10. http://skarnet.org/software/execline/
11. http://skarnet.org/software/execline/execlineb.html
12. http://skarnet.org/software/s6-linux-init/s6-linux-init-make...
13. http://skarnet.org/software/s6-rc/s6-rc-init.html
Not just that though. Otherwise it would have no dependency tracking at all for all those daemons that don't support socket activation (let alone stuff that has to be started in a certain order and doesn't communicate via sockets).
In regards to runit: it does have a mechanism for handling dependencies - starting the other service in the start script (not great imho, but works).
Also I don't quite see/understand your argument (edit: say why I should switch away from it) against OpenRC (or OpenRC+runit) - since it does do dependency tracking and has sane service scripts.
And regards to systemd making everyones life easier: not mine sadly - had plenty of problems I never had with other init+rc systems (OpenRC, the OpenBSD one, the old Arch one, etc.), while gaining nothing I didn't already have previously (hell.. doesn't even boot faster - on average it boots & shuts down slower than before - median about the same).
> And as with other systems, you use a utility to start and stop services - which some object to, but when you objectively look at it I really think you'd be hard pressed to be terribly worried about it.
You mean that you start/stop services with something like systemctl start foo? Yeah, I don't see why one would have a problem with that.
> The ancilliary utilities are not running in PID 1 though.
I am not a fan of integrating everything under the kitchen sink (network management, dhcp, NTP, udev, cron, logging, etc.) into one big project though.
It's possible to run sysvinit sanely, by having a few startup scripts that source in the appropriate boilerplate. Slackware's startup for instance has always been manageable.
Systemd, like a lot of things coming out of redhat these days (firewalld), feels like bloatware to me.
Systemd, Firewalld, D-Bus, etc.. have no place in a server IMHO.
The troubleshooting is ridiculous, too, because of the ridiculousness that comes with journald. Why does it feel the need to make me run an awfully formatted, and potentially complex set of parameters to print that? Especially considering that 9/10 times, the error message from journald is vague, ungreppable without -A or -B, and is often significantly shorter than the "use journald to print the error" output. So frustrating. I don't know how people deal with it.
That said, I'm not a fan of sysvinit, either, but I don't have to deal with as much smoke and mirror behavior with it.
People seem to ignore the kernel for some reason, but it is literally one giant program to rule them all, and it works very well. There have been many efforts to modularize it and move most of the complexity into independent units, and none have particularly worked out.
In contrast, systemd is conceptually complex -- its scope continues to expand to manage every little nook and cranny and configurable tidbit in the system. (I'm not arguing one way or another, just pointing out that the type of complexity is different.)
https://www.digitalocean.com/community/tutorials/understandi...
Aside from that, there are a number of services and utilities, but I'm not sure how they are conceptually harder to understand that any other similar program or utility on Linux...
Don't get me wrong -- I actually like some of the new thinking, appreciate parallelized bootups, and think service units are cleaner than the old /etc/rc.d mudball. I'd rather have things Just Work(tm) than play with wpa_supplicant and hibernate configurations on my laptop. But it's been a lot of new stuff to learn for someone used to the Old Ways, and it's kind of astonishing how fast systemd pieces have taken over different domains.
Oh boy, I've just realised - I'm growing older! Eep.
In kernel space, the global lock is gone, there is a central API that everybody else uses with code that mostly don't touch each other.
Linux is very modular. It is just not a microkernel, and that is probably more due to historical performance bottlenecks than to architectural concerns.
Linux is pretty modular as an engineering project, in that there are ways for developers of one part of the code not to need to care about all the rest of it. But in the sense that systemd gets criticized for not being modular, Linux is also not modular. It is technically possible to implement either systemd's or Linux's APIs faithfully enough for things written to those APIs to work with your alternative implementation. systemd gets criticized because doing this is difficult in practice; it's far more difficult with Linux.
I would be fascinated if Hurd could get the same sort of developer momentum to be comparable to Linux. A completely new (as in 1.0 released) and usable FOSS Unix-alike would be quite welcome amongst Linux and the BSD's I think.
Linux was initially written and released by one guy.
Linux obviously gained a wider team while Hurd languished, but it's still interesting that GNU's kernel efforts started with far more initial human resources and were still quickly overshadowed by Linux.
Also, the userspace of GNU/Linux isn't called Linux. Linux isn't an operating system.
The established Unix philosophy of everything being "one program that does one thing" would be hugely beneficial if applied to an OS kernel. Wouldn't it be great you could just run different drivers and file system in userspace rather than having to compile a big kernel?
Not to mention that it could be much easier to develop them if they could just run as processes and die if they failed, as opposed to inducing a kernel panic.
Except that it makes debugging almost impossible and the overhead of IPC is very large. I'd argue that most popular kernels today (k{Free,Open,Net}BSD, illumos, Linux) are monolithic because that model makes development much simpler than a microkernel (not for historical reasons). Hybrid kernels like NT are a compromise for efficiency against the benefits of having a microkernel.
The panic argument is a good point, until you realise that kernel drivers don't panic --they oops. And a kernel oops just kills the thread it was running in. Linux has kernel threads that can be killed, but they're all playing in the same address space. So you can think of that as having the advantages of a microkernel without the debugging issues.
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/mi...
Last I checked, IPC overhead was not a barrier to performance in modern microkernels. QNX and XNU (used by OS X) both use microkernel-type IPC and are fast. XNU is even able to survive with an old Mach kernel.
If a kernel driver ends up in a faulty state (accesses invalid memory, does the wrong interrupt thing) then surely the whole kernel will panic?
NT is frequently referred to as "hybrid", but it's a frequently misunderstood design. First, while it uses IPC internally, it's not really comparable to microkernels since there's no context switch; every participant in the IPC communication is in the kernel. This is done purely as a way to isolate internals behind a rigorous, message-oriented API. Secondly, almost all of NT's services run in the kernel. A fairly small number of services run in userland: There's the user-mode driver system (UMDF, available since Vista), and graphics drivers also have the option of running partly in usermode.
"[systemd] provides a system and service manager that runs as PID 1 and starts the rest of the system. systemd provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux control groups, supports snapshotting and restoring of the system state, maintains mount and automount points and implements an elaborate transactional dependency-based service control logic." [1]
It comes with a set of utilities that do a lot of things. But that doesn't make it monolithic. In fact, systemd is actually extremely modular.
Ironically, systemd has not gotten any harder to use since it was introduced, at least IMHO, but I guess as with anything YMMV.
But this myth about systemd being monolithic, it really needs to stop because it's not correct.
> It comes with a set of utilities that do a lot of things
"monolithic" vs "modular" has isn't about the number of binaries you build. This is about coupling. Replacing systemd itself is the modularity we're talking about, which becomes harder to replace (less modular) every time systemd annexes another set of features.
Do those utilities use only well-defined, simple interfaces both to the system and between utilities? Can I replace one or more of those utilities with something else that implements those simple, stable, and well-defined interfaces? This what we mean by modularity; each piece of the complex system can be considered in isolation, with only minimal dependencies.
When I can remove systemd-PID-1, OR keep using systemd-PID-1 but replace those utilities with other tools, only then will systemd be a "modular" set of tools. If I have to recompile daemons to remove systemd dependencies (libsystemd-journal0, libsystemd-login0), those daemons are now part of the systemd monolith.
> needs to stop
The only things that should stop are these attempts at changing word definitions.
Do those utilities use only well-defined, simple interfaces both to the system and between utilities? Can I replace one or more of those utilities with something else that implements those simple, stable, and well-defined interfaces? This what we mean by modularity; each piece of the complex system can be considered in isolation, with only minimal dependencies.
Um, yes - you can.
When I can remove systemd-PID-1, OR keep using systemd-PID-1 but replace those utilities with other tools, only then will systemd be a "modular" set of tools. If I have to recompile daemons to remove systemd dependencies (libsystemd-journal0, libsystemd-login0), those daemons are now part of the systemd monolith.
There is no factual basis to this claim. There is only really one component that can't be easily replaced, and that's journald.
I challenge you to tell me which component of systemd other than journald can't be swapped out if you so desired.
Systemd's own documentation[1] disagrees. Several important interfaces - which are used by other parts of systemd- are marked as not being independently reimplementable. Another way to say that is that systemd considers those to be tightly coupled.
> There is only really one component that can't be easily replaced, and that's journald.
You're contradicting yourself. If I remove all of systemd, existing daemons will need to be recompiled because libsystemd-journald.so is missing. You don't just to pretend that isn't a dependency.
The common response that we should just keep the library file is an attempt to pretend this problem does not exist. Klaus Knopper even gave a talk recently[1] about this very problem, which is causing problems (some of which have workarounds) for the next version of Knoppix.
[1] https://www.freedesktop.org/wiki/Software/systemd/InterfaceP...
Interestingly, most of it is reimplementable independently elsewhere. But I think it's instructive to quote why they say what they say:
"A number of systemd's APIs expose Linux or systemd-specific features that cannot sensibly be implemented elsewhere. Please consult the table below for information about which ones these are.
Note that not all of these interfaces are our invention (but most), we just adopted them in systemd to make them more prominently implemented. For example, we adopted many Debian facilities in systemd to push it into the other distributions as well."
So yes, there are some interfaces that are tightly coupled to Linux interfaces, and there are a few things like unit configurations, generators, system updates and presets that are very specific to systemd and, like they say, don't make any sense being implemented outside of systemd!
> You're contradicting yourself. If I remove all of systemd, existing daemons will need to be recompiled because libsystemd-journald.so is missing. You don't just to pretend that isn't a dependency.
I don't believe I have contradicted myself. That component is a sticking point, and is a problematic dependency. I'm curious what the devuan guys did to sort this out. I assume they recompiled everything.
Edit: I'm watching the talk by Klaus, thanks for the link - very interesting! I think I'm a bit confused about his point about the kernel taking an extra 15 seconds to load... does that have anything to do with systemd? I would have thought that's something else.
I'm also not following his point about systemctl being a "GUI"... that's a command line utility, not a GUI based tool?
OK, he's not happy about journald's binary format. I get it, I'm not keen on it either. But I guess he didn't realise that in journald.conf you just change storage to 1 and ForwardToSyslog to 1 also...
He's concerned about parallelization on slower hardware. Fair enough... in that case, setup the unit files to set the order?
OK, and now he's concerned about Ubuntu's packaging... ?? I'm not entirely sure how that is a problem for systemd... and seems to be concerned about a sysvinit package that's not a standard part of Ubuntu any more? That seems a bit of a strawman...
Yup, that's what I thought! It takes 15 seconds when he replaces systemd - so not an issue with systemd in terms of bootup speed.
And now he's saying he doesn't like the fact that his system now requires multi-user session management. What? Oh come on, that's silly. He also doesn't know why the error is occuring...
However, his issues about the difficulties of swapping over packages - well, he has his own distro and he's actually worked out after a bit how to replace it with System V. Linking libraries is not very good, but that's surely more of an issue with developers making bad dependency choices? or perhaps they find it convenient. And he worked out how to get around it with a shim anyway, which obviously increases his workload, but then again he's the one who wants to diverge from what all the other distros are doing!
Of course, there is the other point in that it could be argued that the SystemV system doesn't have features that are provided by systemd... which would tend to indicate that perhaps it's lacking in a few areas.
He also belabours the point of parallelisation. I cannot understand why he just doesn't change the before or after directives in the unit config files. That's really not much different to the pain of reordering services in the existing System V-based systems!
So I'm really not convinced :-) Anyway, that was rather a fun talk, I liked how he was troubleshooting the issue, more power to him that he got his system in the way he likes it.
It also takes over virtual terminals, syslog services, login management, network interface configuration, dhcp, system time, udev device management, containers (machined, dbus, cgroups).
That, to me, is where more of the concern lies.
Which tools did you have in mind?
* http://blog.darknedgy.net/technology/2015/09/05/0/
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/da...
2. Speaking of kernel code, have you looked at how much stuff is going on in the kernel? Why does pid 1 get this much scrutiny, but the kernel, which definitely can't die, gets a free pass? On my system there are 77 kernel threads. Seventy-seven! What are they all doing??
And there are plenty of alternative kernels you can use. Multiple ones even have Linux binary compatibility.
3. Is the argument for suckless init the same as that for EWONTFIX init (on which it is based)? If so, would a fork of systemd that split it into two processes, PID 1 that reaps reparented zombies and tells PID 2 about it (over a one-way pipe, so PID 1 has no increased attack surface), be an acceptable option?
ps -eLf | grep '\['
ps -eLf | grep '\[' | grep root
The point I'm making is not so much that I'm curious (although I am), but that the kernel is exempt from the scrutiny that systemd gets, and the kernel has a ton of fragile crap running at full privilege. If systemd even shipped three XFS-specific binaries, people would be upset, but the kernel can run three persistent kernel threads and nobody cares.
I just tested this out:
chris@libreoffice-ia64:~$ ps -eLf | grep '\[' | grep xfs
chris@libreoffice-ia64:~$ sudo modprobe xfs
[sudo] password for chris:
chris@libreoffice-ia64:~$ ps -eLf | grep '\[' | grep xfs
root 6731 2 6731 0 1 16:53 ? 00:00:00 [xfsalloc]
root 6736 2 6736 0 1 16:53 ? 00:00:00 [xfs_mru_cache]
chris@libreoffice-ia64:~$
chris@libreoffice-ia64:~$ sudo rmmod xfs
chris@libreoffice-ia64:~$ ps -eLf | grep '\[' | grep xfs
chris@libreoffice-ia64:~$My point isn't so much what my three kernel threads are doing as why nobody asks these questions, in general, about the kernel. Just about every criticism of systemd applies double to the kernel, but nobody is forking Debian to replace the kernel. (And very few people are contributing to the existing efforts already in Debian to officially support alternative kernels.)
That is, actually doing the work of init i.e. spawning daemons and services.
It makes no sense for such tool to control network interfaces, logging, device sync, power mode changes, and well... I'm certainly forgetting one or two dozen systemd modules.
As for those other tools... well, the systemd init process doesn't manage those either. The anciliary modules do, and quite well too really. I'm not entirely sure why people think that this is much different to running a SysV script to execute start a daemon!
These guys who wanted systemd muscled their way in to get this software into the Debian system. They also adopted an "all or nothing" approach, depriving the greater community of something bigger than "do 1 thing and do it well", which is "freedom to do as you wish and choose".
Debian wouldn't have split had they maintained they made systemd opt-in or Poettering and co. maintained their own Debian-systemd version. That didn't gel well with Redhats version of what Open Source is, and so many were deprived of their freedom to NOT CHOOSE systemd BUT still use Debian.
For all the talk of how good/bad systemd is, the proponents for systemd should remember how they deprived others with the freedom to choose, no matter how "ignorant" or "backwards" the detractors are (or how "shit" sysvinit is).
Downloading via torrent now, it's 10G via torrent? Anyways it seems the file server there is under pressure now.
I for one don't want to deal with systemd from the start. Never liked its intrusive, all-in-one, hard to debug design approach.
For example, I just had an interesting discussion on GH with Varnish packers about changing a daemon arg (in that case, the bind address/port).
The way systemd "recommends" to change the arguments passed to a daemon is to write a drop-in file (i.e. /etc/systemd/system/service-name.service.d/something.conf) that overwrites the ExecStart line. To do so, you have to do:
[Service]
ExecStart=
ExecStart=<original ExecStart line with wanted changes>
But as you said - those issues, are not enough to give up on the benefits of Debian stable. In theory, all packages in Debian stable that need an init script are supposed to still supply a sysvinit script (I was actually quite surprised by how many packages rely on systemd's ability to load sysvinit scripts and don't ship their own systemd service file) so you could choose to use sysvinit instead of systemd, for now at least.
I do appreciate the theoretical goal - you describe how your daemon should be started, and let pid1 handle the specifics of actually starting it - to me that makes it much easier for a lot more people to create reliable daemon packages.
I just wish the project wasn't taking such a "everything plus the kitchen sink" approach.
The Environment= thing I haven't seen any official comment about its recommended-ness but it has some quirks in syntax so it needs pretty rigorous testing before being used for anything that might have values that need to be quoted/escaped.
However I still dislike its monolithic nature, its folding in of journaling and udev, and all the rest of the creep.
I understand people that dislike monoliths, but I also appreciate the DRY nature of systemd - the System V, bash, cron, at, various process monitors, syslog combo was a lot of code with a lot of overlap. I'd expect systemd is actually smaller in LoC that what it replaced.
There's such a long history of projects starting out of a desire to not change, and so many of them (most, I would wager) fizzle out soon after. Being against some specific feature or architectural change is not the same as having a future vision that can drive a person to work on something, for no pay, for years or decades. Anger fades...the people you're angry at ignore the new project, making it unsatisfying in terms of vengeance. And, again, it just doesn't stick with you the way a desire to do something new and better does.
I'm not even saying systemd is great (it's good and bad in all sorts of novel ways), I'm just saying that the people who are just reacting against it, when the industry and community has been so clear about it being the future they are passionate about, are unlikely to do anything more than spin their wheels for a few months, and then fade away (as a project, I hope they all stick around as contributors to OSS things).
What you observe is emblematic of the problem with Devuan, and a few of the Devuan people themselves acknowledge it. So much of the discussion centres upon "how to remove evil libsystemd0" and suchlike that what is lost is the supposedly intentional focus of the Devuan project on init freedom. That surely means the freedom to run systemd as well as to not run it.
* https://lists.dyne.org/lurker/message/20150208.110824.579d17...
And as you can see, that bleeds over into the press coverage seeking headline material.
Does anyone know if Devuan have given it any consideration?
This is a very good news. Indeed it would have better if the Debian had been a little bit more flexible and sensible on this issue.
Congrats Devuan folks! My heartfelt thanks too.
Edit: added my thanks and clarification for desktop-user.
edit: added clarification for end-user.
why?
* http://askubuntu.com/a/626858/43344
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/sy...
* http://askubuntu.com/a/613814/43344
And read the doco that comes with your operating system (presumed to be Ubuntu, given that you say that you use upstart).
IMHO it is always good to doubt the status quo..but to discredit the winning fight put up by systemd against Shuttleworth and Upstart is very biased. The technical tradeoffs in systemd may not be worth it to you... But it won on technical merit.
FWIW I can't wait to see the next Holy War around snap packages.
I go for popular distros because it's more likely that bugs are found and community help is available.
I also do very little customization. If the default window system is Gnome, that's what I use. If it's KDE or something else, that's fine too. Again, they are all good enough.
I use Linux, I'm a huge fan of it, but I'm not dogmatic. Using Linux is a way for me to get work done, but it isn't the goal itself.
And no, I don't really expect Gentoo to be something most people use. But to me that's the beauty of open source. If someone, or a group of people care enough, they can make it happen, if someone else finds it useful too doesn't necessarily matter.
Problem is systemd helps you booting fast and this is important both in the cloud (where every minute is a billed), and in telecom (where you have a stringent SLA to cover).
I run sever operating systems because I like to have the control, the minimalism, the elegance, the security. By trying to be more of a desktop orientated system, Linux actually looses what attracts people like myself to using it. I doubt they care, I don't think they have to, just as I don't have to use it.
"This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."[1]
"Much of the power of the UNIX operating system comes from a style of program design that makes programs easy to use and, more important, easy to combine with other programs. This style has been called the use of software tools , and depends more on how the programs fit into the programming environment - how they can be used with other programs - than on how they are designed internally. But as the system has become commercially successful and has spread widely, this style has often been compromised, to the detriment of all users. Old programs have become encrusted with dubious features. Newer programs are not always written with attention to proper separation of function and design for interconnection. [...]
This style was based on the use of tools : using programs separately or in combination to get a job done, rather than doing it by hand, by monolithic self-sufficient subsystems, or by special-purpose, one-time programs. [...]
One thing that UNIX does not need is more features. It is successful in part because it has a small number of good ideas that work well together. Merely adding features does not make it easier for users to do things - it just makes the manual thicker. The right solution in the right place is always more effective than haphazard hacking."[2]
Still reading at this point? Great systemd free Linux distributions: Gentoo, Funtoo, Void Linux, among many others and there is also something called BSD (FreeBSD, OpenBSD, DragonflyBSD)... you should check it out. Also see www.suckless.org
[1] http://www.amazon.com/Quarter-Century-UNIX-Peter-Salus/dp/02...
And if you look at the things you care about when running a daemon process -- sessions, controlling terminals, reliable restarts (without killing the wrong process), logging, starting dependencies in the right order, dropping privileges reliably etc. -- none of those things are good ideas that work well together. You can clearly see that things like double-forking and detaching from controlling terminals and syslog were all bolted on later, because the original system didn't work well, and the hacks don't work well either.
I run servers, and I'm excited about systemd because it makes all these things actually work. The Unix philosophy gave me tools that do one thing and kinda sorta do them okay, if you're lucky. systemd does multiple things, and it does them all well. Back when we ran FreeBSD, we had hard-coded sleeps to wait for the network and NFS to come up before we continued with boot, and we had regular problems with being too aggressive with those sorts of hacks and having machines that would get stuck before launching sshd. That is straight-up not a problem with systemd.
It's also documented far more poorly than just about anything in systemd.
Could you clarify?
I think people initially were pretty stoked for the revamped init process, and then it started heaping on a lot more stuff. Poettering's communication style and other history didn't help either.
So what in particular, other than Lennart's supposed abrasive nature, do you have an issue with? I'm genuinely not following some of this reasoning, at least from a purely technical level.
Incidentally, really need to speak to some of the BSD guys about what they think of SysV-style init. Now that could prove to be an interesting flame war :-)
We give you this thing that replaces half the system with utilities (re)written by us, but hey don't complain you're free use something else!
A lot of those components have interdependencies (so they can't all be replaced piecemeal), and some userspace things like Gnome have come to have dependencies on things provided by parts of Systemd.
This has benefits: Things tied tighter together know more about the states of other parts, and that enables various optimizations. This has issues: Replacing individual components (if you don't like their behavior) becomes more difficult. Whether it's worth it or not seems like a philosophical debate.
In terms of Gnome - well, Gnome is Gnome. I'm not going to start defending their decisions, I disagree with quite a few of them myself.
Is udevd used by anything other than the kernel though? If not, then I don't think it's a huge issue if those who want to use a different init system maintain their own version!
Have a look at Devuan, I think you'll find that they use the same udev as Ubuntu and Debian.
It's not hostile if someone forks a project to meet their requirements but the old software is used by everyone else!
If I'm not wrong, then perhaps you should reconsider your actions, because I'll still say what I believe even though you may not like it, karma be damned!
Yes, in amongst all of this systemd discussion, please do look at the notional actual topic of this page. And discover vdev. It's explicitly mentioned in the release notes.
* https://github.com/jcnelson/vdev
* http://judecnelson.blogspot.co.uk/2015/01/introducing-vdev.h...
* http://www.linuxquestions.org/questions/debian-26/devuan-417...
> Unless the systemd-haters prepare another kdbus userspace until then this will effectively also mean that we will not support non-systemd systems with udev anymore starting at that point. Gentoo folks, this is your wakeup call.
https://lists.freedesktop.org/archives/systemd-devel/2014-Ma...
Basically, the only people who've managed to get away with not using udev are those with their own kernel fork and a completely customised stack on top. I'm not sure if any of them even use glibc or the GNU userland either.
Never mind that udev existed as a independent project for nearly a decade before being folded into systemd, thus leaving various distros high and dry. Either adopt systemd, or develop an alternative to udev (that still have to rely on the badly documented and changing kernel API).
http://www.landley.net/notes-2015.html#05-07-2015
At this point in time it seems that GregKH has mastered the art of passive-aggressiveness...
there is a reason why Lennart is good in stirring up controversy, and it's not because everyone is just against him for no reason.
some further reading/links that might interest you:
https://lkml.org/lkml/2014/8/12/459
http://without-systemd.org/wiki/index.php/Arguments_against_...
http://www.infoworld.com/article/2608798/data-center/systemd...
http://www.infoworld.com/article/2608864/data-center/choose-...
http://www.infoworld.com/article/2608870/linux/you-have-your...
http://blog.darknedgy.net/technology/2015/10/11/0//
http://suckless.org/sucks/systemd
edit: systemd might be perfect for you or other people, I don't doubt that, it might make things easier for some, in some respects in might do some things better, but if the debate (as it is now) is whether it follows the unix philosophy or not, I don't think one can reasonably make the case that it does
> edit: systemd might be perfect for you or other people, I don't doubt that, it might make things easier for some, in some respects in might do some things better, but if the debate (as it is now) is whether it follows the unix philosophy or not, I don't think one can reasonably make the case that it does
I can and have made that case :-) I guess we'll agree to disagree.
> The way systemd is structured it's next to impossible to use any parts of it outside of systemd
Well that's not true. If you want to swap out a systemd daemon with another one, then you just have to write another service configuration file - see here: https://www.freedesktop.org/software/systemd/man/systemd.ser...
> nomenclature is ridiculous (system daemon, what's that even supposed to mean?)
It's the daemon that runs the system. init, you realise, is a daemon process? As in, it's a process that runs in the background that is not under direct user control?
> it suffers greatly from feature creep
A lot of folks quite like the features it has introduced, but that's subjective
> took over logging using a binary format
journald can be configured to do text logging...
> gained basic firewall controls
It's a component of the system, which doesn't have to be used.
> assimilated udev
I just cannot understand why this is such a big deal. udev is a userspace daemon that interacts with the kernel, it has had zero negative effect on anyone and it makes total sense for the ones maintaining the init system to integrate it into their tree and maintain it going forward!
> incorporated a bootloader
You can swap it out. Why is this a concern?
> ... and some additional things people took/take offence with: introduced to quickly, criticism was shut out instead of being debated, promises were broken, main developers behind it are quite arrogant and often ignorant when dealing with complaints/bugs... et cetera there is a reason why Lennart is good in stirring up controversy, and it's not because everyone is just against him for no reason.
I'll stick with technical issues, not personalities.
Just reviewing your links:
https://lkml.org/lkml/2014/8/12/459
No technical reasons given as to the concern over systemd, only some personality clashes ("Kick these fucking carpetbaggers to the curb!" - now there's a convincing argument!)
---
http://without-systemd.org/wiki/index.php/Arguments_against_...
Yeah, I've read most of these, I'm really not convinced by any of the arguments
---
http://www.infoworld.com/article/2608798/data-center/systemd...
The guy rehashes the same arguments you are making, and I've already addressed each of them - I disagree!
---
http://www.infoworld.com/article/2608864/data-center/choose-...
Same author, same arguments - but with a side-dish of "what would the grey-beards think?" and concocted outrage over an apparent schism within the Linux community.
---
http://www.infoworld.com/article/2608870/linux/you-have-your...
Same author again, and this time the argument is:
You have your Windows in my Linux
Ultimately, the schism over systemd could lead to a separation of desktop and server distros, or Linux server admins moving to FreeBSD
---
http://blog.darknedgy.net/technology/2015/10/11/0//
Yeah, that article was largely prompted by myself I think. I promised vezzny-ford I'd read this, and I never got around to it. I'll eventually do so :-)
---
http://suckless.org/sucks/systemd
I'll have a read - looks interesting.