What should we expect from a modern init system?
plus.google.com
plus.google.com
Containers of Debian 7 (openvz) start in seconds, with sysv init. My laptop dito.
What am I missing? Android? That takes forever to boot and I have no idea why. Could be that systemd fixes that problem.
To me, Systemd and upstart come across as modern-for-the-sake-of-being-modern.
My boring house server, AMD FX4130, boots from a small Intel SSD in forty seconds from power to 0 load. Debian stable, sysVinit.
I understand sysVinit. I can debug it. When things go wrong, I can stop it, figure it out, fix it and start it again.
Being able to figure things out and fix them is the heart of UNIX. I'm not against progress, I'm against screwing things up in the name of progress.
OpenRC looks interesting.
You may want to have a look at this short conference from linux.conf.au, about systemd from the point of view of a sysadmin: http://www.youtube.com/watch?v=-97qqUHwzGM
There are tools to look at what your laptop is doing during boot time.
I've seen the systemd and upstart propaganda, and find it unconvincing.
sysvinit is long overdue retirement - I've worked with it for many years can't wait for it to die.
It can't handle power management, it calls out to a multitude of scripts/external applications just to stop start and restart (oh, and which command options are the correct ones to support?), it can't manage processes through their lifecycle.
People constantly trot out that it's not the unix way... Linux is not unix. X is going away next, are you going to fight to keep that all the way too?
Also, why use external tools to analyse the bootup? $ systemd-analyze Startup finished in 785ms (kernel) + 395ms (initrd) + 2.184s (userspace) = 3.366s $ systemd-analyze plot > file.svg
"I don't want to learn new things. I labour under the delusion that tens of thousands of lines of shell code is a clean solution" is all I hear.
Linux systems are still too much like unix. There are still some simple components that do a single, simple task, and can be easily debugged. This is clearly not acceptable, and does not fit with the rest of a modern linux distro. It needs to do more things, be less reliable, and be harder to figure out when something has gone wrong. Systemd and upstart both solve these problems nicely.
This is where Lennart/Freedesktop kill me; rather than notice that the underlying mechanism should be replaced, they figure they should just build a better mousetrap on top of it.
If you want to make linux more "desktopy" without screwing around too much with stupid initd decisions, then fix some of these broken underlying mechanisms.
Unless he's really suggesting that my big problem is that my /usr partition might be on a removable device.
Personally, I don't want to know anything about the timing on which each device is available, to figure out manually the right order to start the services. That's the kind of job a program can and should do. I didn't like when I have to set IRQs manually too, or when X asked me about the data of the video chip that was connected to its bus.
Back to the init scripts. How do those scripts know if a daemon is still running? One mechanism is to check the existence of a file with the daemon's pid! IMO the current system is an anti-aesthetic patch. Only inertia keeps it alive.
This is what I would like to see in a linux system:
* A common config language for the basic services. Something like the defunct [elektra initiative](http://freedesktop.org/wiki/Software/Elektra/) * An automate mechanism to orderly start the system. * Something like Dan Berstein's daemontool to handle the daemons, plus the cron jobs. * For Ritchie's sake, keep logs as text files!
It's perfect. No stupid 5 minute timeout like sysvinit, no stupid dependency management like systemd/upstart, very little to consider when adding new services.
If you want to use it as your init.. just look into 'runit-init'. I've been using this for 4+ years, and it's fantastic.
It's a good introduction to the things systemd brings to the table and why it's worthwhile.
This has the advantage of not having you screw up the actual package managed files, too.
My take: if he builds a bicycle it would be lean, fast and he'd analyse all existing designs to make that happen.
Note the time of the blog. Saying systemd is bloated is IMO so 2013 :P
The Art of Unix Programming (the book) suggests that software should do one thing and do it well, with easy, plain-text interfaces to its input and output. Systemd tries to do everything -- replacing by default so many core system components like syslog and cron.
But what really bothered me was when I realized that systemd stores all the system logfiles in binary by default! I couldn't believe it. All the system log files: boot, kern, messages, whatever, were stored in some binary format. At that point, rather than research how this new binary log file was implemented, I abandoned Arch and systemd for the init system that had served me well for 15 years.
Log files don't need to be in binary. It means we can't use our standard tools: cut, sort, grep, awk, etc. It means we can't use scripts we have written built on those tools we may have built over decades of working with UNIX.
I guess my point is, systemd, by storing things in binary format and by trying to be the 'everything' daemon, violates two core UNIX programming philosophies: use plain-text interfaces & do one thing well.
You can also follow the logs with journalctl -f and go from there as well.
The problem that this solves is the classic 'what log file does this daemon log this error level to on this system', which varies a surprising amount between daemons, versions, distributions, etc.
I don't actually care which log file I look at, I just want to know what errors monit is getting. systemd and journalctl provide that handily.
I would agree with an approach to have systemd send log events to e.g. journald (or syslog-ng, or whatever the user wanted) to separate that functionality out, however.
Using journalctl instead of hunting for the appropriate log files in /var/log seems to offer some degree of platform unity. Maybe it's not the right answer, but it works well enough. My primary complaint is how long it can take on a noisy system to filter a large log, but it's not exceptionally noisome. And most of the time, I just want to follow the log.
> I would agree with an approach to have systemd send log events to e.g. journald (or syslog-ng, or whatever the user wanted) to separate that functionality out, however.
While it probably isn't what you mean as you attach the caveat of task separation, it is possible for syslog-ng to collected logging from systemd's journal. It's a little flaky but it works. :)
You are right, of course. Binary logging was not the only reason I ultimately switched. The big reason was that I needed a distribution that did not change so frequently (although I love what Arch is doing). Stability was so important for my use case, though, that I ended up switching to Debian Stable.
Although journalctl does work well, I admit, I wonder if the advantages could have been accomplished without the binary file format. So many people have homegrown log parsing tools that need to be re-written to handle a binary file format. Or at least re-written to call journalctl as an intermediary.
Yeah, this is a perfectly valid reason and also why I tend not to use Arch on servers (as much as I would like to otherwise, though...). Both have their use cases, and I love Arch for desktops, but it's not a particularly great match for situations where rapid, near-constant change is unwarranted, unnecessary, or problematic. Arch is also a distro that requires a fair amount of love and care--failure to do so, as a friend of mine learned last year when I had to help him update a year old install, can result in an upgrade path that's nearly impossible. Fortunately, ARM was still up at the time.
There certainly is something to be said for stable distros with a strict upgrade path.
> I wonder if the advantages could have been accomplished without the binary file format.
I don't know. I'll be honest: I can't think of a time when I've actually made use of the filtering features of journalctl and systemd's binary format. Nearly all circumstances that have required me to examine the log are usually done by observing the last few entries or manual searches. To that extent, I'd agree that it bears a certain sense of overkill.
That said, it is still possible to feed syslog-ng entries from systemd. So I can't think of a reason to fret much over specific tooling or the likes that expects text logs.
IMO, the udev/systemd relationship, repo, etc. is probably a much more valid complaint, especially for distributions that aren't planning on using systemd in the near future as it likely has some impact on developer/maintainer workload. Although from a sysadmin perspective, the benefits of systemd are very real and material, at least in my experience.
Service introspection could probably use some work to simplify, so that's another complaint. I can never remember the commands half the time. ;)
Rachel mentioned why binary logging makes sense for complicated logs that require rather tricky state machines/regex. Apache for example, in her case.
I am a fan of more consistenly structured/filterable logs that are easier to pin to their service. I'm an even bigger fan of the fact systemd goes to a more Unixy model of not making every daemon author re-invent the logging wheel for their double-forking daemon.
Or perhaps its that having to bundle your logs together into a comparative handful of categories and then sort through them is less than entirely optimal.
That being said, I can't say I am a fan of the integration of other services within systemd, it seems somewhat bloated. And I certainly don't like the whole journal thing. When the journal becomes bigger it becomes considerably slower. And I don't get the point for binary logs. I mean, logs are just text, why overcomplicate things?
I've been using it for over a year in Arch it is really good as an init daemon, I believe it is definitely a step to the right direction especially for desktops/workstations, probably for mobile as well. The rest of the package though, I am not so sure.
The problem with this traditional unix approach is that in addition to init, pm-utils, inetd, cron, etc you end up having thousands of lines of shell scripts that glue all this together in a big brittle mess.
While this approach may be fine for servers which boot once and stay on for ages, are connected to fast wired network and are generally rather stable configurations, I don't want such a system for my desktop, laptop or mobile device.
I expect modern devices to be able to deal with changes in power supply (battery/mains/low battery), connecting and disconnecting devices (storage, peripherals, audio devices, etc) and changes in network connectivity (intentional disconnection as well as network failure). This may involve starting and stopping services, mounting and unmounting partitions and taking other actions when the physical configuration changes.
I also expect my computers to boot and shutdown quickly and effectively.
Some of this is already doable in other systems, like the traditional Debian network setup (originally built in the era of wired networks) which can run scripts on connect/disconnect. But the situation ends up being what I described above: thousands of lines of brittle shell scripts.
I welcome the changes and the effort put in systemd and other init systems. Traditional unix init just doesn't cut it any more, in particular on personal desktop and mobile devices.
I'm glad they did, though. Unit files are far easier to write and having a supervisor behavior in the init system alleviates a tremendous amount of administrative overhead. That's not to say shell scripts are difficult, but they can break in mysterious ways. Though I imagine the same is true for systemd, from an end user perspective, its implementation (once you understand it) requires less effort.
There are those who disagree, and I respect that. But the improvements to my desktop boot time are substantial. And writing a quick unit file to run something on boot requires little mental overhead. Just a glance over the manpage is enough to get started. sysvinit scripts on the other hand... well, maybe my memory is really poor, but if it's been more than a few months since I've written one for the specific system I'm targeting, it takes substantially longer. ;)
So, it's likely I'm stupid or systemd is easier. Or both. Probably both.
systemd actually looks like it was designed based on experience. "Worse is Better" as a development philosophy works for getting stuff up and running and finding out what the real problems are so that you don't make solutions for non-existent problems. But when enough time has passed, and enough glue and pieces of string accumulated, it's time to take a step back and consider a redesign.
That's another thing I like about systemd. With the separation of "stock" unit files in /usr/lib and user-supplied ones in /etc/systemd, you can override the ones distributed by your OS vendor without worrying about updates breaking your changes. That's already saved me once or twice during Arch's transition from sysvinit to systemd, in fact.
One thing that comes to mind is an issue I had some time back with running a proprietary service in a manner that would allow the init system to control it when it forked itself a number of times and spun up separate processes. start-stop-daemon was confused by this behavior and it required some tricks to collect the approriate PID just to be able to stop the service (I may be wrong, but I think it was the TeamSpeak 3 daemon which behaves weirdly). systemd's use of cgroups ignores this rubbish entirely, because it's effectively impossible for systemd to lose track of what processes belong to which service. Want to kill the service? No problem.
Some in this thread seem convinced that systemd is a solution looking for a problem, but I suspect some of them lack experience with systemd. Or stubbornly cling to their ideals of what an init system should do. I like the unification of a supervisor-type service and the init system. It makes my job easier. And that's not even half of what systemd is capable of!
No, I don't. Typical linux distros may, but I sure as hell don't. I have a very simple, easy to read and debug set of shell scripts that work very well: http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/etc/rc?...
Future apps will need to copy themselves and their data to other devices, perhaps even suspend on one device and wake-up on another.
Making changes for one use case at the expense of another without knowing the history is an ignorant change for churn's sake. Churn is not productivity, but the appearance of doing something.
To me, it makes perfect sense for these tools to be tightly integrated, both with each other and with the OS kernel, simply because they are interrelated and integration leads to greater stability and a less fragile system overall.
No doubt upstream could be less hostile towards attempts to distill out the portable bits, but I can understand their viewpoint... Systemd is ultimately about making Linux-based systems run well.
This is completely counter to the way we actually design software - which is to create modular, loosely coupled components, using abstractions which allow them to adapt or be replaced, and trying to make as few assumptions as we can, since we can't predict the future. It's well understood that tight coupling is what leads to fragility and maintenance nightmares, due to decades of experience writing and maintaining such abominations.
In this case, close integration with the kernel is required to be able to produce the features envisioned.
You might perhaps find some instances where systemd components are needlessly interrelated, but the claim that it isn't modular is mostly a myth.
an init system should be able to run !BEFORE! /usr is even mounted. Thats the division between /directory and /usr/directory, that /usr is mounted later.
If anyone hasn't tried out systemd or journalctl yet, I suggest running a copy of Arch Linux in a virtual machine.
I realize initscripts are "simple" in the sense that they borrow immediately from the shell and therefore possess all the advantages (and disadvantages) implied by such. Unfortunately, the notion of a sane initscript seems to be greatly varied in the sense that it depends largely on the distribution and the package maintainers. It isn't that they're necessarily difficult to write as much as having the power of the shell available inclines some authors to do very stupid things.
Comparatively speaking, a unit file should be exceptionally easy to understand at first blush.
For many people it seems the answer to the question posed in the post title is "I would expect it to not be designed/authored by the man behind pulseaudio".
Since debian is supposed to be universal (hence, support alternative Unix kernels), choosing systemd barely means that they will have to drop this trait of the distribution.
I have mixed feelings about the BSD & Hurd ports. "You shall not crucify Debian Linux upon a cross of Hurd" and all that.
Lennart hate is fine, but let systemd stand on its technical merits (or lack thereof, if you really feel that way).
https://lists.ubuntu.com/archives/upstart-devel/2014-January...
But yes, it is still a work in progress.
I think your observation is true though.
He and Kay claimed that kdbus couldn't be used for Binder as well. Greg thought it was possible. Only after writing kdbus he came to the conclusion that Lennart+Kay had it right.
Suggest to write a complete patch to make systemd work on FreeBSD. I assume you'll discover Lennart is right :-P
However, I was wondering if platform-specific forks are a sustainable solution.
Having used a bunch of Lennart's software, I'm generally coming to the conclusion that he's better at ideas than code -- but once his projects take off and the bugs get fixed, the end result is significantly better than what we had before.
(Using pulseaudio as a specific example, per-app volume controls and easily mixing and matching sound sources and playback devices; seamlessly switching from speakers to headphones to HDMI to a sound server on another PC entirely, etc -- features that are too high-level for the kernel and too low-level for applications, and thus can only be brought into existence by a fairly large change in the audio stack)
What is interesting though, is to see how the BSD ecosystem will handle this. Will they adapt in time or they will lag even further behind this?
* NetBSD might reached the point of virtual insignificance long time ago.
* OpenBSD is out of funds - which is a shame considering the software they produced (PF, OpenSSH, OpenNTPD), but their indifference for desktop makes them obsolete.
* FreeBSD (I use it in both my servers) is the most up-to-date but now with system-d/upstart not supported, I think they will run into trouble relative soon.
Darwin said it best: the ones most adaptable to change survive. Of course, the other happy scenario is that given the fact that these OSes power some big-corp servers, these big-corps will step in and fund these projects, I wouldn't hold my breath though.
The argument that sysv "is Unix" is pretty much horseshit at this point. Solaris and OS X don't use a sysv style init, most of Linux doesn't any more (either upstart or systemd). At this point one is redefining "real Unix" down to NetBSD and HP-UX.
Who uses NetBSD?
It's overcomplicating things.
Yes, I love automation. I puppetize and ansible-ize the hell out of my serverparks. But udev, kudzu, network-manager and now systemd: they're solving the problem by creating more complex problems.
I just have very bad experiences with automagic systems trying to outsmart me. And failing miserably, costing me a saturday-evening with the missus. And outsmarting me ain't even that hard!
I can also gives loads of similar examples for just a computer, but above behaviour is something I really appreciate every day.
That conclusion was quickly drawn. :)
Also, I refuse to use "make" because I prefer typing long and complicated gcc lines and working out all the header file dependencies in my head. Why on earth would I trust my computer to do that for me?
My experience is with kudzu, magically finding new networkcards all of a sudden. Or udev, which then automagically turns these new found cards into eth12, instead of eth0.
Too often I have 'smart' systems trying to figure out what should be done, and fail miserably. Especially when trying to debug a software-problem, in an enterprise production landscape.
No, I do NOT want systemd to 'figure out for me' that I need network when I start sshd. Because sometimes I don't NEED a running network. When debugging a broken server, I don't want to have to first dive into the configurations of systemd trying to figure out all the dependencies, and stopping systemd from starting, or stopping them.
So once in a blue moon you want to run systemctl with "--ignore-dependencies"? That's a much different argument than "would prefer to do my own thinking, not have the computer try to outsmart me."
You're right, it's not smarter than you—but it does has a better memory than you. To want things dumb for the 1% case seems stupid to me and an untenable position to take, hence my initial sarcasm.
There is no such thing as flat out wrong.
Troubleshooting other peoples servers I need to work around many different situations. A '--ignore-dependencies" is nice, but won't cut it as it depends greatly on the situation at hand. Perhaps sshd also has a dependency not just on networking but also on slapd or something. Perhaps I do not want networking, but I do want slapd to be running.
systemd having a better memory than me is, well, true. It doesn't solve my problems for me though. The Windows Registry also has a better memory than me, but I prefer my configs in a textfile I can parse, grep and maintain with any tool I choose. Same with systemd: better memory than me, but I do not really need a better memory than me. My memory is good enough.
But meanwhile the world has made up it's mind and in all likelyhood Debian will be using systemd. So I might as well start the learning curve and be done with it.
I'm quite sure either would be a massive improvement over sysvinit though.