System XVI: A replacement for systemd
github.com
github.com
Alright, I'm an acquaintance of the author, so let me clear things up.
SystemXVI only began less than a couple of weeks ago, and is still a very early stage project. However, the author (David Mackay) decided to be a bit of an expedient troll and advertise on /g/, to arouse some controversy. It ended up blowing up greater than anticipated, even ending up being mocked on Linux Unplugged and hitting /r/linux as the project was still a skeleton.
First thing's first:
This is not a joke project. Mackay is not some amateur, either. He's a former Solaris/illumos committer and most recently was involved in the Tox community. He actually wrote a small service manager called Charge last year as an experiment, and some of the logic is borrowed on to SystemXVI (though the architecture is different).
SystemXVI, the design is quite interesting. The closest equivalent would probably be Solaris SMF, but it's still different. init(8) is kept small and instead SystemXVI is based on the idea of the service repository (inherited from SMF) and delegated restarters. The repository is a hash table that stores instance information and service properties. It's useful for having a point of fault recovery and to introspect services in a dynamic, transient manner as they run.
The primitive process management is currently in libs16rr, but the actual supervision strategy is defined in restarters. Much like Solaris SMF, there is a master restarter, but certain services (i.e. local socket services) can, for example, opt to be watched by a delegated inetd-like restarter that uses the SystemXVI RPC interfaces. Or let's say you have a Java, Erlang or other service that needs special behavior, you can write a restarter for it while reusing SystemXVI's process management.
TI-RPC/SunRPC plays a big role here. It's much leaner than D-Bus and at least nominally capable of network transparency. Android has a portable implementation in ~5000 LoC.
Dependencies and ordering is actually calculated by a separate process called svcorder, influenced by BSD systems' rcorder(8).
There will be systemd unit file compatibility. Internally, SystemXVI has semantics similar to systemd units, but much simplified - no need for transactions or complicated job modes, since a repository is used instead to contain state.
More here: https://github.com/ServiceManager/ServiceManager/blob/master... (Flowchart: https://github.com/ServiceManager/ServiceManager/blob/master...)
It's a refined SMF-like system with influences from daemontools and BSD rc.d. Very microkernel-ish, it's meant to have strong module/communication boundaries and have its components be easily hotswapped or rewritten per a stable RPC interface.
It's very early and the project really shouldn't be getting such publicity, but a pre-alpha might be out soon, and it's a project well worth watching. I have confidence it will turn out well.
I would love to see a (series of?) blog post(s) about SMF. This wouldn't be a missive against systemd, but given that SMF has seen 10 years of production use, it would be a valuable history lesson. Why was SMF needed for Solaris, what was scope of responsibility, why were the abstractions picked, why were its boundaries where they were, and finally what has and hasn't worked for it?
It seems like the technical community would eat that kind of content up.
I'm fairly certain Oracle blogs about news regarding SMF with relative frequency.
For instance Bryan Cantrill (full disclosure I worked for/with Bryan while at Joyent) was at Sun for the development of SMF, he's had experience using SMF in production, and more recently from his LX Brand work he's been exposed to the facilities that Linux has created to deal with the same sorts of issues.
Bryan is a superior technologist and a hell of a story teller, I'm sure I would enjoy reading/hearing his telling of why things like libproc and contracts were created and how they evolved over time, and if that as any relationship to why SMF is designed and behaves the way it does.
Note, Bryan is just an example -- there are many other people in the community that were there and could tell the stories. He's just the first person I thought of.
The point is that [re]creating interfaces/subsystems is ok, so long as we do that with the full view of history, such that we avoid that repeating cliche.
(I prefer run on sentences)
Isn't that true of D-Bus as well? I mean, maybe it's just semantics, but if you're going to use the word "nominally" (read as "it claims to be") when saying it's network transparent, D-Bus is certainly "nominally" network-transparent as that's an advertised feature of it.
In contrast, SunRPC is RPC, so network transparency is a given. The reason I said "nominally" is because in the case of SystemXVI, having proper distributed service management (not a goal at this point) would require more than that.
Search for "TCP".
(A shared secret sent in plain text is not security).
Which is more or less what CoreOS's `fleet` already does on top of systemd. They used a different protocol for remote communications (etcd, ie. HTTP+JSON) vs. local, which I guess can be a sensible choice as requirements are quite different.
To be entirely fair, I think there are few communities that can claim the honour of being worse and more amateurish than /r/Linux. It's incredible how many Linux experts who have used nothing but Arch Linux for about an year can gather in one single place. Every once in a while there's an OK post, but it's downvoted.
I hope your friend didn't take them too seriously.
The Linux ecosystem is increasingly centralized and most of the organizations that are centralizing it, like the Gnome project, are actively discouraging contributions that don't match their agenda. This is reducing the importance -- and cohesion -- of any community. It's no coincidence that many people are finding less common ground in software they're all using than in software they're all hating.
The only Linux community I'm still vaguely in touch with (I read the mailing lists and sometimes the forums) is the Gentoo community. Largely because my only remaining Linux machine is running Gentoo. It eats a lot of my time, but stubborn USE flags control seems to be the only way to get a usable Linux distribution lately, and I'd rather spend an extra hour a week configuring it carefully enough to keep crap software out than spend a whole evening each month dealing with sudden breakage. It's not necessarily a friendly community, but most people seem to know what they're talking about.
Frankly, as bitterly as it may sound (and it's coming from someone who started using Linux just in time for the 2.0 to 2.2 switch), I'd rather recommend the OpenBSD community.
Arch has both made it easier for new Linux users to adopt it day-to-day while still appeasing the more advanced users, ala Gentoo. The Arch wiki has been the single best contribution to the Linux desktop world in recent memory IMO.
I'm curious how the more hardcore *nix users view it.
> I'd rather recommend the OpenBSD community.
My attempted to use OpenBSD and take part in the community left me disappointed. On the surface it seems to be exactly what I'm looking for (correctness + security). But I was turned off by the culty vibe of the users and it seemed like a backwaters in terms of software versions, as everything outside of the core seemed severely outdated.
I think Arch is good in terms of "learning experience" for someone who wants to learn more about how a distribution is put together, but that's about it. I used it for a while, back in 2005 or so, but our later contacts have been... less than fortunate. I tried it again two or three years ago and my system regularly broke after every update. I eventually ragequit when I ended up with a system that wouldn't boot because, in The Great Move of Everything to /bin, it somehow managed to screw up the bootloader. I figured my data was going to be next, wondered what the fuck they were thinking even in the context of a rolling release, shrugged and wiped it out.
The Arch Wiki is a great repository of information though. It's even better than the Gentoo wiki was, back before it got (disastrously) wiped out.
> I'm curious how the more hardcore *nix users view it.
They probably think lowly of the whole Linux thing, but what do they know :-)
> But I was turned off by the culty vibe of the users and it seemed like a backwaters in terms of software versions, as everything outside of the core seemed severely outdated.
I found the culty vibe of the OpenBSD community to be far less obnoxious than that around... pretty much any major Linux distribution.
There are quite a few outdated packages in ports though, yes. That's often because of a lack of maintainers, but there are programs where that's just because of compatibility problems. There are very few broken packages, if any, in the ports tree -- and sometimes the broken status is very paranoidly assigned (i.e. I've had packages that were marked as broken but it turned out they ran just fine).
Having used Arch for years and experienced the same thing years ago I've found that stability has greatly improved in the last year or two. I believe they were going through some major transitions back then and now have found a good stable foundation which they are now building on.
I haven't had it break in a very long time and having brought this up before on HN I've had many other people share a similar experience with Arch.
Although this is possibly due to the fact I've become significantly more experienced with Linux as a result of my professional work. Dedicating yourself to a single distro and becoming deeply familiar with it can be rewarding in that sense.
I'd be happy giving it to a new programmer to use as a style guide. I have no idea if the architecture is suitable for the purpose, but anyone who is dismissing this based on code quality should be ignored.
Representative Sample: https://github.com/ServiceManager/ServiceManager/blob/master...
#define DbgEnteringState(x) \
printf ("[%s] Unit entering state %s\n", unit->name, #x);
How about: static void dbg_entering_state(unit_t * unit, const char * state_name) {
fprintf (stderr, "[%s] Unit entering state %s\n", unit->name, state_name);
}
Friendlier to humans reading code, IDEs, and debuggers. I went ahead and fixed it writing to the wrong stdio stream too. SetOrExit ("S16.Name");
Oh wait, SetOrExit is a macro ... #define SetOrExit(Name) \
if (!prop_find_name (new_svc->properties, Name)) \
{ \
OnError ("error: %s not set", #Name); \
}
Oh wait, OnError is also a macro #define OnError(...) \
fprintf (stderr, __VA_ARGS__); \
goto on_error;
There's a goto hidden in a doubly-nested macro? Sorry, that is not good practice.It does not use macros or goto and I find that it makes the code easy to reason locally about.
In summary:
struct RefreshDevices {
// fields that require clean up
}
static void deinit_refresh_devices(struct RefreshDevices *rd) {
// clean up rd regardless of what state it's in
}
static int refresh_devices(struct SoundIoPrivate *si) {
struct RefreshDevices rd = {0};
if (error_occurred) {
deinit_refresh_devices(&rd);
return ErrorCode;
}
// ...
deinit_refresh_devices(&rd);
return 0;
}
Whenever the ownership of a resource changes to something other than the stack of refresh_devices, you just null it out and it won't get cleaned up.If calling deinit on every exit path of the function is too much typing you can wrap that function in another function that calls deinit, and the function doing the actual work can just return an error code without worrying about cleanup.
What's the benefit of this way over a more explicit coding style? The only benefit is saving a few lines of typing. Code is typed once and read 1000's of times. It's better to optimize code for reading, not for writing.
Feel free to read the Joint Strike Fighter (JSF) coding standards which says "Goto shall not be used", and "Macros shall not be used, inline functions are preferred". Also, see NASA's Joint Propulsion Labs (JPL) coding standards where they recommend against using goto, and only allow simple macros (hint: a goto inside a macro is not simple).
Some really smart people put a lot of effort into creating coding standards for critical systems. Even if you don't believe me, when Bjarne Stroustrup is hosting the coding standard on his personal website you might want to pay attention.
Also note that systemd employs gcc's __attribute__((cleanup(...))) logic to improve code clarity.
From earlier this year, NASA's 10 rules for safety critical systems [1]:
Rule 1: Restrict all code to very simple control flow constructs.
Do not use GOTO statements, setjmp or longjmp constructs, or
direct or indirect recursion.
I don't think gotos are inherently bad, but I choose to heed the advice of people with more experience than me. (I do think that hiding gotos in macros is inherently bad, though.)[1] http://sdtimes.com/nasas-10-rules-developing-safety-critical...
On the other hand no other engineering will be asked to change the requirement and spec hundreds of time during its lifetime either since that would be impossible to achieve.
2. the return value of malloc() et al. isn't checked (http://www.etalabs.net/overcommit.html)
3. the register storage-class specifier is used multiple times (mainly lib/s16db/translate.c)
4. function declarations are missing a prototype (() instead of (void))
I (and likely the author) disagree with you on the importance of checking the return value of malloc(). Because of overcommit (as you point out), a non-null value returned from malloc() does not mean that you will not crash when you access the pointer. If the pointer is used directly, crash on NULL might a reasonable approach. It's when NULL is retained and then used as the base of an array that it may become a security problem. Configuring malloc() to abort() on failure would be probably my preferred solution.
I agree with the last point, but think it's a minor one. While I'd like C to treat () in a function definition as equivalent to (void), for historical reasons it does not. The author trades off the visual noise of the word 'void' for better error reporting. But depending on your compiler, you may still get a warning. On the computer I just tried, 'icc' and 'clang' gave clear warnings but 'gcc' did not.
nate@ubuntu:~/C$ cat void.c
#include <stdio.h>
int empty() { return 3; }
int main(void) { printf("%d\n", empty(4)); return 0; }
nate@ubuntu:~/C$ icc -Wall -o void void.c
void.c(3): warning #140: too many arguments in function call
int main(void) { printf("%d\n", empty(4)); return 0; }
^
nate@ubuntu:~/C$ clang -Wall -o void void.c
void.c:3:40: warning: too many arguments in call to 'empty'
int main(void) { printf("%d\n", empty(4)); return 0; }
~~~~~ ^See the experience of the D-Bus (which tries hard to be OOM-safe) on this topic: http://blog.ometer.com/2008/02/04/out-of-memory-handling-d-b...
That would be pretty neat. But until then, the world will continue to run on systemd, because it _works_.
SystemXVI will have systemd unit file compatibility. SystemXVI natively uses an INI-like file format itself.
SystemXVI uses SunRPC for communication. Writing a systemctl wrapper could be as trivial as a shell script.
But until then, the world will continue to run on systemd, because it _works_
The world doesn't run on systemd, and it isn't a "just works" thing in the slightest.
Redhat are betting their particular farm on systemd in RHEL7, and Debian (hence recent Ubuntu releases building to the 16.04 LTS) has adopted the suite.
Is the point you are making above based on the percentage of servers running earlier supported Linux implementations or on the existence of alternatives such as the BSDs?
It's basically their product, so...
> and Debian (hence recent Ubuntu releases building to the 16.04 LTS) has adopted the suite.
I think Ubuntu adopting systemd has less to do with Debian doing so and more to do with Gnome becoming increasingly difficult to package without systemd.
Does it?
When Debian switched to systemd my system stopped booting. When I managed to "fix" it, it worked superficially but displayed "Segmentation fault" at every boot [it still does that on another one of my machines, which in fairness never "broke".]. This in addition to boot-time fsck breaking and my bluetooth audio and keyboard setup no longer working. Based on this I am not inclined to think of Mr. Poettering as an "it just works" kinda guy. On a related note I have never successfully gotten pulse audio to perform its intended function of playing audio samples.
After ~15 years of mostly tracking Debian sid on my personal machine, I switched to FreeBSD on my laptop when seeing these issues. No major issues since.
I moved from Debian based distros to Arch a few years ago and found that my machines are a lot more stable. Because it you do the configuration yourself (rather than the configuration being in the package a la Debian), you are a lot more free to jettison dependencies of questionable quality. Though, I had a lot of trepidation when Arch switched to systemd, it has not yet broken my system in any way.
Still, there's nothing wrong with FreeBSD, so if it's working for you, might as will stay with it.
Every third update it will just start working and I'll think "Oh they fixed the bug". Then the next update breaks it again. I don't doubt that bluez must work well for some hardware, but for mine it is practically black magic to get it working.
My only point in fingering bluez is to say that bluetooth going down is not a surprising event on my machine and isn't related to systemd at all. I suspect the same is true of the OP's box. It was probably just a coincidence that bluetooth died at the same time that systemd was installed.
While the Linux code assumed the hardware was required to be up and ready in 2 minutes or less, the 2 minutes mentioned in the spec was just a suggested minimum wait time. Thus it would give up on devices that was still in the process of powering back on.
That not to say that bluez have been in great shape in recent years. But then it is also a complicated concept for *nix, as it straddles root and user. You have their whole pairing interface on the user side, and then all the device nodes (HID, Audio, etc) on the root side.
This is probably the best example I've seen on how NOT to program in C. Unneeded typedefs, random macros for simple logic. Do not use."
Its nice to see a sense of humor and some humility. There is a healthy ecosystem of Linux distributions that do not use systemd and have no intention of doing so. Slackware, Void, Puppy, Crux and many others.
The IPC layer for getting launchd to work involves amongst other things FreeBSD kernel changes to support the Mach IPC API; similarly to how the systemd people have already achieved (subreapers) and have been pushing for further (kernel Desktop Bus) kernel changes for systemd's benefit; and indeed much like how IBM AIX gained kernel augmentations for supporting the System Resource Controller: the SIGFORCE and SIGNORM signals and the SRC_kex.ext extension.
vezzy-fnord is I suspect refraining from pointing to http://blog.darknedgy.net/technology/2015/08/26/0/ (https://news.ycombinator.com/item?id=10127120). (-:
In happy news, nosh has a picture and a Burns allusion:
* http://homepage.ntlworld.com./jonathan.deboynepollard/Softwa...
* http://homepage.ntlworld.com./jonathan.deboynepollard/Softwa...
http://uselessd.darknedgy.net/
I am a systemd user and Arch Linux fan (but definitely wanted to get to Gentoo to have the most choice), but I have been hoping to see news of uselessd pop-up again.
I think the reaction to systemd is quite upsetting, for its tone and not its message about choice which is core to our support of open source values, but I welcome the alternative solutions and approaches and I am very curious to see what percolates to the top.
Personally, as a budding Lisp geek, I wait for dmd.
https://www.gnu.org/software/dmd/
Although, and I know I have seen him post here before, the vitriolic systemd hate on one of his blog posts make me worry about reactionary development, just like this and other projects.
I have heard a lot of good about SMF though, so I will need to follow all of them now.
There will be a successor, though I wasn't expecting SystemXVI to arrive. I might just rethink it to be a layer on top of SystemXVI RPC, I don't know yet.
Well, either way, I wish you luck. You Github testimonials at the bottom of the SystemXVIREADME were very entertaining. I am glad you have thick skin. You will need it, and I cannot wait to see real competitors in this space beyond SysVinit, runit, and of course the dreaded systemd.
Also, this is not yet a replacement for systemd:
#include <stdio.h>
int main (int argc, char * argv[])
{
printf ("defer work on the pid 0 until we have a functioning "
"service manager\n");
return 0;
}
(I think they mean "pid 1")>But then again, if all that appears too simple to you, call it (but never spell it!) System Five Hundred since D is the roman numeral for 500 (this also clarifies the relation to System V, right?)
How are those things nonsense?
Are we really going to spend lots of energy the next n years discussing the best init system?
Why the hell shouldn't we?
Isn't this a pretty simple and solved problem by now?
Nope. It's actually a very hard problem with little in the way of complete solutions. Take your ignorant trolling elsewhere, please.
Windows has no fragmentation issues in the core system.
Whether it is Systemd or System XVI im not really that bothered. Something needs to be decided fast however, and Systemd has a big head start.
I just hope it dosen't end up like the KDE/Gnome mess.
One kernel. One init. One desktop.
Windows isn't gaining anything on the demographics that use GNU/Linux for cloud deployments. You're reading too many articles on HN and extrapolating that Windows must be curb stomping its competition because of some smart business decisions on part of Nadella and co. You're further making the assumption that Linux needs to compete with Windows. It doesn't.
It's not that there is fragmentation, so much as the problem space is quite open-ended and there are multiple solutions. Forcing a square peg into a round hole (systemd ueber alles) is a recipe for impedance mismatch and stagnation, not unification.
One kernel. One init. One desktop.
Ein Volk. Ein Reich. Ein Fuehrer.
If you want an operating system where everything is developed in tandem, you might want to consider switching to one of the BSDs.
It seems like a lot of people are calming down about the issue now.. so lets not do all that again
My question was more about the GP elaborating why a new system would be something "[no one] wants to go through this again."
(I'd characterize systemd as being a sysvinit replacement to be a great misnomer. It's not. It's an entire framework for providing certain low-level userspace daemons, utilities and auxiliaries meant as a common middleware for GNU/Linux distributions. Even systemd-the-PID1 cannot be called a sysvinit replacement in any integrity.)
Because the migration was terrible that is why. And I still continue to find stuff I hate about systemd. The latest was just yesterday, while troubleshooting some disk issues I wanted to fsck root before rebooting. Which normally means just remount root as read-only. Tried that and it didn't work, "disk was busy". Jumped into runlevel 1, still disk was busy. Rebooted into rescue mode, remount ro, still disk busy. After googling and digging through lsof, I could see it was systemd processes and journal that was holding onto files and preventing a remount. Tried killing systemd, journald, even kill -9, it just restarts it self and holds the disk. Darn. Did the touch /forcefsck trick, rebooted, yet no fsck. Darn again. More googling later, I see you have pass kernel parameters at boot time to force a fsck, and no, you can't select which FS either, it just does all of them. Rebooted again, fsck is running of course. I just wanted root checked but now it is doing all my terabyte partitions as well. Gave up at this point and just left the machine and hoped for the best.
chmod a-x journald && pkill journald
of course, it is bad idea to touch damaged FS before fsck.To run an unscheduled fsck of your root fs (and only your root fs), you have to remove the execute bit from a systemd component, kill that component, then remember to reset the bit after you're done?
That's nuts.
It will prevent it from running again. So it is basically a big FU! to systemd.
"So you you think you gonna restart yourself? Oh, no you won't"
https://forums.gentoo.org/viewtopic-p-7811550.html#7811550
Pottering was apparently told about this in 2013, when he simply declared it expected behavior because dbus was "rock solid".
http://lists.freedesktop.org/archives/systemd-devel/2013-Jan...
Incidentally, this might be the original reason for kdbus - it's harder to restart the daemon after it becomes a kernel module.
Yeah, especially given that its stated reason (Moving into the kernel gives us a 2x speedup over userspace dbus! Dbus is slow and we NEEED the speedup!) is mooted by the 10x speedup that cleaning up the userspace dbus libraries has achieved. [0]
Having said that, in the systemd-devel post Poettering says that they're working on...
> ...improving this in two ways: even if it still brings down the system, at least allow logind to handle this case nicely, so that you get a sane getty. [The other way is to push dbus into the kernel.]
Systemd is a sprawling project, so I could see why it has taken more than two-and-a-half years to fix a local DOS caused by the perfectly normal method of responding to a DBUS upgrade. ;)
[0] Fun fact: Those cleaned up libraries were supposed to be released after kdbus was merged into the kernel in 4.1. The Systemd Cabal was so disappointed that they had to ship the performant userspace dbus libs before the dbus daemon got pushed into kernelspace.
I may have read things that are biased but I have a feeling it is a useful project and it does a lot of thing elegantly or right. But the project acts like it is the center of the universe and all should act to meet its needs. I have seen such project/modules at work, they usually makes everyone happy to start with and down the road when they finally get stuck and I am forced to consult we are already long way down the slope.
kdbus is NOT ONLY about performance. It's about the lifetimes, the availiability in the early userspace, the new marshalling format, the new user bus concept, and so on, and so forth. It's about separating the transport (in the kernel) and the policy (in the userspace).
Most of these points can be achieved without putting the transport in the kernel, but some can not.
D-Bus is a powerful design. However, being mostly a userspace solution its latency and throughput are not ideal. A full transaction consisting of method call and method reply requires 10 (!) copy operations for the messages passed. It is only useful for transfer of control messages, and not capable of streaming larger amounts of data.
So yeah he tried to sell performance as the big thing for his version of dbus IPC. I'm guessing he (and group) spent more time with dbus than linus and failed to find the actual cause of the performance issue or tried to hide it give more credence to kdbus. In both cases it makes it that much harder to trust their reasoning and conclusions.I never said kdbus is ONLY about performance. But from what I'm reading they are not performance people but they think they are. The most dangerous people are those who don't know their own limitation or weakness. And your comment simply emphasise the practice of listening whats being said. So yeah given that I don't know dbus or ipc I have to trust your logic and you only make it hard to do so.
FF 40.0.3 on 32-bit Linux.
LWN article: https://lwn.net/Articles/641275/
LKML thread: http://thread.gmane.org/gmane.linux.kernel/1930358
> It's about the lifetimes, the availiability in the early userspace...
Starting dbus in your initrd [0] makes it available to the system from before the first second your service start machinery is started. If you want dbus available to userspace in your initrd, make starting dbus the first thing you do after you load your initrd. Is there some particular reason why this doesn't meet the lifetime and availability requirements?
[0] Remember that the initrd is responsible for things like decrypting the rootfs. When you're in the initrd, your real system is often not even remotely ready to run.
[S]urely, something like:
mount -t tmpfs /run && pkill journald
[ed: that is, for logs on /var/log: "mount -t tmpfs /var/log && pkill journald"] would make more sense? (In my experience systemd itself does something funky on / -- so the above wouldn't be enough -- but still seem a little more sane than running a chmod on a file (presumably) residing in an fs you want to fsck).> If you need to check the root file system, you can reboot into single-user mode with the root partition mounted read-only by issuing the -b switch at the LILO prompt. The -b switch will be passed through LILO to init and will cause an emergency boot ...
-- David A. Bandel (1997-01-01). Disk Maintenance under Linux. Linux Journal. http://linuxjournal.com/article/193
Systemd on Fedora supports this perfectly fine, so I can't really call this a failure of systemd itself.
1) It's actually super easy to get to a read-only root mode.
2) It actually is a problem of carrying over obsolete conventions from prior init systems.
3) It's not your fault because systemd has extremely poor documentation on this AFAICT, to the point that people searching for this problem can't even get answers from others that know how to do it, or at least those answers don't rate very high.
4) Just boot with kernel param "emergency" instead of "single" (or "rescue" which is an alias for "single") and you will get a single user mode with root already mounted read-only for you.
And now for the longer rambling version.
Okay, so I spent a little time on this since I happen to have a virt for Fedora 20, and I finally found a couple of solutions that work. First, the systemd solution, which is emergency mode. It's correct that the journal will cause problems here, but that's because the journal was already running and apparently systemd doesn't want to, or is unable to stop it. The solution is to reboot into emergency mode. This can be accomplished by using the kernel param "systemd.unit=emergency.target", or just the param "emergency". This will not only boot you into a single user mode, but will default to having root mounted read-only.
To clarify, this appears to be a new mode in addition to single user mode. Single user mode can still be reached with the kernel param "single" or "rescue" or "systemd.unit=rescue.target".
I finally stumbled on this when I decided to look at the directions for what I wanted to accomplish for the distro I was using[1] and not for systemd specifically. Which makes sense, but I think we are all to stuck on "systemd is different, how do I do this in systemd" rather than "how does my distro say I should accomplish this now, given the changes they've instituted."
1: https://docs.fedoraproject.org/en-US/Fedora/18/html/Installa...
It actually is not. It's a problem of not knowing the conventions from prior init systems. emergency mode did not originate with systemd. It has been around since 1995-12-03 (Miquel van Smoorenburg's System 5 init clone, version 2.57d).
It is also a case of not knowing the correct/optimal prior convention, but that doesn't make the original statement untrue.
Which distribution was this? I just tried on my laptop (running Debian 8, fs on top of lvm on top of LUKS), and a simple:
telinit 1
# log in on console
mount -oro,remount /
worked as expected.[ed: Actually just tried forcing an fsck as well, and worked without a hitch. Just remember to "mount -oremount /" before running a "telinit 5" (Nothing really bad happens if not, but the system(d) will be confused if rootfs is mounted read-only).]
Of course, knowing that does point to one obvious way to avoid the stated problem: move the journal back into /run again (per the journald.conf manual page) and simply restart journald. No mucking about with permissions required. But that's not the only thing that xe could have tried. emergency mode, for example, is documented as not mounting additional non-API filesystems at all, nor read-write remounting the root volume. See http://freedesktop.org/wiki/Software/systemd/Debugging/#boot...
[ed: Given that it's not wise to write to a fs of questionable state, editing /etc/systemd/journal.conf is out. So it would appear shadowing the sub-tree in question with a tmpfs mount would be the sanest approach?]
[ed2: Just checked that if setting "Storage=persistent" in journald.conf, forcing logging under /var/log (and with /var/log as part of the rootfs) at least here, shadowing /var/log with a tmpfs works, in order to remount rootfs read-only in "runlevel 1" (actually systemctl rescue, when using systemd). Not sure if there's any elegant way to unshadow/unmount /var/log w/o a reboot though.]
> I have to say, I don't really get the hatred of systemd. I think it improves a lot on the state of init, and no, I don't see myself getting into that whole area.
> Yeah, it may have a few odd corners here and there, and I'm sure you'll find things to despise. That happens in every project. I'm not a huge fan of the binary logging, for example. But that's just an example. I much prefer systemd's infrastructure for starting services over traditional init, and I think that's a much bigger design decision.
> Yeah, I've had some personality issues with some of the maintainers, but that's about how you handle bug reports and accept blame (or not) for when things go wrong. If people thought that meant that I dislike systemd, I will have to disappoint you guys.
[0] http://linux.slashdot.org/story/15/06/30/0058243/interviews-...
shrug
[0] Upstart, OpenRC, daemontools...
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/da...
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/in...
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/sy...
* the parent of all processes, including orphans
* the executer of various entries in /etc/inittab, depending on runlevel
* the configurer of local and serial consoles
* the handler of CTRL+ALT+DEL
?
So, I guess that the Debian folks were incorrectly claiming that they were looking for a replacement init. They were looking for a replacement rc system, and were willing to accept a replacement init as part of the package.
As best i could tell, Torvalds didn't have an issue with systemd as the init.
Nor do i think many would (outside of the whole journald thing), except for the snowballing of sub-daemons and absorbed projects (udev for instance existed for a decade as an independent project before being folded into the systemd code tree).
Without all that, it may well be that systemd would just be one init among many (i barely noticed the existence of upstart for instance).
-----
/u/TheReverend403, reddit. [...] the world doesn't like goto. First of all, it can cause all kinds of memory leaks.
Your program is simply an attacker's target.
As soon as a distro maintainer sees that goto, it'll be rejected.
-----
Clearly, TheReverend403 hasn't read any systemd source code.
systemd$ rgrep goto . | wc -l
2690
One big problem with systemd is the internals: all the stinky "char *" string processing which is repeated all over the place and the badly greenspunned OO programming system that it contains and so forth.
The kinds of things that systemd does are best written in a high level language (of course, with suitable bindings for the system calls required).
Old SysVInit gets away with being written in C because it's small and simple and just handles the /etc/inittab, which has a simple format. Anything complicated is farmed out startup, shutdown and runlevel scripts.
The way to proceed was to add an extension language to init so that actions can be written in that language to do more kinds of things.
Perhaps the executable which runs as PID 1 can be a very tiny "init kernel" written in C, which intrisically does some of the things that only it can (like reap processes parented to PID 1), and farms out everything else to a larger daemon. This would be for the sake of fault isolation; we don't want PID 1 to crash. The risk of that is high if you make it a large and complicated program, with a large and complicated run-time infrastructure.
But don't fret. Our car will provide you with it's own desktop built into the dashboard. Our car can communicate other cars, but only those of the same model, and even if our car doesn't have support for something you need yet, don't worry, as we'll build it directly into your car soon!
systemd "aims to unify pointless differences between distributions", so nothing needs replacing