Debian Dropping the Linux Standard Base
lwn.net
lwn.net
In short: because having things in /usr is equally compatible, and makes some useful things like atomically snapshotting /usr to snapshot all executables, mounting /usr readonly, etc. possible.
But it is this kind of stuff that they are trying to change (as usual for systemd, without fully thinking why people do it differently now).
Of course, another part is that this is an old idea, already implemented once in the commercial Unices a quarter of a century ago. They merged /bin into /usr/bin, with a symbolic link from the former to the latter.
It seems to be working pretty good, actually, with compartmentalization and tight security.
Meaning Java, which makes POSIX irrelevant.
Tight security? Well, how about Stagefright? Cyanogen no longer supports my phone. Even when they did, it wasn't well-supported: each new release had a high chance of breaking something fundamental. Right now I'm stuck on a "milestone" release that isn't even the newest one (it's over a year old now!), because later ones all have some unacceptable problem, like GPS not working or cell data not working. I can't just install an updated browser component package, because Android is monolithic. On Debian it'd be in a package like libwebkit, and it'd be a simple "apt-get upgrade" away.
So I'm stuck with a vulnerable Android version. If I try to upgrade to fix it, I'll lose GPS or cellular data. The only way to get a fix is to buy a new phone.
Now Google is trying to push more of the components into its Google Play Service app/framework. So, great: more parts can be upgraded independently of the monolithic core. But that also means that Google can push out updates to more components without my knowledge or consent: not great. That means more instances of feeling something warm in my pocket, pulling out my phone, and finding that Google Play Services has kept my phone awake for 10 hours and my battery is at 12%.
This has happened to me on multiple occasions: one day my phone works fine. The next day it eats the battery like candy. I didn't change any settings or install any updates--Google merely decided to push out an automatic update to its APKs without my knowledge or consent, and Google Play Services constantly crashes every 5 seconds, popping up a dialog box that I have to manually dismiss. I can't even do a simple task like send an SMS or check my email, because every 5 seconds it pops up another "Unfortunately..." box. There's no practical way to roll it back, nor even to report the problem. The best I can do is find people reporting the same problem on Reddit and hope that Google gets its act together so I can actually have a usable phone again; maybe one morning I'll wake up and my phone will not be "sick" anymore. In the meantime I have a really dumb "smartphone" that keeps my leg warm.
I have had high hopes for Android, but I now consider it to be a mess, and consider Google to be a highly irresponsible steward. I don't want an iPhone, but if someone wants a reliable smartphone without nasty surprises, they are probably better served with one.
Similar problems happen with Debian as well. E.g. hibernation breaks or WiFi stops working.
Sure, I could use Android without Google Play Services. That would also mean not being able to use the vast majority of apps in the Play Store--and I don't mean games. That would mean not being able to use, e.g. Google Maps.
So to use Google Maps, I should have to give Google free rein to push updates to its software to my phone whenever it wants, including untested updates that cause breakage, crashes, and battery draining causing high temperatures?
This is what's wrong with Android.
And you can't seriously claim that such drastic breakage happens on a regular basis with Debian Stable.
I would also like to have more control over what Google does in my Android phone while not losing the ability to install apps easily.
In addition to the problems you describe with google moving more core functionality into its "Google Apps" packages, it also means that running android without the google bits is getting increasingly hard. It's missing some core functionality that other non-google apps expect.
Public build service is https://openrepos.net/ and it already has lots of packages that you know from a normal Linux desktop and you can also host your custom stuff there. High-level package manager is packagekit or zypper.
Having a private repo is easy, too. Package format is RPM, repo format is http://createrepo.baseurl.org/
Given the thin APIs exposed in the NDK, Google could change it by any UNIX flavored kernel and no one would notice.
The OEMs might be a bit pissed off if they had to change the APIs, but that's all.
Plus there are a handful of tools with smart option that take advantage of the multiple partition approach, like rm and rsync's --one-file-system option.
[root@mail /]# ls -laF /etc/init
ls: cannot access /etc/init: No such file or directory
If you want to have a separate read-only partition for system programs (and require going into single-user mode to modify that), go for it. But the rest of the /bin /etc /usr /usr/local nonsense is just because it's easier than changing to something better for most people.
The thinking behind the file system layout had nothing to do with the available disk space or having a read-only file system. It was, originally, entirely a hedge against corruption; before journaling file systems and all that jazz, if your system was hit by a power outage and it was in the middle of some operation or another it might corrupt whatever file system it was working in.
So, you would create multiple partitions and have different directories as the mount point for each partition. Say for instance that, as a cautious sysadmin, you put /sbin on its own partition. Excellent! Since /sbin rarely gets touched, if everything goes pear-shaped, the system will still boot into single user mode and you can use tools like fsck and mount, which are under the uncorrupted /sbin partition and are statically linked. If you couldn't for some reason recover the file system on a particular partition, then you might be looking at having to load, say, /etc from backups, instead of everything under /.
Over time, other software started to take advantage of multiple partition layouts to protect you from getting hurt. You could mark some partitions noexec, nosuid, or restrict operations like rm or rsync to specific partitions without having to worry about whether or not they'll crawl out of that partition and eat the world.
But that's pretty old-school now. As file systems improved and corruption was less of a problem and everybody sort of gave up caring about what options rm accepted, more and more systems just mounted everything under / so that you would be maximally boned if ever there was a filesystem problem. Progress!
OpenBSD is one of few operating systems left that has a set of recommendations for how to partition your system: http://www.openbsd.org/faq/faq4.html#Partitioning
So that's why I'm not a fan of "novel file system layouts"; the previous layout was designed and existed for some good reasons and I have a hunch that the people that are happy about novel file system layouts don't know about those reasons.
If my filesystem gets corrupted, I boot into a live cd, just like everybody does.
With btrfs and integrated parity, checksums, cow and scrubbing on the horizon filesystem reliability is only a minor thing to care about. With subvolumes we can snapshot as many things atomically as we want.
Simlicity is now what trumps your arguments overwhelmingly. The filesystem hierarchy has to be intuitive, consistent and simple, which it is currently not.
Booting a LiveCD would not have fixed affected partitions.
You're not the first on HN to say in so many words that the sometimes expensive lessons I've learned over 30 years of computing are suddenly irrelevant now. I used to argue that, but I don't see much point to it anymore.
Another argument was more fine grained permission control. You can do that by other means, for example with subvolumes.
So, after all, possible corruptions are not alleviated by complex partition struture, or only with a bad trade-off for complexity. Instead, do backups. Easy and reliable. Everbody is happy, without many partitions.
Simplicity is crucial and it's one thing the free software ecosystem is consistently getting wrong. I see it as the major showstopper for many people to use it more intensively.
The filesystem hierarchy is just one culprit, but a fundamental one.
Take a look at at some fundamental pure mathematics lectures or books. You can immediately see that this was designed to be as easy as possible. Software must be the same.
Basically that exact capability is why people adopted Linux in the first place to run the servers that run the net.
I don't see how having a / that can get up and running even when the larger system housed in /usr and such is down for the count, is anything more complex that the piles of VMs and containers that is being advocated today.
Don’t misunderstand simplicity for triviality for they are not the same. We need to make non-trivial problems simple.
Linux desktop penetration: ~1.33% (http://www.netmarketshare.com/operating-system-market-share....)
1.3% * 136mm = 1.77mm
Let's ignore Amazon, Google, Microsoft, Facebook, and so on, each of whom are by themselves rumored or reported to have over a million computers in their DCs. Let's further ignore the long tail of the second tier of providers, who are thousands in number and have merely hundreds of thousands of computers each in their DCs. Let's go on to ignore all of the self serve corporations in the hundreds of thousands of data centers across the world, running Linux or a derivative on thousands to hundreds of computers each.
Let's also ignore all the routers, switches, file servers, and miscellaneous other networking equipment running Linux or a derivative.
Let's further ignore all of the televisions, media players, microwaves, appliances, and other consumer goods running Linux or a derivative.
Let's look only at Android phones, which run an embedded derivative of Linux. The phone market shipped 341.5 million year over year in 2015 Q2 (http://www.idc.com/prodserv/smartphone-os-market-share.jsp). Android had an 82.8% marketshare, so that's 282.77mm.
The number of linux computers that have the ability to boot "a live cd" is 6 thousandths (.6%) of one segment of one part of the linux embedded and server market.
Linux on the desktop has not only been dead for the last 18 years, it's barely a rounding error on a single segment of a single part of the embedded and server market.
By the way, "the complexity of the default" has been fine for over 30 years, and it's still fine. Just because you don't understand it, doesn't mean it's not the right answer.
And that is what makes someone take the road trip to the server farm at 2am because a ge-wiz dev thought the exact same and never considered a "rescue" mode with network capability.
I'm 100% certain I remember the instructions for partitioning being related to file system corruption, but that may well have come long after the original design of the FSH.
I was able to recover the machine with a LiveCD but still, it was annoying.
Daniel J. Bernstein, around the turn of the century, proposed a /package hierarchy (for package management without the need for conflict resolution) and a /command directory. (http://cr.yp.to/slashpackage.html) Their major problem was the idea that one had to register things with what was to most of the world just some bloke on another continent. But they had concepts like a hierarchical package naming scheme ("admin/", "mail/", &c.), self-contained build trees with versioned names, and an "index" directory full of symbolic links.
then people that can't work together with others (eg Jobs) does retarded things like /localLibrary which benefit nobody, not even themselves. just throw everything up. then if those folks succeed despite that, it becomes the new immutable thing to follow, like gnome is doing with following blindly apple ui specs.
edit: i never tried to contribute to any larger standard. i did develop against most of them, cursed at their anachronistic ways and coded around it praising consistency. so i am pretty sure i am part of the problem described above. I'm not saying the fine people in those ivory towers are the problem. I'm glad we have that little help for world consistency. it's just not enough.
where it came from originally is not much of concern. but the fact that it is the best we have, is.
But sadly it doesn't present the nice object concept NeXTSTEP and OS X have.
However isn't the problem that we need everyone to agree to do this? Additionally it is nice to have all config in /etc
Heck, it originated as one guy's solution for managing self-compiled programs within his home dir on the university mainframe (or some such).
You can still find Rootless (the bootstrap tool for using Gobolinux in the home directory) on the Gobolinux site, but i can't comment on how well maintained it is these days.
What about OpenSSH does that mean any program that uses SSH needs its own copy and then we have 12 different versions of SSH?
Its not easy but I am sure someone is smart enough to figure this stuff out. NixOS and Gobo Linux have tried.
These are packages that other packages can refer to rather than massively duplicating the content within themselves.
These frameworks will in turn be versioned, so that if package X wants framework Z, while Y wants Z+1 they can both be happy without descending into dependency hell.
Note that Gobolinux gets around this to a degree by using age old soname and symlinks. This in that ld and similar is pointed at a directory containing symlinks.
libxyz.so.1 will be pointing to whatever is the active version of libxyz, while something like libxyz.so.1.1 will be pointing to that specific lib version.
Disabling a specific lib version is done by removing the relevant symlinks.
Gobolinux provides tools for doing this, but they are basically shell scripts. And so in a pinch someone can do all that manually.
At that point the shared libraries are useless, they don't provide security (update in one place and have all aps using it patched), doesn't save RAM (each app loads their own version of library, instead of multiple apps sharing the same code), and you get worse performance (there's overhead for using shared libraries).
What lsaferite suggested makes sense, if the library is used only by given app, why not compile it statically? You have everything in a single file, and at least you're not losing performance by using shared objects.
That said, once compiled a binary package can be produced by simply creating a compressed archive of the content of /Programs/app-name/app-version (an official binary package is a bit more involved to ensure compatibility with the latest released iso).
Beyond that i am not quite sure what you are asking.
This so that it is less likely that anything with hardcoded paths will throw a hissy fit.
Frankly Gobo demonstrate the massive flexibility of classic unix tools, as most of it is shell script and symlinks. There is perhaps a few binaries, purely for performance (and those are third iteration, the first being shell script, the second Python).
The idea of one package for all distros is cute but not at all realistic. And it isn't pleasant for the end user, who wants everything in their native package manager and things to be configured in the same ways as everything else.
LSB was a well-meaning idea, but it doesn't actually do anything to improve openness or interoperability.
A project I was familiar with some years ago used essentially a NFS mount containing a subset of a 4 year old "enterprise distro" and then built with GCC --sysroot plus some vars like PATH, PKG_CONFIG_PATH etc set appropriately.
One of the problems is, beyond the few things listed in LSB (so it's not entirely useless!) it's not at all obvious what libraries you can expect to be on the system in an ABI compatible way - if you want to use something as unstable as OpenSSL you have to bundle it.
I also found that some scripts had the hashbang line as /bin/env which works on RHEL, but not on SuSE.
I have also been trying to write some scripts that work across multiple distros. Detecting distros is wierd. "lsb-release" is not present in most distros. "/etc/issue.net" is present in some, but not on ArchLinux.
No wonder ISVs find it hard to support Linux. I suspect the technical side of this problem is probably trivial, the leadership side, not so much.
'lsb-release' was always hilarious: the one thing about Linux that is almost standard is that the LSB tools are never present.
And why?
#!/usr/bin/env python
#!/usr/bin/env python2
#!/usr/bin/env python3
There are, as you note, issues with the locations or presence of certain files, directories, and scripts. That's one part of the problem, but probably the easier one to solve (we had a whole big function that would try to detect how to open a URL by checking for xdg-open and older variants, checking configuration for KDE/GNOME/etc., and then falling back on detected browsers). This also has to be done for opening arbitrary files using their assigned program.
What's more complicated are library dependencies. Depending on system installs on GTK+ isn't always enough. You may need features not available on that install, due to version issues. You may have to work around versions of system libraries that have a bug present on one particular distro due to a custom patch.
Themes and icon sets are a problem if you hit any version compatibility problems. Your UI can appear broken if you reference icons that aren't registered on that system (generally an issue on older installs). This actually gets complicated quick in some cases.
So as an ISV, you have a couple of options for how you approach all this. You can limit the distros and versions you support, putting out builds for each. This is a perfectly fine strategy if you can get away with it, but not everyone's going to be able to use your program in that case.
Another option, which is what we did, was to basically ship everything we needed along with the program. Libraries (glibc, gtk, x11, everything we were forced to ship for compatibility), icon sets, themes, etc. You then try to dynamically figure out which modules you can load from the system, and which you need to load from shipped modules. You also need to handle setting up paths for anything like icon sets and fallback themes you may need, and deal with the compatibility issues that can occur from that (older versions of the XDG icon theme spec didn't have a properly-functioning fallback system when specifying multiple icon theme directories).
If you ever need to do anything more specialized (support for hardware devices, anything using D-BUS on older installs, more specialized window management abilities), you run into a whole new set of issues.
These days I focus almost entirely on web-based software, so that sort of wide range of compatibility problems are entirely behind me. (Ahem.)
I do the same when packaging upstream software. Sometimes there are build problems due to configure(.ac) not checking for dependencies properly. I'll then open a bugreport that these checks should be added. All that small stuff helps quite a bit.
The problem is, even if we filed a bug report with a patch for a 3.0 release of some distro, and it was fixed in 3.1, we still had to support 3.0. That meant code that worked with both the buggy 3.0, and the fixed 3.1. That's not a big deal when isolated to one distro or one component, but we're talking all the major distros, various third-party libraries, etc.
We may have been a bit nuts, trying to keep such a wide variety of environments supported, but that said, we also got really good at it, and it meant we didn't have to force customers to make changes to their system that they didn't want to make. The products ran on distros so old that they still only shipped GTK 1.2 (though we eventually finally walled off some of the older ones when we started supporting only 64-bit processors).
It opens text files in Emacs here – did you configure your system to associate the text/plain mimetype to Firefox? (The following mostly for anybody who is interested):
$ xdg-mime query filetype <file>
$ xdg-mime query default <mimetype>
$ xdg-mime default <.desktop> <mimetype> [<mimetype>*]
Whether or not the apparent distribution default of opening text files with Firefox makes sense is an issue somewhat orthogonal to whether xdg-open works – in that particular case it may have been misconfigured to open files with Firefox, but that doesn’t mean it a) didn’t open the file (it did) nor b) in a completely unreasonable way (the file was still displayed and such).
If none of those are found, xdg-mime will be used. Relying on xdg-open to work sanely across desktops, much less across distributions, is… optimistic at best.
Try /etc/os-release first (http://www.freedesktop.org/software/systemd/man/os-release.h...). It should be present in all modern distros (even if they don't use systemd).
In theory LSB meant a vendor could ship an RPM that would work flawlessly on any Linux distro supporting that LSB release (which on Debian meant supporting RPM alongside DEB as a package format). In practice vendors would just target RHEL and/or Ubuntu and call it a day.
https://wiki.gnome.org/Projects/SandboxedApps
It's already in a working state - you can try running GIMP nightlies
One of the many reasons that the best standards are reflective of what most people do than prescriptive of what they should do.
I remember back in the day, LSB was announced as a means to avoid the return of UNIX wars in GNU/Linux.
Here we are with each distribution having its own way how to do things, so now we do have GNU/Linux wars, with a few selected distributions just like the old commercial UNIXes, because no one can bare to support X thousand variants.
https://www.linuxbase.org/lsb-cert/productdir.php?by_lsb
As you can see it has the big name distributions on it, except for Debian and Gentoo, plus many obscure ones.
However, the list of LSB certified applications is rather remarkably short, which suggests that the real problem is that ISVs are much less interested in LSB than one would expect.
I think this strategy worked remarkably well for HTML5. Standards is a two-way conversation with the community.
http://pkgs.org/search/xdg-app
Apparently only available on Fedora, openSUSE and Mageia at the moment.
First systemd pretty much ruined linux, I know so many people who lost interest in promoting or using linux what that was forced on us.
Now they seem to want to change this.
Yeah.... I'll just go back to windows.
I don't remember Windows Mobile having any relevance.
Symbian was pretty much alone at the throne for years before iOS and Android.
As an easy example, multiprocessing has become more important between now and then. Why are things like processor affinities still not defined?
In the case of processor affinity, it looks to me like no-one has ever suggested it. Participating in the Austin Group is pretty open - why don't you put a proposal forward? (Maybe affinity can be added to struct sched_param somehow?)
Not really. I know someone who's spent ages trying to get O_CREATE added as an alias for O_CREAT.
http://spiegelmock.com/2015/01/23/fixing-a-typo-the-hard-way...
It is only good for CLI applications and daemons.
EDIT: For the downvoters what are the POSIX APIs for:
- Graphics (besides the clunky CDE and OpenGL isn't POSIX)
- Thread/Task pools
- Bluetooth, NFC, GPS, fingerprint readers, ....
- Application containers
Currently it mostly covers hardware from PDP-11 systems still, anything else via /dev, ioctl, whatever, is vendor specific.
Last time I checked that weren't any portable API for USB for example.
So what was left out of ANSI C, is what became POSIX. Making any OS with ANSI C + POSIX compliant compatible with the original UNIX systems.
But unlike other programming languages, on C's case the evolution in terms of standard libraries stopped there.
I guess portable interfaces may just end up needing to be defined over non-portable interfaces, and C may end up not being practical for anything that needs to be portable.
Higher level languages aren't bad, and it would probably be a good thing if a language like Rust became the de-facto for low-level. My feeling is that C is still the simplest to compile though, which I think is an advantage.
But that is a talk for other day.
Aliasing is a problem, but I don't think it's strictly inherent to C.
Nowadays yes, you are forgetting C has 40 years of research into compiler optimization.
We could easily beat the register allocator in the 90's with Assembly coding, specially in x86.
There is a quote from a famous woman in compiler research from the mid-80's, where she claims C semantics have thrown the optimization research back to the dinosaur age.
Sadly I cannot find it, as I don't remember the exact words.
> Aliasing is a problem, but I don't think it's strictly inherent to C.
Most languages have stricter semantics for aliasing and do not decay arrays into pointers.
Also they tend to have better type information, which can help the optimizer.
A final single constants file could be handed back to the app Saying "CONFIGDIR=/foo/bar/bad"
Symlink everything?
Funny Linux guys.
systemd as base system will certainly happen, regardless.
My machines start much faster with systemd too, which is a nice side effect.
That's also when I learned systemd was running an entire virtual fstab layer. Because, you know, /etc/fstab just isn't good enough anymore.
I would file a bug on systemd mentioning this behavior. But I like the fact that that my usb wifi card will not screw my system.
What was intelligent about that?
I don't need intelligent tools, I need tools that do what I say! That's why they're tools, damn it!
Do you prefer sysvrc over -say- OpenRC, Upstart, runit, etc, etc?
I'm not sure how you could, with a straight face, claim that sysvinit fits this description. It does many things, none of them particularly well.
The only thing this accomplishes is ensuring no progress is made, ever.
It's a red herring, anyway. People have been replacing classic BSD and SysV initialization schemes since 1992 [1], but systemd was the first to penetrate the scene quite massively for a plethora of reasons that no one can seem to agree on.
Honestly that's unfair to XP/Windows Server 2003 which had much better supervisor capability than sysvinit provides.
That's really a shame. Systemd is almost comically bad.
I've seen a lot of political stuff, and a lot of claims that sysvinit ain't broke so we shouldn't fix it (which makes little sense to me), but haven't seen a good resource discussing technical merits, or the lack thereof.
I could probably do it, but in all honesty, the "debate" has been so spoiled that I doubt anyone would take it seriously. My relatively technical article about FreeBSD and launchd (which had only one error about where plists were parsed, a relatively minor gripe in the text) was near-universally misunderstood and derailed by commentators who went on rants tangential to what I actually discussed. Same with my init history article.
But, if enough people want to, maybe I'll bite the bullet. Not sure if I'd like to subject myself to it, but I think I might end up writing a rather long treatise if there's demand and I do go with it.
There are other pages out there claiming to be neutral overviews, but they generally just take a neutral tone while rehashing the same long-debunked myths and spreading FUD.
The specific thing I like about systemd is the init replacement. It's just so much better! Automatically tracking daemon child processes, even detached ones. Supporting all the different modes daemons work, so I don't need to write yet another damn clean-up-all-the-children function in a SysVinit script. It's wonderful.
But systemd is a lot more than an init system and I don't really have strongly formed opinions on those.
* http://blog.darknedgy.net/technology/2015/09/05/0/
* http://uselessd.darknedgy.net/ProSystemdAntiSystemd/
Another persons writings to read, for similar reasons, is Ian Sutton, who is the bloke who did SystemBSD:
You can't just go "attack surface" and have said anything useful, nor can you point at volume of code if that code is actually needed. All criticisms of systemd usually depend on people completely ignoring the very serious problems you can't solve with sysvinit and which do make actual, practical systems less secure (i.e. cgroup process isolation is amazing, and a lack of real service management is a huge problem and any service management is going to be running as root period. Yet there's been a real claim that service management is "what your sysadmin does". Why even bother with computing then?)
Here's an example:
http://lkml.iu.edu//hypermail/linux/kernel/1404.0/01331.html
Literally everyone with a clue in the matter seems to hate systemd with passion[1].
A long-term Debian Maintainer quit over the issue[2].
Linus himself (as you quoted) seems more than a little annoyed by systemd and its developer.
And yet somehow systemd made its way into all major distros...
Linus called out on some behaviour. That is not him saying that people should stop developing. It's him saying stop this behaviour.
Christopher Barry is most certainly not a troll.
As shown in the LKML and Debian debates, the systemd infestation was firmly opposed by nearly everyone outside the 'vested interest' RedHat filter bubble. And still is.
Some high profile names would be Eric S. Raymond or Patrick Volkerding (creator of Slackware).
Debian has been forked over this; https://en.wikipedia.org/wiki/Devuan
That is not him saying that people should stop developing. It's him saying stop this behaviour.
It's him finally calling Kay Sievers out for being incompetent and an asshole. Which was long overdue.
Sadly it's by far too late and will very likely not change a thing about systemd's "break and extinguish" strategy.
Systemd will continue to break other peoples projects in its effort to replace every mature service implementation with a hacked up systemd imitation.
At least until, hopefully, the major distros finally acknowledge their mistake and roll this disaster back.
1. It was not a good bug report. 'Do not parse "debug" command line parameter' is not a bug. The actual bug is "assertion in systemd-journald" which is a clear regression, but independent of kernel parameters. According to comments 17 and 22, it was promptly fixed.
2. According to projects process, discussions about design are to be done on the mailinglist and not in the bug tracker. Most commenters violate this.
3. No systemd dev explained why they parse the debug parameter.
4. Sievers closed the bug report without explicitly acknowledging the assertion problem.
5. The kernel devs (especially Linus) are too hot headed and misread this. Systemd caused and fixed (!) the assertion problem.
On the other hand, developers seem to enjoy it quite a bit.
That's what happens when you have developers developing for themselves, not for sys admins (the end users of their work).
Process management is built in and sophisticated (restarts, timeouts, conditions, limits etc.).
The dependency system is amazing for ensuring your system fails sensibly: I can link mountpoints to processes, and easily guarantee that a mail daemon only runs when its network mount is ready, and then make it also shutdown if that mount goes away for any reason. This is huge for self-documenting infrastructure - you can ensure that even manual intervention in the system won't leave it in a weird state, and the why is all right there.
Taking a conversation that started in public offline when there is an expressed interest in this very thread just doesn't follow.
Is it so hard to avoid change for change's sake? I'm not opposed to learning new things, but we're all really busy, and when the standard way of accomplishing things changes in every major release (which you're usually trying to deploy to stay on top of the latest developer demand, or audit requirements or whatever) just adds a massive learning curve. Now I've got to learn something new, rewrite a bunch of tools, all because.. why?
I don't expect it to change again in the next 10-15 years, except potentially to a competitor that is designed very similarly but just plain better.
I would like to see more empathy for the fact that systemd is a complex, underdocumented system. Lennart's blog series is great for the "why," but very sparse on "how" and especially "how do I fix it".
As a developer I go through that "new tech" cycle every 2-3 yrs. Why? 'couse world's evolving. Some evolution cycles are not necessarily good, but we have to steer the ship forward. It's not always nice, but I don't see the other way...
It took a while to get used to, but it does what I need very well (dependency resolution, process supervision, etc).
The arguments against it appear to be philosophical, not practical, and at no time have I encountered any defect in it. After a brief learning period I came to fully embrace it, and in actual use it makes my life easier in several minor but appreciated ways.
It's understandable that people become used to convention, and are resistant to change. However, most complaints about systemd seem to be appeals to tradition.
This is a comment I was supposed to send to chris_wot in the systemd v227 announcement thread, but his posts got nuked by dang.
Here goes:
----------------------
systemd tries to force various system resources into the Unit abstraction, to a rather mixed effect. Some are IPC resources (sockets), others are abstract entities which are driven by a lower-level primitive (services that drive jobs), yet others still claim to wrap resources but really drive util-linux binaries (mounts and swaps). The information hiding/encapsulation is inconsistent, too. I'm not shielded from the dirty details of transactions and the seven different job queuing modes, and when they actually blow up in my face, the debugging context is very scarce.
All of this takes place in an ad-hoc graph engine that I cannot really observe. I can look at approximations of it (systemd-analyze), but I have no way of directly manipulating it, only indirectly through normal operations on system and service state of which the effects I cannot reliably gauge as they undergo systemd's heuristics. Moreover, I cannot checkpoint the graph state to disk, so I could e.g. replay and reboot from a known-good system state manifest. systemd snapshots are ephemeral (in-memory) and moreover do not fully preserve the graph info, anyway.
systemd clearly has an identity crisis between whether it wants to be an eager or lazy service manager. Publicly, it recommends laziness through so-called socket activation, but in practice people mostly tend toward using the dependency system. The dependency system, of course, exists mostly to calculate ordering requirements for parallel startup. This however comes at a cost of non-deterministic ordering, whereby a service can be malignantly reordered, breaking the boot sequence (NFS seems especially prone). This is somewhat inherent to parallelism in general, but is particularly undesirable given parallelism serves no other purpose beyond startup speed optimization (for which process checkpointing I maintain is a better solution).
Between all of this intermediate complexity, I can end up in stuck start/stop jobs, cyclical transactions and dependency loops without knowing the exact origin, because the graph engine doesn't supply me with sufficiently good debugging context.
And the worst part is that most of this is not even a reliability/fault tolerance thing. The Unix process model has no semblance of anything resembling a "transaction", nor is the Unit object versatile enough. It's all heuristics on a private graph engine meant to topologically sort the edges, so as to accommodate systemd's dependency system (which conflates hard service relationships with softer ordering requirements, and is generally inflexible)... yet at the same time you're actually expected to lazy load everything, but barely so in the real world.
Oh, and of course, handling anything related to Unit objects in PID1 creates an unnecessary coupling between global system state and service state in particular, on top of all the intermediate complexity of the graph.
P.S. Interesting insights though, you make a few fair points. When I'm feeling better I might ponder some more on them. Peace.
Seems adding something to record how it started things and why it did so would help quite a bit. There's some debug mode where it limits how much it starts at once. Then some way to visualize this easily (not just a GUI/picture).
Whoa, please don't drop things like that casually in the threads. People are quick to believe them, but what you said is false. I didn't touch chris_wot's comments.
We never delete comments outright unless the author asks us to, and only kill individual comments (i.e. mark them [dead]) if we are banning a user for breaking the site guidelines, which was not the case here.
Overall I'll hire the sysadmin who knows the most init systems still found in common linux and unix systems and is the most ready to adapt to whichever trend wins domination. Evolve or die.
Can't have looked very hard, then. There's dozens of us. Dozens!
"Ok" with sysvinit, sort-of "like" systemd and "hate" supervisor.
Btw: almost nobody considers the point about supervisor. A lot of clients' systems I manage are sysvinit or upstart systems, which use the init system to start supervisord and then use supervisor to (poorly) manage some services.
So even if I get why someone might like sysvinit better than systemd, I have no idea how someone can like sysvinit+supervisor better than systemd.
Here's another one: I spent 18 mos in Google Site Reliability Engineering (and did a lot of pager work at startups before that) prior to transferring to software engineering, I think I can speak to both perspectives.
Systemd is incredible and is a huge advance in maintaining linux systems. It boggles my mind that anyone could compare it to what we were all doing before and walk away saying: "nah, I'll stick with the stringly-typed shell scripts and the stale pid files, thanks."
Specific technical reasons in favor /for sysadmins/:
- unit files are rigidly structured data and this is a Good Thing. Configuration is not code. Unit files are also very easy to write.
- systemd understands cgroups and manages them properly. Every single critique I've seen of systemd is from someone who either doesn't understand cgroups or doesn't realize why they are so useful and important. Cgroups manage jailing, process isolation, resource control (CPU shares, memory limits, block io shares, etc), privilege separation and dropping, etc., and this is all configurable by systemd using simple and declarative unit file directives. Like it or not, we are moving into a containerized world.
- machinectl is already useful for managing VMs and will only become more so as it matures.
- people shit on the systemd journal, but, I'm sorry -- you're wrong about this too. Structured binary logging is simply more efficient for this problem, both on the write and on the query side. The journal also can integrate with the systemd journals running in any VMs or containers running on your system, so you can get a unified and metadata-tagged logging stream of everything happening on the host
- socket activation enables zero-downtime restart/deployment for network services, any incoming connections to the port will be added to the listen queue by systemd before the target system comes back online