Show HN: Interactive systemd – a better way to work with systemd units | Hacker News Reader

Show HN: Interactive systemd – a better way to work with systemd units

isd-project.github.io

546 pointskai-tub204 comments

I created a TUI for systemd/systemctl called isd (interactive systemd).

It provides a fuzzy search for units, auto-refreshing previews, smart sudo handling, and a fully customizable, keyboard-focused interface for power users and newcomers alike.

It is a more powerful (but heavier) version of sysz, which was the inspiration for the project.

This should be a huge timesaver for anybody who frequently interacts with or edits systemd units/services. And if not, please let me know why! :)

Open articleView on HN

Looks nice.

One thing I found systemd really confusing was its treatment of ExecStop in a service script. ExecStart is the command to run when systemd starts the service at system boot up (or when a user tells systemd to start the service). However, ExecStop is run when the starting command has finished running. You have to set RemainAfterExit=yes to have the desired function of running the stop command on system shutdown or on user stopping the service. ExecStop is basically the "on-cleanup" event rather than "to-shutdown-the-service" event.

I think about them as “on start” and “on stop”.

It is important to keep in mind that systemd is tailored towards daemons. So if your service just runs a command that eventually exits, you need to explicitly tell systemd to treat it differently than a daemon.

Edit: As others noted, you’re probably looking for oneshot + RemainAfterExit.

It is a little asymmetric, because 'ExecStart' is actually normally 'Executable that is the service', not just script that starts the service, but I think that's a hangover from the self-daemonizing approach to init scripts.

True, but it still makes sense to reason about them this way. Say you have an HTTP server:

- on start: start the server

- on stop: do nothing, because you are already terminating the server

But suppose you need to perform an additional task when the server is terminated. That is where you would add a ExecStop command or script.

Then ExecStop is basically on-cleanup, not to-stop. ExecStart really is to-start, not on-start. In the httpd server case, ExecStart runs the httpd command.

If that helps you understand it better, then sure.

All I am trying to say is that the name of the option makes sense as-is: it literally runs the provided command(s) on service stop (“execute on stop”).

Similarly, ExecStart literally means command(s) to execute on start.

If the command runs a blocking daemon/server (usually the case), then the server will be implicitly stopped by definition - because if you’re stopping the systemd service, you’re interrupting any commands that are still running/blocking.

I think part of the problem is with how one "naturally" or "habitually" thinks of a Service. From systemd's perspective/terminology, the Service is the thing that starts and stops. But whether because it is inherently more intuitive, or because of how daemons traditionally worked on *nix, the mind tends to think of the process which the Service starts as the thing which starts and stops. I'm not able to double check currently, but I also think that systemd isn't totally consistent with the mental model it bases the choice of keywords on when it comes to ExecStop, because with that mental model one would expect ExecStop to only run when the Service is stopped from systemd, but I'm fairly sure it runs in other cases when the process started with ExecStart exits, but the service isn't necessarily stopped. I could be wrong about that though.

ExecStop works the way you want for type=forking

Still has the same problem even with type=forking. Only way to get it working was RemainAfterExit=yes

I think you actually probably want type=oneshot (and also RemainAfterExit=yes) for the kind of service you're describing

This was ultimately what I needed to do when I wrote a systemd service that managed some firewall rules. It really was a footgun though, what with having essentially different meanings/purposes for ExecStop whether you’re doing a Type=forking, a Type=oneshot, or a Type=oneshot with RemainAfterExit=yes.

And relatedly, I honestly have no idea when I’d want to use ExecStartPre, or multiple ExecStarts, or ExecStartPost, and so on.

Having different semantics with different proprieties on the same command is really confusing.

I would argue the semantics of ExecStop are always the same. It's the command that's executed to stop the service. On the other hand, what it means for a service to be "running" or "stopping" naturally depends on what type of service it is (i.e., is it a daemon or not?)

> the command that's executed to stop the service

That’s what is assumed. But in reality it runs after the started process stops.

Yes, so whether the service is stopping as a result of the process exiting, or whether you requested the service to stop manually, it will run the ExecStop in either case.

That makes sense to me personally. What would be the more intuitive design in your mind?

Stopping as a result of the process exiting or requested the service to stop are two very different things. Systemd overloads the term ExecStop for different semantics, relying on different property settings. That's where the confusion comes from.

The name sounds like it means "this is how I want you to cause the service to stop" to me (and clearly to others as well). That would be symmetrical with ExecStart meaning "this is how I want you to cause the service to start". If it runs after the service stopped it should be called "ExecAfterStop" or something like that.

That is what ExecStop means. It specifies how you want to cause the service to stop. But the lifetime of the service isn't exactly the same thing as the lifetime of the process that got started in ExecStart.

Maybe think about it this way: ExecStart is what the system will run to transition the service from the "starting" state to the "started" state. ExecStop is what the system will run to transition the service from the "stopping" state to the "stopped" state.

For a service with RemainAfterExit=no (the default), you enter the stopping state right away once the processes that got started in ExecStart exit. That's useful when you are starting some long lived process as a service, and in that case there is usually no need for an ExecStop. But semantically, ExecStop has the same meaning either way -- it's what needs to be run, if anything, to transition the service from the stopping state to the stopped state.

I have now found the documentation for ExecStop (in systemd.service(5)), which hopefully improves my understanding.

It definitely seems to be both "cause to stop" and "after (unexpected) stop" in one. You can look at $MAINPID to see which case you have. This design apparently makes sense to you, but to me and several others in this thread a service that has already stopped isn't in need of being stopped and shouldn't execute commands intended for that. (There is a separate ExecStopPost for "after stopping, for any reason".)

Think of it as an enum, Type branches the logic.

    enum Service {
        Exec { ExecStart: String, ... }
        Forking { ExecStart: String, ... }
        OneShot { ExecStart: String, ... }
    }

You can argue that sometimes that ExecStart could be a different term, but it'd still end up being the same across multiple enum variants.

Yes. ExecStart works the same for all the cases. ExecStop works differently though. While ExecStart is the event to kick off the command for the service, ExecStop is not. The asymmetric semantics are where the confusion comes from.

It's been enlightening to me to read through some of the distro-provided .service files to see what can be done, with services I'm more of less familiar with.

Love this. I use raw CLI commands until it hurts, and have recently embraced tools like lazygit/lazydocker to get visibility into otherwise opaque system/tree states, and it’s been a huge level-up.

I have several user and system level services I manage, but debugging them is tedious. Your opening line that lists common commands and their pain points really resonated with me.

I’m on NixOS, so editing immutable unit files directly won’t work, but the service discovery, visibility, and management will be really helpful. Nice work!

I am also a NixOS user and this is exactly what motivated me to work on this project!

I am planning on adding some "guides" in the documentation but in short: You should check out `systemctl edit --runtime` for debugging units on NixOS. It makes debugging sooo much easier.

This looks very good, thanks for sharing! I maintain a similar project and working with the systemd/dbus APIs has been pretty painful; eager to try this and see what I can learn from it.

https://github.com/rgwood/systemctl-tui

This is incredible! I will use this a ton. Only thing missing is a deb package...

At long last, systemd-client: merry meet! Next step is such a TUI for non-Linux such as macOS, FreeBSD, Windows. For macOS I use LaunchControl.app but it isn't a TUI.

Just one thing: I had to do

  $ uv tool install git+https://github.com/isd-project/isd

instead of

  $ uv tool install https://github.com/isd-project/isd/isd@latest

and uvx wouldn't work at all, version: uv 0.5.21. That said, uv is way more quick than pip(x) so I just switched.

Yeah, silly me. It is fixed now, thanks for letting me know!

So cool that with uv it becomes so easy to install such tools.

What's missing in the install routine is uv installing this tool ignoring the Python dependency. My box has 3.10 and isd won't work with it. Fixed with `-p 3.13` option. May be worth mention in the docs.

Hey, thanks for mentioning this! I have included the flag in the docs.

> If you ever became frustrated while typing:

Hey, that's me ! (And I love systemd !)

I haven't installed it yet so quick question: can it connect to remote host ? I often use systemctl --host <hostname> status foo.service (status, timers, logs etc. )

Dang. I have never heard of `systemctl --host`. Sadly not. It is more or less a fancy wrapper around (the local) `systemctl`. But there is also an appimage that should make it (hopefully) easy to run it on remote servers.

Either way, feel free to open an issue and I will have a look at it.

> Dang. I have never heard of `systemctl --host`. Sadly not. It is more or less a fancy wrapper around (the local) `systemctl`.

Sounds like you could easily support it by letting users pass in $REMOTE_HOST and when you use your `systemctl` wrapper, add `$CMD --host=$REMOTE_HOST`, after that everything should work as before.

I will investigate it!

https://github.com/isd-project/isd/issues/6

I can not install this stuff on remote servers and docker images. I would like multiple backbends to execute commands and gather informations (local, ssh, docker).

It should be installable locally, and run commands on remote machine via ssh! And via 'docker exec' commands.

> If you have nix installed, this AppImage will not work! Either use the nix or uv installation instructions!

Is this really true? I understand why it does not work on NixOS (I tried just out of curiosity and it seems like it is unable to exec the host systemctl for some reason) but I don't think there's any reason it wouldn't work on other OSes that merely have Nix installed.

Interestingly though, on Nix v2.24.11, I can't use the provided Nix command either:

    $ nix run https://github.com/isd-project/isd
    error:
           … while fetching the input 'https://github.com/isd-project/isd'
    
           error: Failed to open archive (Unrecognized archive format)

Even if that did work (you could adjust it into a Git URL to make it work) it would probably not be ideal since Nix has a native GitHub fetcher that is more efficient. I think this should be the actual Nix command:

    nix run github:isd-project/isd

Anyway, this is cool. I actually wanted to make a similar thing using systemd's DBus API and Qt instead of a TUI and even started writing code for it, and if you wanted to I'm sure you'd find that the DBus API probably provides all of the functionality you would need (admittedly it is a lot easier to just call `systemd -H` than to implement your own SSH tunneling, though.) It kind of frustrates me that systemd and modern Linux in general is absolutely teeming with data and interfaces that could be exposed and make administering systems, especially desktop systems that were traditionally very inscrutable, much easier. e.g. in the past, how did you know what was going on when an xdg autostart app failed? Now with systemd running xdg autostart apps in some desktops, it would be really easy to provide a GUI that can show you the failed autostarts and even provide a GUI log viewer, and yet somehow, such a tool does not seem to exist, at least in the realm of things that are maintained and relatively feature-complete. Rather frustrating.

I am completely confused as to why this comment seems to be poorly received. Can someone explain?

That may be a bit of nix backlash, don't sweat it.

I agree Linux could use better system APIs than "put file here" and "run these commands" which are much more error prone than making calls to properly documented interfaces.

> Is this really true? I understand why it does not work on NixOS (I tried just out of curiosity and it seems like it is unable to exec the host systemctl for some reason) but I don't think there's any reason it wouldn't work on other OSes that merely have Nix installed.

Yeah, it get's complicated and I don't want to recommend it and explain the details. In short, I am creating the AppImage via nix. And the AppImage "mounts" (not overlays!) the AppImage's /nix/ directory. So calls from the TUI that would access /nix/ wouldn't go to the systems `/nix` directory, which leads to all kind of weird issues. For example, you could install your EDITOR via home-manager on Ubuntu. isd would start correctly because systemctl is "accessible" but if you open your EDITOR under `/nix` it wouldn't find it, which is super confusing as a user. -> So it is just easier to say to use the nix installation method if you are already using it :D

And sorry for the wrong docs, it is fixed now.

I also agree with your frustration. Personally, I would really enjoy working on such a tool but it wouldn't be an easy task, and who would support the work? This TUI had a manageable scope but it was still quite a bit of work. So I don't see myself investing too much into "higher-level interfaces".

PS: I have no idea why your post is ill-received :/

> In short, I am creating the AppImage via nix. And the AppImage "mounts" (not overlays!) the AppImage's /nix/ directory. So calls from the TUI that would access /nix/ wouldn't go to the systems `/nix` directory, which leads to all kind of weird issues.

That makes a lot of sense. There's not really much trivial that I think you can do to deal with that.

> PS: I have no idea why your post is ill-received :/

It's not really a big deal, I was just surprised and wondered if anyone who was compelled to vote down might come back to explain why.

Looks great, well done. It's a shame that it's needed at all. The vast majority of my interactions with systemd are trivial: (re)starting a service, looking at a log file to figure out what's wrong, and making sure a service starts on boot. I find it baffling that the ergonomics of systemd for those common tasks are so lacking. But the TUI seems to help, so thanks.

And sure, systemd it's more deterministic and includes the kitchen sink, unlike initd.

Thankfully these days I can automate most of such interactions out of existence, so I no longer feel the burning hatred that I once did. More like a smoldering ember.

I mean, I get why they don't want to "bloat" systemd with a complex/opinionated TUI, but I would've also really liked a more "upstreamed" interface. Though, I guess making systemd more beginner-friendly/accessible is not really that important for their funding?

How's security handled? Not in terms of system permissions which Linux handles well, but in terms of guarantees that it can't be hijacked and remotely controlled by an external attacker.

Author here: I also find this an important thing to ask yourself when you are running applications/scripts that do anything with sudo and which is why I have written a fairly in-depth "Security" section on the isd documentation page:

https://isd-project.github.io/isd/security/

Let me know if anything is missing!

As a suggestion, since your repo is open I think you can leverage sonarcloud without costs. It would make for another independent check that your code does what's intended and for instance won't ping a remote control server, either via dependency attacks or via malicious pull requests that could dodge reviews.

Where I work we also use defectdojo to catalogue and manage CVRs in our projects, but it's more involved to setup the testing pipeline and deploy the required services.

On search and editing system unit files:

1. My life improved a lot after I found that you can do "systemctl status $PID" and systemd will find what service (if any) is responsible for the process in question. This has been a life saver many, many times. But, more search would still be welcome, especially for cases when the system fails to boot, or fails to reach a particular target etc.

2. I think systemd didn't go far enough with unit files. The motivation was to escape the hell of Shell scripts, where each system was defined in its own unique way, and was failing in a dozen of unique ways. While, initially, it might have seemed that a simple INI-style format could manage to describe service requirements... I think, it's way overdue to realize that it doesn't. And sysadmins on the ground "fix" that by embedding more Shell into these configuration files, bringing us back to the many unique ways a service will fail. Perhaps, having a way to edit these unit files so that it doesn't expose the actual format may lead to improvement in the format (more structure, more types, templates).

This looks almost as easy to use as slackware’s init/syslog back in the Linux 0.9x days.

If you add a sane cli with tab completion support, it’ll come full circle.

Maybe a bit of a gross simplification, but would you say this is analogous to something like k9s for Kubernetes? It looks handy, to say the least.

That was my first thought as well, seems to fill a very similar niche, just for systemd instead of kubernetes

I have never used or touched k9s nor Kubernetes, so... Maybe?

Haha this is great. It’s funny that others also end up doing this enable, start, journalctl, start dance. Good stuff. I shall try it.

My only problem is that I wish there were a way to install it on my machine and have it connect to a remote systemctl but that probably is a lot of work to reliably work (port may not be open etc etc).

Maybe it is possible with `systemctl --host`... Though, I would have to make quite a few changes to the UI design since it would mean that there are "multiple modes". One for local and multiple remotes. So... Check back in a month or so :D

Maybe you can keep it uniform by always presuming "host" as one of the parameters (and localhost just happens to be the default for it).

Nice! How did you find working with the Textual library?

It was great! The extensive documentation and the vast amount of widgets were definitely the main reasons why I have chosen to do it in Python/Textual rather than other languages/libraries.

Though I don't agree with all design decisions, and there are some bugs, having such well-maintained documentation and clean repo makes it easy to dive into the project and understand what is going on and how other components are built.

Thanks for the library!

Looks super cool, I've been working quite a bit with systemd recently, and typing systemctl and journalctl + their flags gets old rather fast.

Can it connect to remote hosts like you can with systemctl --host?

This looks super well done and polished. The rich docs are crazy detailed for something that was just released. You're really setting the bar for what other projects should do. Congrats!

Thanks!

I've put quite a lot of work into it and wanted the project to give a good first impression. This was important to me since I believe these "niche" TUI libraries need to immediately show what problems they are solving or how they save time, as users need a reason to _use_ an additional abstraction layer.

Absolutely! Really well done. I now think you're the best person to do the same for ffmpeg lol

Keep in mind, systemd is still a default-allow access control list (ACL) and has a very long road ahead in form of Linux security, as long as CAP_SYS_ADMIN continuea to exist.

May make it easier to customize but it doesn't close the security loopholea like SELinux, GRSecurity, TOMOYA, or AppArmor does.

https://www.cyberciti.biz/tips/selinux-vs-apparmor-vs-grsecu...

Yes yes and a reminder that BSD Jails are better than anything Linux does and a bunch of other dead horses we like beating on the regular around here.

Which is a fanciful way of saying that I don't understand the relevance of your comment at all to the topic at hand, which is an interactive frontend.

You read across as a security-indifferent.

Honestly it's not all or nothing. It really depends on your threat model. If you assume all native code is both not malicious and incapable of being compromised to run untrusted code then the existing model works. At that point it's more important to focus your security efforts on how native code ends up running, and therefore the distro specific packaging ecosystem.

Now for many that threat model is not sufficient as they both run increasingly less trustworthy software, obtained by less trustworthy mechanisms such as npm or off a website, or simply want to protect against bugs that cause otherwise non malicious software from being compromised and resulting in security incidents. I'm in this latter camp but we can't ignore the fact that there are many who happily operate in the former. There also exist solutions such as we browsers with their sandboxes and VMs that somewhat fill the requirements for running untrusted software for these individuals.

This is amazing, I had a bunch of work to do on some remote hosts this night and tried using isd as much as possible, to see how it can help me.

Some feedback :

- it is.. relatively slow ? especially when focusing on different panes (tab/shift+tab). on my machine it takes at least half a second to react

- the unit list is missing page-up/page-down handling

- in some unit attributes, the ordering of some values frequently changes (for example, on unstarted services, in `TriggeredBy`)

- it could be interesting to integrate the output of `systemd-analyse security`

Nice work !

- That the tab pane switching is the slow part is a bit weird. Is there a difference when you see all units and a subset of units? Maybe open an issue and I can provide more possible settings that could be changed.

- Jep, page up/down handling will be added. Thanks!

- Not quite sure when this happens. Maybe an example screen shot (before/after) could help?

- Yeah, that is definitely on the roadmap.

I've opened an issue about the reordered attribute values (+ another bug I found). Tab pane switching stopped being slow (I suppose my laptop was simply overloaded, and I'm too used to things being fast even when it is).

Need to see this in action. The example videos overuse the fuzzy search for commands in my opinion. Makes it hard to concentrate what the actual action/information is when a huge part of the screen changes. Not to say I don’t like fuzzy searchable command prompts. But it can be clearer to either just type out a command or have a menu system. As I said I need to check this out in action for myself.

The security section is good to see. Thanks for that!

Funny that you mention that. I honestly thought not too many people would care about it.

Though I am by far no security specialist. Please let me know where I can improve the section!

Isn't `service nginx status` a shorter way to do it? I've always done that instead of the longer systemctl commands.

Thank you for shipping this. Need to test it but going to add this to the deps bundle for my deployment service (relies heavily on systemd). Looks like it would make day to day maintenance/debugging far less of a headache.

Looks very useful. Standard tooling for systemd is such an annoying maze to navigate.

This would be extremely helpful. Although, I would love a feature where you can add different linux hosts and manage them all from a single place, using ssh

Most systemd functionality has built-in support for targeting remote hosts and local machines (containers, VMs) and mostly utilizes SSH under the hood to do so.

Check the -H and respective -M flags of many systemd CLI utilities.

I use systemd heavily. I'll definitely check this out!

Happy to hear!

Please open an issue on GitHub if you encounter any silly alpha issues.

Is there any way I can run a service before a given other service AND as late as possible (without rewriting the other service's unit file)?

You can override the other unit without rewriting. Place you config here.

> /etc/systemd/system/<unit>.service.d

I've used PartOf to enforce "dependency". Lots of other ways to enforce order and dependency once you have the override in place.

Why have you picked Python for this project? I feel like Rust/Go have better TUI ecosystems and the ability to use native bindings.

Hey, mostly because the underlying TUI framework (textual) is _amazing_.

Not gonna lie, I am not a big fan of Python anymore because of the dependency hell you can run into when working with CUDA libraries but uv is a breath of fresh air and textual is just sooo easy.

Don't get me wrong, I am not saying that I would never consider rewriting it in Rust or Go, but the documentation and guides from Textual were great resources and the creator, Will, also seems like a really nice guy :D

Edit: To be explicit, I believe that Python has currently the most advanced and accessible TUI ecosystem. At least, that was my impression after checking a few examples on an afternoon.

The switching of command and service in the argument order from the init script days still catches me out.

foo restart vs restart foo

I absolutely love this, and love it even more because of the Nix integration. Fantastic job.

Out of curiosity, any tools that provide a systemd gui minus the fzf dependency?

This looks very nice, I’ll have to give it a try!

This looks really well done. Congratulations!

Is there no GUI for SystemD?

kudos, this is super neat and useful !! Thank you

This looks neat. I have to look up the very fiddly and unintuitive systemd commands all the time. service start? service.foo start? start foo.service? Oh right, sudo systemctl start service.foo

And the feedback is so bad. It should know everything in its own config dir and tell me how to do what I want to do. Was it enabled? I forget. How do I look at logs? Oh right journalctl. Also the layout of things with lots of symlinks and weird directories in places that annoy my 90's linux sysadmin brain. Why am I looking at /lib/systemd/system

I am annoyed by the redundant "systemd/system" directory name every time I have to go there. At this point, just promote it to /etc/systemd and build a better CLI.

As a very occasional linux sysadmin just trying to make things work, the "typing at a console" systemd interfaces are not fun to work with. Maybe nobody should be doing that. In an enterprise, sure that's different. I think interfaces should be human, and linux should still be fun.

I haven’t had much trouble with the cli, but it is kinda wordy. I made this alias:

    alias sc='sudo systemctl'

Now it matches the sc utility introduced in Windows many years back. Then remember the verb goes first.

Yeah, and you can omit the ".service" suffix, and you could use ".timer" suffix, too, if you have a timer for that service.

As much as I do not like systemd, I do not think its commands are an issue, and I use "systemctl status" and "journalctl" as well with some flags at times.

"systemctl" would be less of an issue if "sysctl" didn't already exist and do something completely different.

Yes, it is something I had to learn in the very beginning, but I never made the mistake ever again.

It is a bit crazy to me how everyone says “dont use cron systemd is in now” but cron just does what it says on the tin with no problems. I have lines that work fine ran in script or on my crontab but when wrapped in a launchd command no longer work (log says things work until the db is to be updated which tells me launchd ran processes lack sufficient permissions perhaps to update my db but its not clear why this is the case or how I can elevate launchd sufficiently.

Launchd is not used in systemd - why would anything be wrapped in it? The config is simple - User for the user and Group for the group. You can print it the result of "id" if you're not sure what the result is.

Sorry I conflate the two since they share a lot of similarity. Launchd is what I use as its a macos system.

> cron just does what it says on the tin with no problems.

I can name a rather large problem with cron that systemd timers solve handily: long-running job duplication. When jobs take longer to run than the space between their triggering times, duplicates start piling up. I've had to rescue numerous systems from such states, which are difficult to detect until things have gotten quite bad. Sure, you can write a bunch of boilerplate to handle this yourself with cron, but with systemd timers it's all handled for you along with other niceties like capturing all output in journald and ensuring that the next run starts at soon as possible.

> ...cron just does what it says on the tin with no problems.

Yeah, tell me about it.

We had a production-down support ticket filed for one of the things that I work on at $DAYJOB. The customer's VM's disk was full. Why? Because the 'timer unit' that was supposed to run logrotate every day had never been run and was never scheduled to run. (The VM had been up for a month at this point.) No other customers had ever reported this issue, and we'd not changed anything about that timer unit or what it is supposed to run in ages.

No amount of gyration or agitation with 'systemctl' and friends OR rebooting of the VM kicked the cron replacement into proper functioning. 'logrotate' was simply never being scheduled to run. This fucker was WEDGED, and the tools weren't helping us understand why.

We read through all the docs on all the various kinds of units and don't see what we're doing wrong. We do a BUNCH of digging, and find an -IIRC- open Github issue from years back where someone was running into this problem. More or less the last word on the issue was Poettering saying something like "Well... actually, now that we've said that that particular cascade of options isn't actually supposed to work, now I'm not so sure that they're NOT supposed to work.". And that was -apparently- that.

IMO, when your cron replacement can be easily configured in such a way as to never even try to run the thing that you scheduled it to run, you really need to go back and rethink how you've built your cron replacement.

At least systemctl status tells you when next the timer will trigger. Cron doesn't.

Actually it does, have a look at cronnext

First time I hear about it. And it's not surprising, because it's not in cron. It's in cronie, and was "only" added in 2016.

I only found out last week!

Was not aware that vixie cron was first released 1987(https://github.com/vixie/cron/blob/master/Documentation/Chan... 1.md), and still has fresh commits.

Considering how many things of questionable veracity are said about systemd in general, and Lennart in particular, I would like a link to that issue before I take your characterization at face value.

> I am annoyed by the redundant "systemd/system" directory name

It's not redundant, you also have /etc/systemd/user (and /lib/systemd/user) where units that run in the user context (as opposed to system-wide context) are stored.

It's also worth noting that this is a fairly standard pattern in /etc

  /
  | etc
  | | fail2ban
  | | | action.d
  | | | fail2band.d
  | | | filter.d
  | | | jail.d
  | | ssh
  | | | ssh_config
  | | | sshd_config
  | | systemd
  | | | network   # network wide context
  | | | nspawn    # containers
  | | | system    # system wide context
  | | | user      # user wide context

Personally I like it more than

  /
  | etc
  | | cron.d
  | | cron.daily
  | | cron.hourly
  | | cron.monthly
  | | cron.weekly
  | | ...
  | | firewall
  | | firewalld

Keeps things less cluttered. Hierarchical categorization is >> than lateral

It could have been named 'global' instead of 'system'.

I find that at least systemd means that it's consistent across distros. I spent way more time looking up this kind of thing when every distro rolled their own init system.

This is why I have instated a policy of using systemd --user services whenever possible.

If you don't need elevated permissions this is ideal.

All you have to do is enable linger using loginctl if you want your service to auto start as the user on boot unattended.

Your user services live in ~/.config/systemd/user

I used to do this but frankly it's easier to run a system-level unit as whatever user you want and keep all the files in /etc instead of scattered around /home.

The user-level units are most useful when running an actual multi-user system. If you trust your users to not abuse them, anyway.

I tried, but found it very confusing around the user session. Apparently an environment variable need to point to a started dbus. I can't get consistent results over local session vs ssh vs su vs sudo

What are you trying to do with dbus?

My biggest annoyance is "systemctl status" gives you just enough of the service's log to make the output take up most of the terminal each time you run it but never enough of the service's log to get a useful picture of what's actually happened with the service lately.

Not to mention unless the problem with the service completely prevented it from running (it advises some commands to run in that case) you're supposed to just always remember "journalctl -xeu $SERVICE" was the incantation, less you want to go look up the flags again or manually parse the entire "journalctl" output.

Overall I generally like systemd though. The syntax can just be a burden sometimes.

In case you don't know, you can use the `-n` argument to `systemctl status` to tweak the log output, e.g. `-n0` to disable the log output and `-n40` to get more than the default 10 lines.

That's a great option to tweak the behavior and I hadn't known about it (or if I ever had, I'd well forgotten). Thanks!

From the man page it ?looks like? if you want reverse or full then it's still off to the journalctl command and arguments but at least "-n9999" is better than "always 10 lines".

FWIW, I don't want that behavior. 10 lines is great for my usage.

And IIRC you can change the default behavior. In the worst case, just alias it if it is bothering you that much.

Glad you like it! I'd rather learn to deal/work with the rough spots than stick my head in the sand with an alias and be even more lost when I don't have it or am working with someone else though. At least for the super common tools like git, systemctl, grep, and the like.

  > I'd rather learn to deal/work with the rough spots than stick my head in the sand with an alias

I highly encourage you to learn the commands. But it seems like you did.

I would not call something so stylistic as `git branch -vv` as "sticking your head in the sand". Your fallback is not going to really confuse you or put you at risk of not knowing what's going on. I can totally understand if we were talking about aliasing something much more complicated, but at that point it's a script...

Nice! I didn't know that was an option. Definitely something I should make configurable in `isd` :+1:

> My biggest annoyance is "systemctl status" gives you just enough of the service's log to make the output take up most of the terminal each time you run it but never enough of the service's log to get a useful picture of what's actually happened with the service lately.

How about

systemctl status foo | tail

it's the same mentality that brought us the git design - the easiest, least typing options are rarely, if ever the thing you want to do.

Instead, these invocations give cryptic messages, throw errors, or sometimes, even break things.

The most common and helpful things are hidden deep behind multiple flags and command line arguments in manuals that read like dictionaries more than guides.

I'm always at a complete loss as to how such decisions are made. For instance, "git branch -vv" is the useful output you would like to see, every time, that should be "git branch". Why not make the current output, "git branch -qq"? Is a humane interface too much to ask for? Apparently...

I know people defend this stuff, but as a senior engineer in programming pits for 30 years, they're wrong. Needless mistakes and confusions are the norm. We can do better.

We need to stop conflating elitism with fucked up design.

  > Why not make the current output, "git branch -qq"? Is a humane interface too much to ask for? Apparently...

Yes, it is too much.

You have the wrong mentality, and I hope this can help make your life easier. Programs are made so that the simplest option is the base option. This is because there is a high expectation that things will be scripted AND understanding that there is a wide breadth of user preference. There's an important rule

  DON'T TRY TO MAKE A ONE SIZE FITS ALL PROGRAM

Customization is at the root of everything. We have aliases that solve most of the problems and small functions for everything else. You default to an non-noisy output, showing only __essentials__ and nothing more unless asked. Similarly, you do no filtering other than hidden files. This way, everyone can get what they want. Btw, this is why so many people are upset with default options on things like fdfind and ripgrep.

For your problem with git, there are 2 solutions you have.

  # alias git branch using git
  git config --global alias.branch 'branch -vv'

  # write a simple function and add to .bashrc or .zshrc or .*rc
  git() {
      case "$1" in
          branch)
              shift
              command git branch -vv "$@"
              ;;
          *)
              command git "$@"
              ;;
      esac
  }

  > We need to stop conflating elitism with fucked up design.

The design isn't fucked up, it is that you don't understand the model. This is okay. You aren't going to learn it unless you read docs or books on linux. If you learn the normal way, by usage, then it is really confusing at first. But there is a method to the madness. Things will start making more sense if you understand the reason for the design choices. (In a sibling comment I wrote the abstraction to command patterns that makes the gp's confusion odd. Because systemd follows the standard)

Side note: if you try to design something that works for everyone or works for the average person, you end up designing something that is BAD for most people. This is because people's preference is not uniformly distributed, meaning the average person is not representative of any person in the distribution. This is because anything that is normally distributed has its density along the shell while a uniform distribution has a uniform density all throughout it.

Fully this. For all its foibles, Linux was built to never presume too much, and its users tend to be power users who will almost certainly have dotfiles to tune their systems to their needs. In the context of making choices that will necessarily be universal, I admire how thoughtfully most standard Linux packages have been designed to never interfere with the users’ intentions.

And to all the linux noobies[0], you'll be a hell of a lot more efficient if you learn the philosophy of the design early. It will make it so that you can learn a new command and instantly know how to use several options. It will dramatically reduce the number of things you have to learn. I also HIGHLY suggest learning a bit of bash scripting.

Take a look at the manual

  https://www.gnu.org/software/bash/manual/html_node/index.html

and bookmark "Bash Pitfalls"

  https://mywiki.wooledge.org/BashPitfalls

Live in the terminal as much as you can. It is harder at first, but you will get huge boosts in productivity quicker than you would know it (easily <2 weeks). It sucks doing things "the hard way" but sometimes that comes with extra lessons. The thing is, those extra lessons are the real value.

[0] no matter how many years you've been using it there's no shame in being a noobie

Making things customizable doesn't mean the defaults should be useless. zsh, tmux, emacs, vim and bash, out of the box, for instance, have both pretty nice defaults and are highly customizable.

I know it's hard to make things like this but let's do it anyway.

Sounds like entitlement. Ever consider there are more users than yourself?

You're fundamentally misunderstanding things.

If every person has the same point of confusion than they are not the problem, it's the thing they're confused by.

There's better ways to do things and calling people naive for suggesting the obvious is the problem.

And about your side note: no. For example, when people checkout a branch, they want to track the remote branch 99.9% of the time. It should be the default.

The default journalctl should show where things have failed, that's why people are invoking it.

Also there's plenty of counterexamples that do these things. "ping host" genuinely pings the host. "ssh host" genuinely ssh's into the host.

You don't need to specify say, the encryption algorithm, that you want a shell, use say, "--resolve=dns" to specify the hostname resolution... It has sensible defaults that do what most people intend.

Under the model you advocate for "ssh host" would simply open up a random socket connection and then you'd have to manually attach a virtual terminal to it and request the invocation of shell separately, stacking each piece on top of the other, before you log in.

This design could be defended in the same way: Some people do port mapping, tunneling, SOCKS proxies, there's too many use cases! How can we assume the user wants a shell? Answer: because they do.

Most things are reasonable like certbot, apt, tune2fs, mkfs, awk, cut, paste, grep, sort, so many reasonable things. Even emacs is reasonable.

But systemd and git are not and the users are not the problems. Choices were made to be hostile to usability and they continue to be defended by being hostile to usability. Things like lex and yacc are inherently complicated and there's nothing to do there. Other things are intentionally complicated. Those can be fixed.

  > The default journalctl should show where things have failed

How? What do you filter for? Emergency? Critical? Error? Alert? (see `dmesg -l`). What's the default? Do you do since boot? Since a certain time? Since restart?

FWIW, I invoke it all the time for other reasons. I am legitimately checking the status. Is it online? Do I have it enabled? What's the PID? Where's the file (though I can run `sudo systemctl edit foo.service`)? What's the memory usage? When did it start? And so on. The tail of the log are useful but not the point of status.

If I'm looking to debug a service I look at the journal instead. I hope this helps

  # Show 10 most recent logs (you don't know what service to debug)
  journalctl -r -n 10
  # Show recent logs for a known service (can tab complete)
  journalctl -r -n 10 --unit foo.service
  # Some useful flags
  -S, -U
    Since and until. Time filtering including keywords {now, yesterday, today, tomorrow}
  -b
    Since boot. You can even do since a specific boot! Or even `--boot=-1` for the last 2 boots
  -p, --priority
    You can use the numbers 0-7 representing {emerg,alert,crit,err,warning,notice,info,debug} respectively
  -g, --grep
    No comment needed 
  -f, --follow
    This is the tool while debugging since it'll update.
  -e, --pager-end
    Start at the end of the pager 

  > that's why people are invoking it.

That's why __you're__ using it, but don't assume that your usecase is the general. Remember, linux has a large breadth in types of users. But their biggest market is still servers and embedded systems. There's far more of those than PC users.

  > Choices were made to be hostile to usability and they continue to be defended by being hostile to usability.

Idk, when systemd became the main thing I hated it too. But mostly because it was different and I didn't know how to use it. But then I learned and you know what? I agreed. This took awhile though and I had to see the problems they are solving. Otherwise it looks really bloaty and confusing. Like why have things like nspawn? Why use systemd jobs instead of using cron? Why use systemd-homed instead of useradd?

Well a big part of it is security and flexibility.

I write systemd services now instead of cron jobs. With a cron job I can't make private tmps[0]. Cron won't run a job if the computer is off during the job time. Cron can't stagger services. Cron can't wait for other services first. Cron can't be given limited CPU, memory, or other resource limitations.

Nspawn exists to make things highly portable. Chroot on steroids is used for a reason. Being able to containerize things, placing specific capabilities and all that. This is all part of a systemd job anyways. It really makes it a lot easier to give a job the minimum privileges. So often nspawn is a better fit than things like docker.

Same goes for homed. You can do things like setting timezones unique to users. But there's so much more like how it can better handle encryption. And you can do really cool things like move your home directory from one machine to another. This might seem like a non-issue to most people but it isn't. That whole paradigm where your keyboard is just an interface to a machine (i.e. a terminal, and I don't mean the cli. There's a reason that's called a terminal "emulator"). This is a really useful thing when you work on servers.

Look, there's a reason the distros switched over. It's not collective madness.

[0] https://www.redhat.com/en/blog/new-red-hat-enterprise-linux-...

https://systemd.io/

I'm not assuming I'm the default. I've worked countless jobs where I'm the "linux guy". These are the most common points of failure for everyone.

You have yet again showed how you are one of the 0.1%.

Why should the defaults be to accommodate for you?

I've been using linux for over 30 years. I'm in that group photo on the Debian.org homepage, given Linux conference talks spanning decades...

I know I'm in a very small minority and don't expect everyone to be me.

I work a support shift at a cloud hosting company and constantly deal with our customers struggling with linux on the same pain points over and over and over again. We have a bunch of auto-replies to explain things because everyone trips over the same broken sidewalk and people insist it's not broken but instead, it's beautiful design. Alright, whatever. Keeps me employed. Cool.

This is the same problem that mathematics has. Elite mathematicians insist that they should give zero shits about trying to explain things and their convoluted impenetrable explanations are elegant - it's an ultra-exclusive nerd club and most people aren't invited regardless of their interest.

Making things more approachable is somehow, cognitively unavailable to them or against some deeply held constitution. Whatever it is, it's not happening.

Some people think iOS is successful and some people think Mobian should be. I try to understand the first group and not insist everyone be in the second.

  > Elite mathematicians insist that they should give zero shits about trying to explain things and their convoluted impenetrable explanations are elegant

I'm here trying to explain...

I'm very much not trying to be like the Arch userforum and say that this problem was solved 3 years 7 months, 22 days, 9 hrs, and 13 minutes ago but will also not provide you a link or any more reference to this because I'm closing the topic. There 100% are elitists. But not everyone who agrees with their point is an elitist. That experience is always frustrating because a noob doesn't know what questions to ask or what terms to search. It's true that in any domain that a task is literally easier for the domain expert than the noob because the noob doesn't even know where to start looking.

As for users, I do agree that there's a lot of common stumbling blocks. But I also notice (the same problem exists in math) that a holistic view is not taught with or before specifics. These things have to be taught together, but if you try to just learn linux a command at a time (math an equation at a time or programming a function/algorithm at a time) you're going to be stunted for a very long time. The abstractions matter.

But why are aliases and functions disallowed? I've thought the TUI boom has been a great thing for users, although I wish TUI developers would spend just a bit more time doing design and understand that "has vim movements" means more than h,j,k,l arrow keys... There's lots of great alternatives to the most common coreutils. But I am still a firm believer that people should be making their own rc files. Especially today where it is trivial to carry them around with you. Hell, I even throw notes in my dotfiles. The whole thing is 84M and 81M of that is vim plugins, so not even part of the git repo.

  > Some people think iOS is successful

Well I still don't know how to get my iPhone to stop correcting two words back. You'd think turning off General > Keyboard > Auto-Correction would do it. Nope. Maybe Predictive Text? Also no. But then again, you'd think that Auto-Capitalization would capitalize the letter I when it is used as a single character but again, no, and you can always recognize an iPhone user because their "i"'s are never capitalized.

I'll tell you this. Linux is a pain. All of software is a pain. 90% of software is a pain and has no fucking reason to be. I can give you a hundred examples where trivial things make a huge difference. From fucking Meta's download for Llama not telling you how much disk space it uses and hard coding the destination (2 problems), how calendard won't merge holidays and instead you have 5 instances of MLK Jr's day, or how I can't disable Apple Music or remap my media keys so I can stop the god damn Music player app from opening or get it to stop playing some random youtube video in a lost tab rather than play the song on spotify that I literally just paused. But the reason I choose linux is because I can fucking fix those things. I hate it. So much of it. You're so right that so many things don't have to be this way. But at least with linux and it's design choices, I can fucking fix things. It isn't just the open source aspect that makes that possible and it isn't just documentation, it is the mentality that things should be small and simple. To really take to heart the concept of monads.

here's another for the list:

systemctl reboot -f should be the default for "reboot".

People generally already make sure there's nothing they care about running and then issue a reboot often because something is broken.

It's already mostly a manual override. The default makes it not a manual override but instead yet another operation of a system that is, under most reboot conditions, assumed to be tainted in some way.

If you want reboot to potentially require IPMI or some kind of manual intervention, it should be an option like

reboot --ask-everything --stall-on-crash-ok --wait-indefinitely-ok

or something that makes it very very clear what you're asking it to do.

Because how it currently is, reboot does not reboot, instead it inquires a potentially malfunctioning system what its opinion is on rebooting.

It effectively assumes every system is always in pristine condition and that users reboot out of boredom. If someone’s issuing a reboot, it's because something is wrong, and the system should treat that as the primary assumption.

  > systemctl reboot -f should be the default for "reboot".

I very much disagree. This is not what I want and certainly you can recognize that this is an unsafe operation, right? Less safe operations should ALWAYS require more work. Not everyone checks and it is also easy to forget or miss something.

  > reboot --ask-everything --stall-on-crash-ok --wait-indefinitely-ok

I like the --ask-everything option but I'm not sure how the other 2 can work. A crash, especially during shutdown, can have no guarantees of being caught. Or do you mean crashing user programs? Well then that's --force, right? The --wait-indefinitely-ok seems a bit weird too. Shouldn't that be configured in your boot options (or bootctl)? I do think there's reasons you might want this in one situation but not another so flag sounds good.

  > If someone’s issuing a reboot, it's because something is wrong

On personal machines: >9/10 times I'm rebooting because I installed a new kernel. Granted I'm usually running an arch distro, but even on other machines it's pretty similar.

On servers, yes, you're right, I reboot far less and am usually rebooting because a specific GPU server has a defective GPU that is often a pain to solve with rmmod and resetting manually, being just far easier to reboot the entire machine.

But I still think it's clear that what you think is "average" or "usual" is not. Literally the fact that you and I disagree on what our typical use cases is proof of this (note: I'm not saying _my average_ use case extrapolates, nor your average use case, nor anyone else I know or even don't know. I'm saying _average_ is not a meaningful thing. I mean this in the same way as taking the average of 2 samples from a normal distribution; doing so gives you a number that is not representative of the distribution).

Average is not the advocacy, it's plurality.

Some behavior has to be chosen for some invocation.

I continue to express what countless people struggle with and potential usability fixes, and you continue to insist your personal preference is supreme and unilaterally invalidates everyone's frustration.

It's like insisting all cars should have tillers, manual chokes, and hand cranks because you personally like them and then handwaving some "average" argument when someone advocates for easier systems.

And that's the attitude what keeps Linux useful but not usable.

I've heard this reasoning multiple times, but that doesn't make it right. Like you said, you ARE presuming something: That scripting and a (useless) base case are the most common usage.

How many people script Git? More importantly, how often do you change that script? Rarely, and rarely. That means it's FAR FAR less work for the scripter to look up the weird incantations to remove the "human" parts (like a quiet flag).

Conversely, the human is far more likely to write git commands impromptu, and I dare say, this is the FAR FAR more common use case.

That means Git (and a lot of these kinds of commands) optimize for the RARE choice that happens infrequently over the common case that happens often. That's horrible design.

TL;DR: Just because there's a consistent design philosophy behind obtuse linux commands does not make a good, helpful, useful, modern, or simple philosophy. If a library author wrote code this way, we'd all realize how horrible that design is.

  > That scripting and a (useless) base case are the most common usage.

You're thinking as a user. I'm sorry, but the vast majority of linux systems are still servers and embedded systems.

Here's two versions of the Unix Philosophy

    - Write programs that do one thing and do it well.
    - Write programs to work together.
    - Write programs to handle text streams, because that is a universal interface.

    - Make it easy to write, test, and run programs.
    - Interactive use instead of batch processing.
    - Economy and elegance of design due to size constraints ("salvation through suffering").
    - Self-supporting system: all Unix software is maintained under Unix.

Yes, scripting is a common use case (I use git in scripts! I use git in programs. You probably do too, even if you don't know it). Piping is a common use case. And I'm not sure what we're talking about that is "useless". I've yet to run into a command where the default is useless. Even though I have `alias grep='grep --color=always --no-messages --binary-files=without-match'` I still frequently run `\grep`.

But we're also talking about programs that are DECADES old. git will turn 20 this year. You're going to be breaking a lot of stuff if you change it now.

And look, you don't have to agree, that's fine. But I'm trying to help you get into a mindset so that this doesn't look like a bunch of gobbledegook. You call these weird incantations. I get it. But if you get this model and get why things are the way they are, then those incantations become a lot less weird and a lot easier to remember. Because frankly, you'll have to remember far less. And either way, it's not like you're going to get linux changed. If you don't use it, great, move on. But if you do, then I'm trying to help, because if you can't change it you got to figure out the way.

https://en.wikipedia.org/wiki/Unix_philosophy

> Here's two versions of the Unix Philosophy

Man, we're talking about git. It has never subscribed to this philosophy (and most programs in *nix don't)

Git does about fifteen thousand different things with about fifteen million flags.

And it was never designed. It was thrown together as a kitchen sink of useful things with commands adding new and crazy incantations as new things were added.

"Just write this script to do the useful thing" is a weird hill to die on

Why are you assuming that your use case is the most common?

None of those problems are unsolvable. The basic idea is: your terminal and script can get two different outputs. Git can already do that with colours, so it can also use more verbose for basic commands.

The other part is - scripts can use longer options while the console commands should be optimised for typing.

This has nothing to do with understanding the model or model itself. Complex things can have good interfaces - those just require good interface design and git did not get much of that.

  > Git can already do that with colours, so it can also use more verbose for basic commands.

You already answered your own question. There's plenty of terminals that don't support colors. Plenty that don't support 16 bit colors. Do a grep into a curl and you'll find some fun shenanigans.

  > This has nothing to do with understanding the model or model itself.

You're right. This has to do with you thinking your usecase is the best or most common.

As stated before, there is no average user[0]. So the intent is that you have the command that can be built upon. Is everyone unhappy with the base command? Probably. But is is always easier to build up than to tear down.

[0] Again, linux servers and linux embedded devices significantly out number users. Each of them do. So even if there were an average user, guess what, it wouldn't be you.

> There's plenty of terminals that don't support colors.

Colour support is irrelevant to this case. They can check isatty() and give different output depending on that.

And verbose information is better in average case, if it's well formatted. Unless you're flooding the screen with useless data of course, but that's not what's being talked about here.

  >  if it's well formatted.

Let's be real, how often do you see this?

  > verbose information is better in average case

Honestly, no. I hate this. I spend far more time filtering output than trying to generate more. I'm glad I have the option to generate more, because sometimes the needle isn't in the haystack. But usually more verbose output just means adding more hay.

Again, it is really easy to add more output and this is actually an easier problem than the reverse. FATAL ERROR WARN INFO DEBUG TRACE log levels have been around for awhile and they've been working really well.

I'll let you in on something. If you're frequently having to reach for verbose outputs, there's probably a better tool. That is, unless if it is personal preference. Which in that case, alias is for you. I have to reiterate, the "average case" and "average user" does not exist. This argument will not get you anywhere because you're talking about a unicorn. No matter how much you think they exist, they don't. It is only average within a more specific subdomain. It might appear average to you because you work with those types of people and are part of those communities, but someone else from a different community will have very different viewpoints. So it is very intentional that things are not designed for "the average" because there is none.

In my experience, most of the times a service fails, it fails on startup (misconfiguration, missing resources). So the status is OK. Also, you can always request more lines, if you think that's helpful.

This wouldn't be my problem with systemd. Not by a long shot.

> I have to look up the very fiddly and unintuitive systemd commands all the time. service start? service.foo start? start foo.service? Oh right, sudo systemctl start service.foo

I don't get this complaint. It's the same order as almost every other command-line utility that has subcommands: <command> <subcommand> <thing to operate on>. To me, that kind of consistency is very intuitive.

    systemctl stop my-service
    systemctl status my-service
    git add my-file
    git remote remove upstream
    apt install my-package
    docker run my-container
    adb push local-file remote-file

For me:

* /etc/init.d/my-service stop

and Ubuntu’s:

* service my-service stop

both lurk in my brain.

Sure, it's different from the old way, but I don't think "unintuitive" is the right word for that. systemd forced people to change their habits so that it could be more intuitive. Of course, people are going to disagree about whether it was worth it - it's the age-old question about breaking backwards compatibility for the sake of minor improvement. Personally, I got used to it pretty quickly and I like it more than the old commands now.

Yet, the "ease" of use is a joke. Every try to stop and start a service at different points? At points you choose? Have fun!

Ah well, it's all been said before.

Yeah, I gave up resisting and I’m rolling with it. Ok, fine, this is the way now.

And yet my fingers still want to type it the other way.

The way I remember this is that the old way didn't allow for applying the verb to multiple units. Now you can "restart" multiple units, i.e. `systemctl restart nginx webapp`, etc.

Thanks. I'm going to use that to remember it now.

Plus, you could always write a quick shell script to swap the arguments if it detects this specific failure. Inability to remember or learn which commands expect which arguments is, at some point, not the responsibility of the software to fix. Unfortunately, this field does necessitate some amount of memorization and ability to problem-solve.

I feel the same way. The big part for me is that it tells us that owyn doesn't use tab completion if they're forgetting about the ".service" part. Sure, I don't remember either, I don't have to.

I'll add the abstraction for anyone confused

  program [command [subcommand]] [flags] [object]
  e.g.s
  systemctl status sshd.service
  systemctl enable --now sshd.service
  touch -c test.sh
  echo 'Hello World'
  echo "Hello ${USER}"

Anything in brackets is optional and might not appear or be available. By command I mean a category of commands. Such as 'pip install' vs 'pip uninstall', which are sub-programs inside the main program. But this can have layers such as 'uv pip install'. Often flags can be used in any order because you'll just loop over all the arguments but this is still the standard order.

There's also the two actor pattern

  program [command [subcommand]] [flags] source destination
  e.g.s
  cp /foo/bar/baz.txt "${HOME%/}/"
  scp -i "${HOME}/.ssh/foo" ${HOME}/to_upload.sh user@remote:~/
  dd if=/dev/urandom of=/dev/diskToDestroy
  rsync /mnt/hdd1/ /mnt/hdd2/

Also the `.service` part is optional anyway for most commands (including the start, stop, restart etc. ones they use in their examples). Only commands that can operate on services and other non-service units require it.

When it comes to starting and stopping services I want the verb to go last. Way easier to press up, backspace backspace backspace o p to change service ssh start to service ssh top. This is a frequent pattern I follow as I start/stop/restart/reload. Having to go back at least one word adds keystrokes that aren’t necessary.

Doing it the way it does allows specifying multiple services.

systemctl status myapp mydb

Why not allow for both and accept the option at any location?

Something like:

systemctl --stop myapp mydb

systemctl myapp mydb --stop

unnecessary flexibility brings unnecessary bugs

All `--` strings should be "options". Starting/stopping a service is not an option, it's a command. I don't enjoy `pacman` for this very reason.

I would rather do that as two commands than to scroll to see the output of each.

There is no output for success.

Only failure is rewarded!

NOBODY ask for a status of two services.

Hence the verb-last CLI rulez!

Press alt+B twice?

Alt + Bikeshed?

(Kidding aside, quick reminder that on Mac you have to enable Settings>Profiles>Keyboard>Use Option as Meta key, or else Alt doesn't work)

I agree, but here's a handy Bashism:

^art^op

Converts the "start" in "foo start bar" to "stop", ie runs "foo stop bar". Append :p to do the substitution but print the command instead of running it.

Learn something new every day. What is the ^ representing in this case?

Caret is a word designator in bash's history scheme. Here it acts as a way to mark the thing with the thing being replaced. There's a more detailed explanation in [1] and a quick read in [2]. Essentially bash let's you do manipulation and re-execution of commands from your history and one of the manipulations is via word substitution.

[1] https://www.gnu.org/software/bash/manual/html_node/Word-Desi...

[2] https://www.johndcook.com/blog/2022/10/21/shell-replace-word...

It actually should be possible to switch them around. No-ones going to call their service 'stop' or 'service' right?

Someone should create a set of systemd units that show something funny when entered, like how `sl` does for `ls`.

Do you use your space bar to handle heating ?

What do you mean?

It's a reference to an XKCD comic https://xkcd.com/1172/ that pokes fun at users with strong attachments to some software behaviours and/or interfaces.

Also, it looks like you are one of today's lucky 10,000 ! https://xkcd.com/1053/

Haha excellent! Thank you!

Just use an alias. I don't understand why a problem with such a simple solution should cause such issues.

Because I switch computers (often, for work), and what's muscle memory on one then becomes "command not found" on others without the alias. Many of those computers I don't control and can't say "well, everyone should just run my aliases".

Because I have to share commands with other people who are troubleshooting their own systems, and copy/paste from history becomes useless if I have specific aliases.

Because someday I or someone will want to script these interactions, and aliases are not available in subprocesses.

People are telling you why. You're just choosing to disregard their answers as insufficient. This is not a them-problem. It's a you-problem.

It’s because service was the other way around, I’m sure.

systemd services are named "foo.service", and you do not need to specify the ".service" in almost any case.

They're referring to Debian's service(8) command.

  NAME
       service - run a System V init script
  SYNOPSIS
       service SCRIPT COMMAND [OPTIONS]

(the manpage wasn't updated, but the same command also supports systemd services nowadays)

If I had to guess it's because of the 'service' command, which goes 'service foo start'.

It took me ages to unlearn that pattern for using systemctl, even though as you say: it's far more consistent

I think the previous post mentions the `service` command, which has `service foo start` pattern.

I'm used to `service` command, so I have muscle memory which `systemd` breaks. The way `systemd` commands are laid out is better, it just messes with `service` command muscle memory.

It's been a decade at this point since the last big distro (and surrounding drama), Debian and Ubuntu following shortly after, moved to systemd.

Many distros still provide the `service` command, and don't print any warnings when you use it. At best, you might get an informational message that the equivalent systemd command is being called. There is no recommendation to call the systemd command directly, or any sign of the `service` command being deprecated any time soon. As a result, a lot of people are probably still relying on their muscle memory for the `service` command.

I disagree that the way systemd is laid out is better.

With initV, issuing many command in sequence to the same service (enable, then start… stop then start… etc) is much quicker and easier as it’s “up-arrow, control-W, type new command. With the systemd option I either need to navigate one word to the left (or more depending on how the shell is configured wrt word separation) before I replace the command, or I need to delete both the command and the service name, and retype more.

It’s not a huge difference but small downgrades in ergonomics add up over time.

A huge typing pain.

A backward step in intuitive CLI usage.

We can be such a fuddy-duddy gray-beards.

Aliases

duct tape

superglue

Systemd does have 'restart' and 'enable --now' (which starts the service too).

Too bad it doesn't have a '--status 2' flag to show the status after two seconds too.

The systemd layout allows you to send the same command to multiple services, which is much more frequently useful than sending multiple commands to one service.

I can't remember the last time I had to send the same command to multiple services/units.

Normally when I'm doing things on the terminal, I do one thing at a time.

I on the other hand DO get this complaint. It’s less about the “word order” and more remembering the words to type. As a software engineer I have to google stuff all the time. I haven’t touched C#’s PLINQ in five years, I doubt I could do it from memory. Likewise, I can count on one hand the number of times I will set up a systemd service in a given year.

Many (most?) of us are all over the place, expected to “wear many hats” and aren’t in a single IDE or language all day every day. Certainly if I were a “Linux admin” I would have systemd pretty committed to memory. Anyone else probably wouldn’t.

sudo service my-service start

sudo service my-service stop

I just wish systemd allowed abbreviations for the subcommands, like "ip" (and git, for long options).

Author here: Yeah, I agree.

It is a bit weird. On the one hand, I understand that it makes sense to have [command] [verb] [object] on a "logical" level and that viewing logs should be a separate command (`journalctl`), but it is definitely not ergonomic. Especially if you frequently have to switch between start/stop/restart.

> As a very occasional linux sysadmin just trying to make things work, the "typing at a console" systemd interfaces are not fun to work with. Maybe nobody should be doing that. In an enterprise, sure that's different. I think interfaces should be human, and linux should still be fun.

This was precisely the case for me. I "enjoy" playing around with systemd and am super interested in better understanding it, but the feedback loop just felt sooo slow. So hopefully this TUI can make it "fun" again :)

When I'm really working with a service and get tired of typing sudo systemctl restart or status all the time, I'll just do a quick alias or two right then and there.

alias s=sudo systemctl status alias r=sudo systemctl restart

And if I'm only working with one service, I'll throw the service name in there too.

I am going to be that guy, but I went full runit for similar reasons.

Give me files for my logs, give me a single place for my service definitions, be simple.

After years of systemd, I have never really "got it", while it took a weekend for runit.

It's not always our choice, corporate world is something else, I know...

By the time you have it all memorized, they'll change it. Again. Because... because.

Fiddly and unintuitive? A lack of exposure to something does not really allow room for valid criticism of the thing.

Just a tip, if you remember the service name, status will show the directory the unit file is in. That will hopefully get you over your issues with directories. Complaining about Linux directories seems weird though. Have you looked at... anything else at all?

Why did a single person tossed out decades of CLI wisdoms devised by many gray-beards?

Bring back the verb-last CLI.

    systemctl rsyslog stop
    systemctl rsyslog start

Don't be suggesting `^op^art` over up-CtrlW This is worse than pacman.

Because this is awful? The way most other tools work is <command> <subcomand> <sub-subcommand> --option=argument argument.

There are so many ways to recall the command you typed from history... I don't understand your complains at all.

You must not be an sysadmin

Ah yes, famously one can only be a sysadmin if they're unable to use a different cli verb order.

Come on now. Either present a real argument or accept the fact that tooling isn't forever going to be frozen to what you used in your 20's. Newer tooling uses newer best practices and the improved verb order is part of that.

I work in a company that creates (admittedly crappy) product for sysadmins (I have no say in design though, so don't blame that on me). I work in the infra / automation, so close to sysadmin, but not quite.

So, this isn't a foreign world to me... I still don't think that:

1. Starting a service is a big issue in general (most of the time this happens automatically anyways).

2. The command syntax of systemcl is defective in any way. Of all things that might be problematic with systemd, this is just not in the list at all. It's such a bizarre complaint to have... kind of like arguing whether "gray" or "grey" is the right way to spell it. Really, it's a whatever. There are much larger issues, even when it comes to comfort, not functionality of systemd tools. Like, for example, the autogenerated names of device units. I hate trying to recall the rules for writing these, and on top of that, having to escape characters just to type the unit's name. That's just a ridiculous design.

Cause you can manipulate multiple services in a single command? Not a big deal when you're looking at one or two services but when you've got a 5 to 15 interlinked and interdependent services and in situations where you may not want to restart everything, it's nice to just have one CLI with tab completion to deal with that.

I think this isn't shoddy engineering; I think you disagree with a design tradeoff that was consciously made here.

If CLIs are designed around facilitating the easiest means of editing the most commonly-edited word, different programs will end up with semantically very different CLIs. In some programs, the "noun" (e.g. file path) will be the thing most frequently edited. In some others, like systemctl, it'll be the "verb", like start/stop. That means that different programs designed around this principle will be semantically extremely inconsistent with each other.

On the other hand, if consistency of "base_command sub_command --subcommand-arg sub_sub_command --option argument" is taken as the guiding principle, many different tools will act semantically similarly. This enables unfamiliar users (talking first-time users or people who have just opened the manpage for the first time) to intuit the right way to present commands to those tools, rather than expecting them to memorize (or re-look-up as needed) each one's specific semantics.

While there's merit to both, I think the second one--systemd's CLI approach--is better. Put another way: the user-centric and consistent (with other applications) design are sometimes at odds in this way. A tool that is hyper-focused on optimizing the most common tasks for its (power) users risks oddball status and hostility to new users. It's important to pick a point on the spectrum that appropriately balances those two concerns. Python ("There should be one-- and preferably only one --obvious way to do it") and the Apple HIG ecosystem both understand the truth of this: that it is sometimes necessary to trade away efficiency for familiarity/consistency. There's a reason Perl languished while Python grew.

Like, I get it. I've been a sysadmin/devops for decades, and the paper cuts add up. But it's easy to forget the paper cuts I'm not getting: modern tools are (generally, with exceptions) more consistent; there are fewer commands that are resistant to memorization and need to be looked up each time; fewer commands that lead to questions like "wait, is it 'cmd help subcommand', 'cmd --help subcommand', or 'cmd subcommand --help'? Am I going to have to Google how to get the help output of this command?"

Are there other cases of verb-last besides pre-systemd init systems?

apt-get/yum/dnf install foo git add file

And the normal CLI utilities: rm/touch/cat/ls/mv/cp etc are all verbs, and they act on nouns that come last.

Reply on news.ycombinator.com