Exploring CLI Best Practices
eng.localytics.com
eng.localytics.com
Don't use -v for version. It's very uncommon (though I know software that does it, it's confusing) and people will often blindly add -v for verbosity. A better alternative is -V (capitalized) and of course --version. These should both work, just like -h and --help should both work.
Turns out -v negates the match so I killed every process on the box but one.
I had to email someone to reboot the server.
It makes sense in the context of pgrep, since that emulates grep. But kill does not have a -v flag that means inverse.
Arguably grep predates any common practices, but I'd have to look that up.
-v Reverse the sense of the matching; display processes that do
not match the given criteria.
- It means `-thing` != `-t -h -i -n -g`, so the latter has to be that long;
- You also can't have `readable-long-things-as-an-option`;
- You can't (or not without crazy logic and a confusing UX) have variables passed with no space, like `-ffoobar`
This behaviour is defined in POSIX, btw:
>If the SYNOPSIS of a standard utility shows an option with a mandatory option-argument (as with [-c option_argument] in the example), a conforming application shall use separate arguments for that option and its option-argument. However, a conforming implementation shall also permit applications to specify the option and option-argument in the same argument string without intervening <blank> characters.
I suppose I don't mind that syntax anymore now. Thanks for changing my mind :)
This is trivially achievable with getopts(1):
while getopts u: Option
do
case "$Option"
in
u)
User="$OPTARG"
;;
esac
done
Trying the first few programs off the top of my head I found node, ruby, docker, clang, npm, and even the man command all respond to the `-v` flag. I think that's common enough for me to support it when I don't need a verbosity toggle.
Commands that do something without arguments, like cat (even though it just blocks, waiting for input) might still print the version to stderr I suppose.
That would be a way to solve both cases.
I'm not sure why version information has to have an option on its own. For humans it can be provided in help. Considering the limited usefulness it has as a separate thing, it's just command option garbage.
The convention is to consult the manual page first.
This means, for example, supporting both space-separated and equals-separated option values (--name value and --name=value). Some programs only support one or the other, so you have to guess which will be supported when invoking a program for the first time. Both forms have their uses too. --name=value is more explicit and should throw an error if the --name option doesn't take a value, so it's good for use in scripts. On the other hand, a lot of shells don't tab-complete file names unless the file name is in a separate argument, so for interactive use, --name value can be more convenient.
One of the greatest crimes which GNU is not UNIX has committed are the --long-options, and that horrible practice stems from the fact that command line applications on GNU/Linux have poorly designed usage displays, and even worse (often non-existent) manual pages, and when there is a manual page, it lacks an EXAMPLES section, in stark contrast to UNIX systems (Solaris, HP-UX and SmartOS are exemplary).
Ease of UNIX and efficiency thereof comes from brevity, and that particularly concerns command line options: the lowest amount of typing possible is the goal.
The solution is not to invent an arbitrary and horrible convention, but to design better usage output, and to deliver high quality manual pages with lots of examples.
Treat the root cause, not the symptom.
The GNU style makes things more explicit and readable, which is why people love it -- long options have been implemented far and wide outside of GNU.
And it has one nice feature: It lets you treat short options as concatenable (so instead of "-v -f" you can just do "-vf") without ever accidentally conflicting with long options.
-vf usage has absolutely nothing whatsoever to do with GNU getopt; it is the core functionality of both getopt(1), getopts(1), getopt(3C), and getopts(3C).
http://www.tpsoft.com/museum_images/Teletype%20ASR-33%201.JP...
https://en.wikipedia.org/wiki/Ken_Thompson#/media/File:Ken_T...
the ergonomy came first and foremost, not the slow links, and the ergonomy still plays an extremely important role on UNIX and UNIX-like systems, even today, because more typing and mistakes are still time consuming; those factors haven't gone away, and no link, however fast, can help with that.
Ergonomy is extremely important on UNIX. It's one of the major reasons for preferring the command line as opposed to a graphical user interface.
To test this: run non-customized /bin/ksh and see how fast you're able to type long command names without [TAB] completion and command history, and what your input error rate will be.
You can even use a modern shell, like tcsh or zsh which have extremely powerful command history and autocompletion facilities, and see how well you fare with the shell completion of --some-long-option=/path/to/somewhere. You can even use bash and see how well it will complete --some-option for you.
The message here is: if you're developing a command line application, please do not use --long-options, and please consider ergonomy very carefully. Save your users typing, and think about how you can reduce their error rates. --long-option isn't the way to do either of those things, it actually makes things more strenuous and slows one down.
As for "cop out" — it's not like long options prevent you from writing good documentation! Those are two different, unrelated things. If my tool has an argument "--force", that's sort of self-documenting, to be sure, which "-f" isn't. But most tools do need to document exactly what "--force" does, irrespective of whether it's a long option. GNU's lack of documentation can't be claimed on option style.
Your argument also sets up another false dichotomy: The presence of long options doesn't preclude to possibility of having short ones (--force maps to -f and so on), or indeed the possibility of short command names.
PS. I noticed the other comment about shell autocompletion. I use zsh, and I autocompletion of long options works just fine (including just typing -- + tab and getting a list of available options).
I must vehemently disagree with you and anyone else that these are two different, unrelated things, because I assert that there is a direct link between poorly or non-existent manual pages and --long-options, and no amount of punting to GNU texinfo, a solution proprietary to GNU/Linux, will change that.
As noted in a related post, brevity in UNIX has to do with ergonomy, and nothing whatsoever to do with link speed. That git has so many options is either a failure of the interface, or poor documentation, or both (after trying out git, I'm completely skipping it and going from Mercurial to Bitkeeper, and the interface is one of the reasons for that).
Actually Mercurial is a perfect example of just how irritating and anti ergonomic long options are, example:
hg revert --no-backup some/file
As for zsh, how could it possibly know how to complete the long options of every single application on the system? How does that work?
Also a point on your coreutils argument: two letter application names, where names can be meaningfully shortened, is good design every programmer who cares about their and their users' productivity should follow; where that isn't possible, the name should be shortened as much as it can be. For example, I wrote a program which builds a list of SVR4 package dependencies, but instead of naming it find_dependencies, I named it fndeps. The manual page should explain what the program does, not the --long-options; software without a comprehensive manual page on UNIX is fundamentally shoddy software.
You've also yet to show that long options is some kind of "great crime" that hurts anyone. It's ridiculous. People invented long options because short options are not intelligible unless you already know what they mean.
Your Mercurial example actually undermines your argument completely. I don't know "hg" at all, but when seeing "--no-backup" I immediately understand what that flag does, something which would not be true if the command were "hg revert -C".
Moreover: The fact that "--no-backup" exists does not exclude anyone. You can use "-C". If you can remember it. There's your hallowed ergonomy, you already have it.
As for zsh completion: A program simply needs to follow a specific protocol (take input, reply with the right output), and then be registered with the "complete" command. zsh comes with autocompletion for lots of things.
Are you a casual user of command line tools? If you aren't, you know that command line options of anything become muscle memory if you use it long enough; at that point anything that stands between you and the correct result is nothing but a hindrance.
The workflow on the command line is:
1. manual page;
2. SYNOPSIS section in the manual page;
3. EXAMPLES section in the manual;
4. run the application.
If your operating system of choice has inadequate manual pages because developers say "I built in --long-options because I didn't want to figure out how to write a proper manual page", it's time to seriously contemplate ditching such a system, and replacing it with a real UNIX. At that point, when that happens, you lost and you lost big.
As for zsh completion: A program simply needs to follow a specific protocol (take input, reply with the right output), and then be registered with the "complete" command. zsh comes with autocompletion for lots of things.
So basically there is a hardcoded file with all the programs which the developers or the packagers of zsh were aware of, so that --long-options would work? So now you're assuming that everyone is using the application on the same substrate as you? What if that file isn't there on my OS? And how does that hardcoded hack scale?
Yes, I can remember short command line options of tools I use often enough, and if I don't -- my operating system substrate comes with comprehensive manual pages, so I don't have to.
I am empathically not a casual command line user that I would need --long-options because that's more intuitive, so --long-options suck because ergonomy trumps casual use.
I will never provide GNU style of long options in my software, because I deeply care about my users, and I'll go out of my way to spare them from typing in the long term. What I will do is write a comprehensive manual page that will be a faithful companion and a good friend to them on their journey. And if they just don't care and want to hack something up on the quick -- I'm not that kind of a person, and it's not the kind of a relationship I want to have with my users; they can go somewhere else if they just needed to run some application casually. My tools are designed for long term, efficient use and comfort. Let people type --no-backup all day long "because it's more intuitive". Good luck with that.
It's unnecessary, and potentially even burdensome, to provide short aliases for less commonly used options. `ls --quoting-style` doesn't need a short option — you're almost never going to need it.
>Provide a version command that can be accessed by version, --version or -v.
Don't use `-v` for that. It's true that some utilities do, but i think even more use `-V` — including most GNU tools, Python, PostgreSQL, cURL, OpenSSH, iptables, iproute2, procps-ng, just about any Symfony Console app, &c. Even if you don't need a verbosity option right now, you might some day, and it's best to have that work as expected.
>Sometimes your script will take longer to execute than people expect. Outputting something like ’Processing…’ can go a long way toward reassuring the user that their command went through.
If you do print status messages like that, make sure to send them to STDERR if your utility outputs any actual data (like a report or file listing). Same for progress meters.
>Conversely, don’t exit with a nonzero status code if your CLI didn’t encounter an error. Your cleverness will end up confusing and frustrating your users, especially if -e is set.
Using `set -e` is the 'cleverness' here. It's a bad idea in all but a few cases. Also, i'm not sure what qualifies as 'encountering an error', but i think it's perfectly reasonable to return a non-zero status in certain non-error conditions, like when `grep` doesn't find a match, or in other cases where output is valid but empty.
It may _also_ be useful to show them on the terminal, in case the user would like to see them now, and has redirected stderr somewhere other than the terminal. In that case a program might show them on /dev/tty as well.
This has bitten myself and several colleagues when trying to use grep with hadoop streaming. If any of the input splits doesn't have a match, grep returns an error code, and hadoop interprets that as a failure. We switch to awk in those cases.
To be fair, grep only does that if you explicitly pass it an option that whose only purpose is to enable that behaviour.
> Normally, the exit status is 0 if selected lines are found and 1 otherwise.
$ touch bar
$ grep foo bar
$ echo $?
1 if grep -q foo file.txt; do
echo found
fi
"Start with supporting stdin/stdout as the only input and output. This ensures that it is composable with other utilities. You may find you never need anything else"
Need to read a file? `my-cli < file`
Need to write a file? `my-cli > file`
How about read from a URL? `curl url | my-cli`
vim <(curl -fsL https://news.ycombinator.com)
Read standard input in a shell script:
#!/usr/bin/env bash
cat <&0 > $(date +%s).txtOtherwise it still might be useful, but in general it's kind of unnecessary. It adds logic to your application that it doesn't need, which violates "do one thing, and one thing very well" (albeit only a very small violation).
Sure. However, for example, GNU sort doesn't work that way. Most utilities don't work that way. Most utilities accept a file as an source of input, and most of those don't act on the inode. That's the status quo.
sort < bigfile > bigfile
$ sort "$name" | sponge "$name"
$ sort -o "$name" "$name"
I'm wondering if there's any practical use for the behaviour, or if it's worth hacking a shell such that it produces a warning/interactive confirm prompt for it (or transparently buffers to DWIM maybe?)
Picking one input to still be consumed from stdin can make sense, though. A special case of this is config files, which are almost never read from stdin (they usually have a default location or several).
There are few usage scenarios where such behavior isn't enough, like for example fsck, but even there this paradigm is flexible enough to work - for such applications could be split into an analyzer and a repair program. There is nothing stopping one from outputting a binary data stream on stdout; Lots of applications do exactly that on UNIX, compressors come to mind.
What use case have you where stdout/stdin/stderr isn't enough on an operating system family where the core paradigm of usage is that everything is a binary stream of data?
Please note that we already agree here.
> What use case have you where stdout/stdin/stderr isn't enough on an operating system family where the core paradigm of usage is that everything is a binary stream of data?
My argument isn't that I have a use case where it 'isn't enough.' My argument is that many people come to a cli utility expecting that <foo /path/to/myfile> simply works.
So are for or against that? If you're for the < /path/to/my/file argument, then we are in complete agreement. A command line application should read from stdin where that applies.
I think `foo /path/to/myfile` is what the parent was trying to say.
I want stdin to work. I also want specifying a file name to work. I think supporting both is good UI design, and follows the principle of least surprise.
This is highly useful if you want to allow composition into scripts without forcing users to dynamically build parameters. Meaning, they can use, e.g. `FILE_NAME=${FILE_NAME:-"-"}; some-util --output $FILE_NAME;` and not have to decide whether or not to use the --output parameter in their script.
I know that it could be done with a loop, but that's a little unwieldy compared to `ls | xargs my-cli`
lol, this is really turning the whole 'useless use of cat' thing on its head.
The problem with that example is parsing `ls` output.
Yeah, I left that alone to keep things simpler and was assuming `ls` was simply being used as short-hand for "some unspecified approach to outputting file names". In a context where hard-to-handle filenames are possible, you'd of course need to do something a little more robust.
`cat -- *` might be the right choice. Maybe `find . -maxdepth 1 -type f -exec cat -- {} +`
I would constrain this even further to "Write to stdout only if information is useful as input to another program."
Even useful information for a human reader can make downstream integration overbearingly complex if the information is intermingled with a lot of extraneous, albeit human-readable, information. Machine-readable layouts (structured somehow: csv, tsv, json, xml, etc.) are vastly more useful for integration.
May not always be possible to know this in advance.
Might be of interest:
https://www.google.com/search?q=the+unix+programming+environ...
http://jugad2.blogspot.in/2014/09/my-ibm-developerworks-arti...
Also, stdout and stderr can be redirected independently, and either one can be redirected to the other (cmd args 2> file, 2>&1, 1>file, 1>&2, etc.). There are even more advanced options. "man bash" has the details.
Edit: Fixed a URL typo.
No. `foo --help' should never ever write to STDERR. It breaks piping the message to `less', so I need to either juggle with file descriptors (bash) or use special syntax sugar for this juggling (zsh).
Whatever lands in `less', it's hard to be considered "input to another program", because it's mainly message for human that just happens to be paged.
> 8. Don't go for a long period without output to the user.
If cp receives a SIGINFO (see the status argument for stty(1)) sig-
nal, the current input and output file and the percentage complete
will be written to the standard output.Maybe devise a new std console channel for this integration purpose alone? "stdipc" (as in standard inter-process communication) maybe?
Conversely, a structured way of exposing program outputs and state should be the preferred way of interprocess communication.
Because of convention and deliberate design choices, on the Unix command line, these two often find themselves in conflict. In my opinion, the solution isn't to compromise human usability to support machine-consumability of outputs.
I can very easily understand the output of `ps` or `ls` and I can very easily capture their output with a subshell: `for file in $(ls)` or `for process in $(ps -opid | tail -n +2);`, etc.
The more ways there are to accomplish the same goal, the greater the number of users who can figure out some way of automating what they want to do. So yeah, make output format an option, with `--ouptut` or `--format` for those who want to use `jq` or some other tool, but strongly prefer defaulting to table output so that standard tools (awk, cut, etc.) can be integrated easily.
A CLI is fundamentally for executing from the command line. Thus, embracing the power of the command shell is best practice.
If you want a human to see it, put it on stderr.
If you want a machine to see it, put it on stdout. Also I strongly agree with making it "structured" output. At minimum, a line-oriented record output is fairly easy to process downstream.
EDIT: If it is an interactive console application (REPL), then stdout is okay i guess.
Unfortunately, with e.g. `docker ps`, I can't do as much as I'd like, because it's purely human readable by default. For example:
~ docker ps | awk '{print $7}'
PORTS
Up
ago
UpUsing something like Golang for distributing compiled binaries means that as long as they have the CLI, it will continue to work. With Ruby, Python, PHP, etc, there is absolutely no guarantee that your application will work in the future.
Why is the flags package trash?
Having a `--help` option is great as a cheat sheet, but nowadays these help outputs are much too large and hard to navigate because.
Alongside complete man pages, help commands can be helpful high level overviews focusing on common use cases with examples.
The GNU project deprecates man in favour of info. This is a category error for numerous reasons. The good news is that the documentation still exists, man can be configured to fall back to 'pinfo' or similar, and projects (Debian are particularly good about this) can schlep the Info format into a manpage.
There's an argument in favour of info: it was a hypertext document format which is, arguably, more powerful than man. On the other hand, it has a format-specific reader (info), there's a far more dominant hypertext document format (HTML), and there are utilities to provide manpages in HTML format and over a local HTML server (dwww:https://packages.debian.org/stable/doc/dwww). GNU should have bailed on this decades ago.
Several projects, notably GNOME (a GNU project) and many Red Hat utilities, lack good / updated / any manpages. This is particularly frustrating. KDE similarly fails frequently to provide manpages.
Again, Debian frequently will create and provide manpages, but they will frequently run behind package development, so new features aren't clearly documented.
Increasingly, standalone packages (say, imagemagick) don't fully document themselves in manpages. Worse are vendor utilities which lack any manpage, a useful --help or -? -h page, or anything else vaguely resembling sane practices.
This is entirely inexcuseable.
Another point that I think is valuable is: "Output one line per record data"
This is useful if your CLI that has to output data to the user. Makes it easier to interface with programs like `grep`
ps | tail -n +2 | cmd_with_just_records
ps | perl -lanE'say $F[0] if 2..eof'
But also, don't conflate options with arguments. Options should be optional (the clue is in the name) and prefixed with - (single letter options) or -- (long form options). Arguments are typically positional, but as the author says, they can become confusing if there are too many, or the ordering is non-obvious, in which case the syntax may need a rethink.
A blanket "don't use positional arguments" would be tedious though. For example, if I ignore what I say above, imagine having to type "cd --directory=foo" or "cp --source=foo --destination=bar" all the time.
Cf. `dd if=foo of=bar`
Not really disagreeing, just adding context.
Do you know which job control language of yore?
dd is horrible on purpose. It's a joke about OS/360 JCL. But today it's an internationally standardized joke. I guess that says it all.
How the hell does one direct-link to a G+ comment? Well, cllck the thing that says "View 384 bazillion previous comments" and then search for dd.
https://github.com/yargs/yargs
https://github.com/SBoudrias/Inquirer.js/
When using these tools correctly (following readme), you get most of the stuff written in the article out of the box.
They should print a usage statement if I haven't provided enough info.
> 1. Every option that can have a default option should have a default option.
The exception to this are output files. A program should only output to a path that has been explicitly given to it. Too many scripts litter their working directory with output files.
> 8. Don't go for a long period without output to the user.
If there nothing's wrong, then nothing needs to be printed. Especially once a script gets called from another script, I don't want to have any output from a script unless it requires my attention.
I lean this way, too. That said, a rare "this may take a while" before things that may take a while might do more good than harm. Another option is letting the user ask for a status update (maybe with a signal, like dd?).
This is the fundamental reason why I've grown tiresome of the phrase "best practices". They probably are the best you can come up with, but that doesn't mean they're the best (or sometimes even close) for everyone else.
I've only written a handful of small CLI utils, and each time I end up looking through tools I regularly use and adapt ideas from each. But this is more time consuming than being able to run through a list of suggestions prepared by someone with more experience in this domain.
Some argument-handling libraries like Python's argparse and PHP's Symfony Console generate usage help for you, and they all use a relatively similar formatting that i think was inspired by the formatting of man pages: http://man7.org/linux/man-pages/man7/man-pages.7.html
docopt (http://docopt.org/) is an attempt to standardise it from the other way around — instead of you defining the options in the code and it generating the usage help for you, you write the usage help and it generates the code to define the options. But i don't know that it's super widely used.
If it really is a command, then require a command without the -- or -.
Today's examples:
1. centos6/redhat6 /sbin/chkconfig; it needs one of --list or --del or ... so why not just list or del or ?
2. the kafka command line tools, e.g. kafka-topics. It requires --create or --delete or --list ... So just use create/delete/list/... !
That's all good and everything but what about the parameters for which optional is only the explicit digression from an existing default value?
- Env vars provide a key/value interface, which is trickier to handle with arguments (e.g. "get the argument immediately after '-f'").
- Env vars are in-scope at the point they're needed, whereas non-trivial arguments usually have to be parsed up-front all at once and passed into the processing code somehow. Yes, it's good to check things up-front; we can check mandatory/mutually-exclusive env vars up front too, with the bonus that we can keep such 'invocation checking' separated from the rest of the processing.
- Setting env vars in preparation for a subprocess call can be done incrementally, by extending the environment over and over until we're ready. In contrast, commandline arguments must be accumulated into a single (correctly parsable) array.
- Sharing env vars across a program and its children is easy (e.g. "VERBOSE=1 ./foo.sh").
- We can use env vars to control a program even when it's called deep in the bowels of something else, without having to fiddle with all of the layers in between.
I find this works very nicely, as long as we:
- Treat the environment as append-only, i.e. no changing a variable once it's been set. We can append new variables, e.g. "export FOO=bar", and we can invoke processes with extra variables, e.g. "FOO=bar ./baz.sh", but we don't unset or alter a variable.
- Don't use env vars for communication within a process, i.e. we can use the environment variables we're invoked with, and we can append new variables for use by the programs we invoke, but within a program we should refrain from setting variables which affect our own execution (since that causes the coupling and spooky-action-at-a-distance which globals are notorious for).
What are others' thoughts on this?
That said, I agree that it can work quite nicely in the small, but it should probably be reserved for things like VERBOSE where you are likely to want propagation.
> Env vars provide a key/value interface, which is trickier to handle with arguments
Many (most? all?) languages practical for assembling a shell utility have a getopt-ish library that should make this easier (though still likely not as easy as grabbing from a shell var, granted).
Is there a language commonly used for cli tools that doesn't have an inbuilt options lib? Even bash has inbuilt 'getopts' to handle this for you.
while getopts "ab:" option; do
case $option in
a) VAR="foo";;
b) VAR2="$OPTARG";;
esac
done
shift $((OPTIND-1))
By all means, use env vars if it works for you, but it should be right tool for the right job. Imagine trying to use 'ls' with env vars...
Yes, namely C/C++ under the Visual Studio toolchain: https://stackoverflow.com/a/12689342
Two of the primary diverging Linux/Unix standards are FreeBSD (arguably the older, from AT&T & BSD traditions) and GNU.
The GNU programming standards specifies standards for command line interfaces in section 4.7, strongly influenced by GNU getopt(3). https://www.gnu.org/prep/standards/html_node/Command_002dLin...
This references the Table of Long Options (this needs to be worked in to Game of Thrones): https://www.gnu.org/prep/standards/html_node/Option-Table.ht...
For FreeBSD, the equivalent guide appears to be (I'm not an expert at this, so salt appropriately) the kernel source file style guide: https://www.freebsd.org/cgi/man.cgi?query=style&sektion=9&ma...
There may be another source.
I've seen other significant projects impose their own argument styles. Among those:
MIT/X11 has its own family of arguments.
Various toolkits seem to engender their own styles.
The GNOME and KDE projects have tended to both adopt idiosyncratic and largely undocumented argument formats. I find both tremendously frustrating.
Major applications often develop yet more idiosyncratic command and argument syntax. Chrome, Firefox, and LibreOffice come to mind.
I'm going to pretend Java doesn't exist at all. Nope. It's a myth.
The notorious 'dd' owes its syntax to mainframe JCL notation, from which it is derived.
i.e., the auto-completion facilities are general / extensible.
Especially if many programs follow a general format of
program subcommand arg arg arg --optional-flag --option
then it should be easy to see how a small specification of what subcommands take what for args or options should be enough to enable a pretty powerful auto-complete. (This is an example; I think zsh's autocompleters are actually small scripts; see https://github.com/zsh-users/zsh-completions/blob/master/zsh...)
(This, in zsh, combined with zsh's fuzzy autocomplete, is amazing.)
$ help complete
complete: complete [-abcdefgjksuv] [-pr] [-DE] [-o option] [-A action] [-G globpat] [-W wordlist] [-F function] [-C command] [-X filterpat] [-P prefix] [-S suffix] [name ...]
Specify how arguments are to be completed by Readline.
For each NAME, specify how arguments are to be completed. If no options
are supplied, existing completion specifications are printed in a way that
allows them to be reused as input.
Options:
-p print existing completion specifications in a reusable format
-r remove a completion specification for each NAME, or, if no
NAMEs are supplied, all completion specifications
-D apply the completions and actions as the default for commands
without any specific completion defined
-E apply the completions and actions to "empty" commands --
completion attempted on a blank line
When completion is attempted, the actions are applied in the order the
uppercase-letter options are listed above. The -D option takes
precedence over -E.
Exit Status:
Returns success unless an invalid option is supplied or an error occurs.
Note that it's settings for a specific shell process, not global across all instances of bash. "The right file in the right location" is only relevant 1) to set up your shells a particular way by default, and 2) if the function called references a file.
The theory here is that these tools aren't well-known, so it's an extra level of safety. It also means that colleagues don't have to hunt me down to ask about what it does...
> 3. Use common command line options.
The GNU options standards are good, but see also the "Command-Line Options" chapter in "The Art Of Unix Programming" for some of the reasoning and history behind these conventions. Actually -- if you're writing unix programs often, do yourself a favor and read all of TAOUP. It really helps crystallize the Unix Philosophy in a way that a grab bag of suggestions doesn't.
http://www.catb.org/esr/writings/taoup/html/ch10s05.html
> 4. Provide options for explicitly identifying the files to process.
Lots of options pointing to different files is a CLI smell. If your program "does one thing and does it well" (the Unix Philosophy), it often works like a filter: take one or more input files, apply the same transformation to them, output the result. `cat`, `head`, `tail`, `grep`, `cut`, `sort`, `join`, `sed`, `gzip`, `tar`, `cc`, `curl`, `md5sum` -- at the heart of all these programs is a unix filter.
In this case, just have your program accept input files as positional arguments beyond the last option. A nice side effect is that your program will be `find | xargs` -friendly. That puts the user in total control of the inputs. It also means they can parallelize your program with `xargs -P`!
Also consider whether you need input files at all! Can your unix filter program just transform stdin to stdout? If so, it can be used to process streams of data much larger than available disk space. Often you can trick programs that require a file argument by passing `/dev/stdin`, but note that `/dev/stdin` won't be accessible in some environments.
> 8. Don't go for a long period without output to the user... Outputting something like ’Processing…’ can go a long way toward reassuring the user that their command went through.
I get the reasoning behind this, but as a heavy CLI user I'd prefer if programs didn't crap progress all over the terminal by default. If I want verbose progress, I'll pass `-v`. If I want even more insight into what's happening, I'll pass `-v -v`.
See Rule Number 11 in TAOUP:
Rule of Silence: When a program has nothing surprising to say, it should say nothing.
http://www.catb.org/esr/writings/taoup/html/ch01s06.html
> 10. For long running operations, allow the user to recover at a failure point if possible.
One way to perform a sequence of expensive steps iff they haven't been done yet ("recover at failure points"): orchestrate it with a Makefile.
>I get the reasoning behind this, but as a heavy CLI user I'd prefer if programs didn't crap progress all over the terminal by default. If I want verbose progress, I'll pass `-v`. If I want even more insight into what's happening, I'll pass `-v -v`.
>See Rule Number 11 in TAOUP:
>Rule of Silence: When a program has nothing surprising to say, it should say nothing.
Agreed. In fact, informally, that rule goes back to much before TAOUP was written, I think. (TAOUP may have only formalized it.) I remember something like it from classic The Unix Programming Environment book (UPE, I first read it several years ago) which I referred to in another comment in this thread.
IIRC, in UPE it may have been phrased a bit differently, that's all - something like:
if the program succeeds (in some cases, don't generate any output, at least on stderr - of course if the program generates output as part of its normal behavior, like filters and some other programs do, then it has to write to stdout even if no errors). The behavior of the cmp command (compare two files) is an example of that - it produces no output on a successful compare (the files match), only on an unsuccessful one (the files differ).
Also, part of the reason for the brevity of Unix commands and terseness of output, is supposed to be because the first Unix versions were actually developed on teletypes (which were like the old telex machines) for output - that actually printed the output as you typed commands, on rolls of paper. Video screens came later. And the same is the reason for the Unix editor ed's brevity, and even it's p(rint) command - you actually had to (physically) print the changed lines after an edit, to even see them ... :-) All in all, I'd say it was an even more incredible job to develop an OS like Unix under such constraints.
There is also pipe viewer. Peteris Krumins wrote a post about it a while ago:
http://www.catonmat.net/blog/unix-utilities-pipe-viewer/
You insert it at points in the pipeline and it lets you see the progress of the pipe.
And totally agree with your recommendation of TAOUP. The wording can be a little ornate / verbose at times, but that is just ESR's style. Worth tolerating it for the content.
1. I personally prefer to present a terse message to stderr and stack traces to a debug file. This helps a user give support more information when they're hitting transient issues.
2. I fully agree with the recommendation for a --dry_run option, but I think the output needs to be actionable. Do people have good examples of actionable --dry_run output that lets the user verify their intent? E.g. The list of files that would be deleted by "rm -rf"
I'd argue that good software has what I call "good factory defaults".
Pine was a good example of this. It allowed people to use email right out of the box with plenty of options for customization in it's config settings.
Defaults should be sane, non-destructive, intuitive, and suit the common case. People don't change defaults, and actions which can be destructive should not be easy to invoke accidentally. Even (or especially) on a CLI.