Filenames and Pathnames in Shell: How to Do It Correctly (2010)

One of the reasons I like zsh is its copious expansion flags. If you aren't wedded to POSIX or bash syntax, consider zsh for this reason alone.

In the case of the first example he shows, the cat -n problem, you can do the following in zsh to expand the results of the wildcard globbing automatically:

    cat *(:A)

This will give cat full paths. Not significantly different from the PWD referenced with ./* in his example, but more universal in applicability, particularly when you start to use things like the file type filters:

    cat *(.:A)

giving you only files, not directories, and also expanding to absolute paths, while

    cat **/*(.:A)

does the same for plain files in the working directory and all subdirectories as well.

Remember to test your patterns with a print statement first:

    print -l **/*(.:A)

before passing them to a command.

The original strength of Unix also ends up being such a commonly frustrating feature: Everything is marshalled through strings.

With human-manageable strings comes ambiguities, especially in concatenative situations like commandline expansion and SQL injection susceptible code.

There's really no good universal solution. Judiciously adding explicit boilerplate as the article describes, or using less open-ended syntax which ends up adding common syntactic overhead as well, are both more painful to the user in common cases.

Shellcheck will find a broad variety of unsafe shell operations, including most (all?) of the issues on this page: http://www.shellcheck.net/

Today I learned globbing happens after word-splitting.

Can someone explain the following:

   for file in ./* ; do        # Prefix with "./*", NEVER begin with bare "*"
     if [ -e "$file" ] ; then  # Make sure it isn't an empty match

1. Why prefix with a "./" ? Is that just to help avoid the `cat $filename` scenario? (i.e., that $filename will be "./-n" instead of "-n", and that

  cat -- *

is perfectly valid?)

2. What's the -e check for? It says "an empty match" — -e means that the file exists, but * would only return files that exist, so -e must (with some caveats) be true. (The caveat being that there's a race condition between the globbing and the test, but with the added test, there's _still_ a race condition between the glob, the test, and the command execution. Are we just attempting to minimize the amount of race-condition by testing?)

Note also that some[1] commands will have a "--" (dash dash) flag indicating "end of flags", so (eg): "cat -- -n" really would cat a file called "-n"[2].

[1] on my BSD system, many internal Tcl commands honor this convention. Damned if I can find a section 1 shell command that uses the convention, but I'm sure I've seen them.

[2] my version of cat doesn't have a -- flag, so my example is contrived; not sure if GNU cat differs.

EDIT: typo, perl(1) supports "--". See perlrun(1) for details.

First step for doing things correctly:

  import shutil

I'd love to know if there's a Windows equivalent to this.