asdf banned_commands | Hacker News Reader

asdf banned_commands

github.com

71 pointstosh23 comments

I've spent a lot of time writing portable shell scripts which needed to run on Arch Linux (GNU coreutils), Alpine Linux (busybox), and MacOS (whatever their defaults are). Even used shellcheck to lint and ran all scripts with dash shell to validate POSIX-compliance.

Now? I just pick a minimum version of Python and limit myself to the standard library (Python package management still sucks) and occasional subprocesses (ex. preferring to invoke the zstd cli vs installing the Python package). This tends to be more verbose than shell scripting, but also much clearer - especially when compared to using single-letter flag arguments for options I seldom use.

I also don't generally trust those I share my scripts with to have a deep enough understanding of shell scripting to ensure portability and safety (argument parsing, quoting, and iterating over paths).

I also find myself reaching for Python if it’s more than a terse, readable 1-2 line script.

A colleague of mine said it pretty well - if your script is doing even minutely complex with jq, it’s probably time to reach for Python.

This is a valuable effort for portability. I had no idea `source` was not POSIX compliant but `.` was, or what flags `grep` doesn’t support on macOS.

But it is ironic that the script uses `echo` to show a message that `printf` is preferred to `echo`. I guess what is meant is that `printf` is preferred to `echo -[en]`.

echo is fine when its argument is not a variable.

The script does echo variables:

    echo "banned command $cmd: $output"

This is non-portable if there is any possibility that these variables contain backslashes.

TIL:

  dash$ x="foo\nasd"
  dash$ echo "$x"
  foo
  asd
  dash$ printf "%s\n" "$x"
  foo\nasd
  dash$

  zsh% x="foo\nasd"
  zsh% echo "$x"
  foo
  asd
  zsh% printf "%s\n" "$x"
  foo\nasd
  zsh%

  bash$ x="foo\nasd"
  bash$ echo "$x"
  foo\nasd
  bash$ printf "%s\n" "$x"
  foo\nasd
  bash$

    bash$ echo -e $x
    foo
    asd
    bash$

Enlightening

From POSIX:

>echo - write arguments to standard output

>If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined.

Wow, TIL indeed. That's conformant to the Single UNIX Specification as well: https://pubs.opengroup.org/onlinepubs/007908799/xcu/echo.htm...

The script is for static analysis, so it only needs to run once against each code change, and in theory it shouldn't need to be portable code, unless the code under test is the static analysis code itself.

In other words, they control where this script runs and there is no need to run it on more than one platform, so it's okay for it to be non-portable.

That's fine if the script only runs on shells that behave the same way, but the script runs under bats, which runs on bash and appears to not change its xpg_echo shell option. This shell option controls echo's behaviour on bash and has a compile-time-configurable default; even scripts that are only meant to run on bash cannot assume either behaviour for echo, as different vendors use different defaults for the option.

sed, grep, and find are radically different on macOS (4.4BSD + early FreeBSD + many hacks since).

Once upon a time I was writing a few portable shell scripts for Solaris and Linux distros. Limiting the scripts to a common subset of commands was fighting windmills - so often you stuck with something easily solved by one GNU option, but requiring 10+ lines of shell script with UNIXy commands on Solaris.

In the end, it was much easier to just proclaim GNU coreutils as a dependency everywhere, and use g-prefixed versions of commands (gfind, ggrep, and so on).

Even the GNU commands on Solaris were so old, and from a time where so many were changing and flakey. I could reliably make the Solaris and early Linux bash and gsed segfault. Options weren’t consistent with Linux distros or coreutils from eg ports on FreeBSD. I still have a bias against GNU from those battle scars.

I had to stick to (most of) POSIX across Unixes with ksh88/mksh/pdksh back in the day when my work had to be portable across Linux, BSD, Solaris, HP-UX, and AIX. I could rely on process substitution, etc. but stay vanilla outside the shell itself.

Honestly the only really annoying thing to me was that sed -i didn’t mean in place, but even then it wasn’t reliably atomic IIRC.

Handling the actual OS commands was much harder. Separate commands instead of getent (maybe getpw was one?) on HP-UX pops into my head.

I vaguely remember that `source` appeared in Bash at some version—presumably because it's clearer than the rather random `.`, and doesn't stand out among other built-ins for no good reason.

It's interesting they banned the "echo" command but then use it in the tests [0].

0 - https://github.com/asdf-vm/asdf/blob/master/test/banned_comm...

Banned is a bad term, these commands are non-portable or unpredictable and therefore not allowed in asdf scripts. Tests are totally different and might be constrained to run on specific platforms where you can and should use all the platform features to better run tests.

If you were as confused as I was where "run" is defined and how "output" gets set, and what the heck the bats extension is:

(1) https://github.com/bats-core/bats-core/blob/ca5a2dce94/lib/b...

(2) https://github.com/bats-core/bats-core/blob/ca5a2dce94/lib/b...

(3) bash automatic testing system.

Are these meant to be programmatically enforced via CI? If so, I wonder if adding an Alpine-based build to the pipeline would cover all the unsupported commands automatically.

That'd be a great addition!

Historically, the list of architectures that public github action runners make available has been quite limited. Having an alpine runner to surface unsupported commands would be a huge win.

That being said, it's very useful as a plugin dev to have this test suite while working on a WSL2/Ubuntu so that I don't need to run a full CI build every time

Alpine uses busybox for its shell and core utils. While closer to pure POSIX than bash or GNU, it has support for a few non-POSIX features (the shell supports the source command and brace expansion, and iirc some of the utils support non-standard flags).

It would be better to ensure POSIX shell script compliance by running the scripts with dash shell instead.

Check out also shellcheck. It's been an invaluable tool keeping my scripts portable.

Where does asdf hide bats? I don't see the source of it. Austin maybe?

Reply on news.ycombinator.com