Hacker News Reader

Comment by peterwwillis | Hacker News Reader

And this has led to de facto standards, like '#!/bin/sh' is a 'standard' shell shebang, but not POSIX; if you want your script to be portable, you have to use '#!/usr/bin/env sh'. env is forever in '/usr/bin' probably because it first came about after new tools started moving to '/usr'. And I don't think env's file path is even in POSIX, it's just a de facto standard.

Another fun fact is that execve() (which executes shebangs) only allows a single argument to the shebang program. Meaning if you use '#!/usr/bin/env sh', you can't specify arguments to 'sh' in the shebang. In any case, the name of the file being executed is appended as an additional argument at the end of the shebang.

View on HN

env hasn't always been in /usr/bin. I've worked on systems (probably SunOS 4.something) that had /bin/env but not /usr/bin/env. But you're unlikely to run into any reasonably modern Unix-like system that doesn't have /usr/bin/env .

See also my Stack Exchange answer on the advantages and disadvantages of the "#!/usr/bin/env ..." convention: https://unix.stackexchange.com/a/29620/10454

Quick summary: The advantage of "#!/usr/bin/env INTERP" is that it will use the first INTERP in the user's $PATH. The disadvantage is that it will use the first INTERP in the user's $PATH.

If neither /bin/sh nor /usr/bin/env are standard but both are defacto standards, then where is the relative value of `#!/usr/bin/env sh` over `#!/bin/sh` ?

https://unix.stackexchange.com/a/29620/10454

As a data point, my phone has /bin/sh but not /usr/bin/env. But not many shell scripts need to target Android, probably.

env is part of POSIX, so I guess your phone just doesn’t make the symlink. (If it’s busybox, perhaps it still has the applet compiled in?)

It has /bin/env, it just doesn't have /usr.

> And I don't think env's file path is even in POSIX, it's just a de facto standard.

You would be correct about that. And that sucks.

There's some debate about this if I recall correctly from the last deep dive I did about the pros and cons of it.

If I am writing a script that needs to be that portable, then I'll use #!/usr/bin/env sh but otherwise /bin/sh will work in all the cases I need it to.

Since bash's path might vary more, I might consider calling env to locate it then more often.

If you're writing a script that needs to be that portable, you're doomed anyway and you might even find the autoconf manual's "Limitations of Shell Builtins" section useful.

If you are using /bin/sh you need to check sure it is a POSIX shell script and not using bash features [1]. On Debian and derivatives /bin/sh is ash not bash.

[1] https://linux.die.net/man/1/checkbashisms

I'm not aware of anywhere that /bin/sh won't work. /bin/bash is extremely non-portable though.

/bin/bash is popular in Linux systems, but not BSD or Mac or probably other Unixes which may not always have bash and if they do have it, it will be somewhere else. Also, a number of niche-ish Linux distributions are trying to replace bash with something else. It's no longer as portable as it once was.

It's not just about whether it exists but also the version.

For example, on my (Mac) laptop, /bin/bash is bash 3.2.57 (released in 2007), while "/usr/bin/env bash" gets the version installed by homebrew which is 4.4.19 (released in 2016).

POSIX does not specify the path of anything. It says that "sh" is a shell, but doesn't say that "/bin/sh" is that shell. See: https://pubs.opengroup.org/onlinepubs/9699919799/

I believe that in Solaris before Solaris 10, /bin/sh was the original Bourne shell (which didn't comply with POSIX), and in Solaris 10 it's ksh (which isn't either, but is way closer): https://unix.stackexchange.com/questions/538844/does-posix-g...

Also, I don't think that "#!" is specified in POSIX. It's extremely common, it's just not in the specification. This is similar to find's "-print0" option, which also isn't in the POSIX specification but basically everyone implements it.

If you want a specification of "where things go", the closest one in practice (and one that most modern distributions try to follow) is the Filesystem Hierarchy Standard (FHS): https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html

(providing more detailed references)

POSIX specifically calls out that it doesn't guarantee that the path is "/bin/sh":

    107271  Applications should note that the standard PATH to the shell cannot be assumed to be either
    107272  /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by
    107273  getconf PATH, ensuring that the returned pathname is an absolute pathname and not a shell
    107274  built-in.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/s...

POSIX mentions "#!" several times, but doesn't require it; saying things like "on systems that support executable scripts (the "#!" construct)".

As the lack of a hard-codable path is frustrating, it recommends setting the shebang at install-time:

    107279  Furthermore, on systems that support executable scripts (the "#!" construct), it is
    107280  recommended that applications using executable scripts install them using getconf PATH to
    107281  determine the shell pathname and update the "#!" script appropriately as it is being installed
    107282  (for example, with sed). For example:

As I wrote at https://unix.stackexchange.com/a/567405/5132 , I have a strong suspicion that Arch Linux dropping the FHS is the tipping point, and adherence to the FHS is now the exception not the norm.

Why would Arch dropping the FHS be the tipping point? Surely Arch represents hackers and not mainstream Linux users. In much the same way I wouldn't cite Linux users are a tipping point in any computer system convention (but Mac OS or Android or Windows), I wouldn't cite Arch in a Linux system convention (but Debian or Ubuntu or Red Hat). Why do you consider Arch as the tipping point?

Because it would be, if my suspicion proves to be the case, the point where adherence to the FHS tips over from being the norm to the exception. MacOS, Windows, and who Arch users are have nothing to do with it. How many distribution creators have made each of the various choices, does. Arch's decision to use systemd had some influence (in both directions, c.f. Hyperbola Linux). Arch's decision to abandon the FHS in favour of the systemd file-hierarchy may well do likewise.

Reply on news.ycombinator.com