Bug: true and false are not Posix (2016) | Hacker News Reader

Bug: true and false are not Posix (2016)

lists.gnu.org

36 pointsjraph39 comments

This is a textbook example of a bikeshed: http://bikeshed.com

Nobody is calling true or false in a script with options/arguments. Nobody hss ever seen true/false fail because they crashed in setlocale. Nobody's highest, second-highest, or 1000th-highest risk to their code is the performance of their true/false implementation.

But it's a really easy thing to have opinions about. You use UNIX for one week and suddenly you're exactly as qualified as Dennis Ritchie to have opinions on the matter.

I've seen... (ok, actually I recommended)... someone do the following:

    if [ -x /usr/bin/blah ]
    then
       BLAH=/usr/bin/blah
    else
       BLAH=/bin/true
    fi

    $BLAH foo bar baz

They wanted BLAH to be a noop if blah didn't exist. I could only think of /bin/true as a noop at the time, but I needed it to ignore parameters.

The man page defines that it does.

(in retrospect, we could have defined an sh function noop {} )

Yeah, options include a noop function (which is nice because you can change it to noop () { echo "Skipping $@" >&2 ; }) or the shell builtin :, which definitely ignores its arguments.

I was going to mention that it came up just days ago on Unix and Linux StackExchange. (-:

* https://unix.stackexchange.com/a/497561/5132

The conditionally defined shell function in one of the other answers is a more appealing approach, especially given the native behaviours of some shells with respect to variable expansion. But also given that /bin/true is not necessarily the pathname.

    % export BLAH='/bin/true too'
    % dash -c '$BLAH 1 2 3'
    % dash: 1: /bin/true: not found
    % sh -c '$BLAH 1 2 3'
    /bin/true: not found
    % ksh -c '$BLAH 1 2 3'
    ksh: /bin/true: not found
    % zsh -c '$BLAH 1 2 3'
    zsh:1: no such file or directory: /bin/true too
    % bash -c '$BLAH 1 2 3'
    bash: /bin/true: No such file or directory
    % 
    % export BLAH='/usr/bin/true too'
    % zsh -c '$BLAH 1 2 3'
    zsh:1: no such file or directory: /usr/bin/true too
    % bash -c '$BLAH 1 2 3'
    % ksh -c '$BLAH 1 2 3'
    % dash -c '$BLAH 1 2 3'
    %

Yeah, that was my answer -- and actually I wrote /usr/bin/true on SO but got sloppy here.

I wrote that answer because I recalled it as a common idiom in autoconf or something -- some automake system I'd seen where the system seeks out the tools it needs and defines them as variables.

OT but, I've always thought the true and false summaries would make great t-shirts and inspirational posters. "do nothing, successfully" sounds zen, and "do nothing, unsuccessfully" sounds like a sportswear slogan.

2017 POSIX, Section 2.1.1, paragraph 4:

"The system may provide non-standard extensions. These are features not required by POSIX.1-2017 and may include, but are not limited to:

Additional functions

[ ... ]

Additional options for standard utilities"

[ ... ]"

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...

A standard utility that takes no options can be extended locally to take options. "Some options" constitutes "additional options" relative to "no options".

Since the descriptions of true and false say they take no options; therefore POSIX-conforming scripts don't pass any. A POSIX-conforming script has no way of finding out whether true or false take implementation-defined options (i.e. any options whatsoever).

The authors of the utilities seem to make a pretty good case on how GNU’s ‘true’ and ‘false’ are POSIX compliant, because their extensions to them are covered where POSIX leaves the behavior as undefined, and in fact it seems to be the invocation of ‘true’ with any arguments itself that is not in accordance to POSIX.

Whether it’s “not common sense” is not very apparent when “common sense” is pretty difficult to define here (as often the case).

> The authors of the utilities seem to make a pretty good case on how GNU’s ‘true’ and ‘false’ are POSIX compliant, because their extensions to them are covered where POSIX leaves the behavior as undefined, and in fact it seems to be the invocation of ‘true’ with any arguments itself that is not in accordance to POSIX.

Ok, so undefined behavior is seen an escape valve through which the devs can change longstanding behavior in the interest of consistency with a newer set of interface standards they've devised. Fine.

But then this response from a dev on that mailinglist[1]:

> Note that it is a high bar to change the behavior of something like 'true'.

Hm... now I'm slightly intrigued.

Did the change that introduced the standard "--help" flag and addition of locale-related code flow directly from the "escape valve" of undefined behavior in POSIX? Or was the undefined behavior merely a starting point for a much more involved discussion about whether to change a long-standing interface that could introduce subtle bugs in old scripts/programs that relied on implementation details of undefined behavior?

Edit: clarification. Also if someone can point me to a mailing list discussion that shows evidence of the latter I would be much obliged.

[1] https://lists.gnu.org/archive/html/bug-coreutils/2016-03/msg...

The behavior of --help in GNU's true is not undefined, it is clearly defined in its manpage. Implemenations conforming to the GNU spec are a strict subset of those conforming to the POSIX spec.

Changing thr behaviour is a braking change because they are no longer conforming to their old spec. Adding the flag is not a breaking change, as they stayed consistent with the old spec

You're talking as if this were a recent change. GNU true --help has existed for about 28 years now (it was introduced in sh-utils 1.7). That's more than half of Unix's existence.

Yes, this is a very very old bug. I doubt anything depends on that bug.

That was only 3 and a bit years after the publication of POSIX 1003.1:1988 itself.

I don't see the relevance, but maybe I'm confused about the development history of this command.

Was there a time when GNU existed without the "--help" flag? Or was it originally committed with the "--help" flag present?

Careful: If you go by "longstanding behavior" instead of standards, you quickly get to the point where you have to ask yourself the question which longstanding behavior you want to honor for "consistency". This may not universally be true, but in the case of the shell(s) and everything that surrounds it, I'm pretty sure it is.

Shells are pretty much the wild west out there. I wouldn't be too surprised if some AIX shell variant or something came along and consistently bails out with an error when any argument is given to 'true', including --version. It might not be a common shell variant for you, but it might for other people. Hence, standards.

I'm not really arguing in favor of --help and --version to 'true'. I just don't see much of a problem with it.

The first and best /bin/true was an empty file — possibly the only bug-free program ever written.

The worst may be the System V version, which added a copyright notice.

That is, considering that your shell interpreter is bug free.

> possibly the only bug-free program ever written.

Oh come now. I'm quite sure my hello world program is bug free. It prints "Hello World" successfully, and there are zero other cases that could exist.

You're obviously joking, but for those who aren't aware, even a naive "hello world" can have bugs.

https://www.gnu.org/ghm/2011/paris/slides/jim-meyering-goodb...

I actually wasn't joking, but that link made me rethink, so thanks for that!

2016

true once was, and IMO should be again, implemented as a 0 bytes executable.

Since then, it has infinitely bloated on most systems.

It actually began at AT&T, where they apparently decided to copyright the trueness. From there it went downhill, ever and ever.

http://trillian.mit.edu/~jc/humor/ATT_Copyright_true.html

Doesn't the 0 byte executable involve spawning a shell to run it? Better would be a tiny elf executable that just returns 0 (maybe hand written to avoid the compiler adding bloat)

Correct. true being a 0 byte shell script is a good example of classic Unix's bad habit of doing the overly-cute thing instead of the sensible thing.

It would be a valid approach but it is less elegant and harder to maintain. Probably the kernel could special case 0 bytes executables, if you really want to optimize the scripted truth.

If you want maintainable, you should just compile "return 0" with your standard toolchain. A 0 byte file risks confusing developers who might see it and think their install is corrupted (surely the program just got truncated). If you want to use an empty program, the maintainable solution would be to comment it; but then it is not empty.

Special casing 0-byte executables is not necessarily the best solution, either. It's a special case after all, and one that's essentially only there to support /bin/true. This may create security gotchas or inadvertent bugs resulting from the "codeless" address space that gets created (or not created if you implement it that way, with its own set of potential gotchas). I'm not saying it necessarily does create problems, but it can.

And it would actually change the historical and current behavior of exec returning ENOEXEC for 0 bytes executables (see also https://news.ycombinator.com/item?id=19081124 ) . So I am changing my mind: a binary true is better. I still prefer truths as small as possible, though :)

It can be an extra-special 0-byte executable though, running far faster than anything else, like so:

/proc/bin/true

/proc/bin/false

The exec doesn't even need to create page tables or run any non-kernel code. This is a benchmark-winning design.

No it does not. The 0 byte executable involves first trying to execl() it, the OS returning ENOEXEC, and then spawning a shell.

Of course "spawning a shell" can be very cheap, as you are already in a shell, it's just fork() plus some initialization code (compared to fork() plus exec() for running a non-shell program)

Note that shells must try to interpret at least any text file for which the system returns ENOEXEC [1]

1: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3... Read subsection 1.e.i.b

Does this mean that putting #!/bin/sh at the head of scripts actually slows them down (assuming you are running from an sh like shell)?

https://stackoverflow.com/questions/25165808

I suggest measuring; there's overhead to the failed exec() call certainly. It probably depends on a lot of other factors too (which, if any rc files are read in each mode, how fast your shell is to startup &c.).

I just tested this.

Files:

1.sh:

    true

2.sh:

    #!/bin/bash
    true

test1.sh:

    for i in `seq 10000`
    do
    ./1.sh
    done

test2.sh:

    for i in `seq 10000`
    do
    ./2.sh
    done

Times for test1.sh:

    11.484s
    11.590s
    11.405s

Times for test2.sh:

    14.273s
    14.391s
    13.847s

So including the #! adds about 23% to the overhead of calling a shell script. Tested on Mint 19.1, bash.

EDIT: for reference, sourcing the scripts instead takes an average of 0.0842s for test1 and 0897s for test2. (and all test2 trials were still slower than all test1 trials), which isn't particuarly suprising.

Inlining true takes about 0.029s

Why did you use bash and not sh for the interpreter?

To keep the comparison fair. My interactive shell is bash (which I assume is a more popular choice for interactive shell then sh), and the version without any shebang will use whatever the parent shell is. To make the comparison fair, I wanted the version with the shebang to be using the same shell.

Besides, /bin/sh would also require dereferencing a symlink (to /bin/dash), which also doesn't seem fair.

Oh it is not nearly as good as I thought then. I actually tested it and zero byte exec still make exec() returns ENOEXEC on Linux. So a small binary is definitively more versatile...

Does any shell implement exec of a script as fork and reinit?

Probably Inferno, considering that it also runs proper binaries that way.

Reply on news.ycombinator.com