Hacker News Reader

Comment by scythe | Hacker News Reader

Okay, so... from UNIX v5, OpenBSD added a -n flag that prevents a trailing newline, Plan 9 adds the -n flag and pushes the argv into a buffer (why?) before printing, FreeBSD does all that and also prevents a trailing newline if the last argument ends with "\c" (why?), and GNU does... something complicated.

View on HN

"adds the argv into a buffer (why?)"

It's calling "write", which is a system call. Without the buffer, it would call write once for argv, and once for the newline if nflag is not set. Calling a system call twice would result in twice as many context switches, and thus be very slightly slower.

Oh, that makes sense.

Speaking of clever tricks, would it be faster or slower to use c99's variable-length arrays instead of malloc?

IIRC faster, because variable-length arrays in c99 just bump down the stack pointer, while malloc can be pretty expensive.

But a single malloc is nothing compared to the cost of a context switch into kernel mode.

You can simply use alloca() as a replacement for malloc() to avoid depending on VLA support.

Or just declare a large static buffer.

"Nobody will ever echo this much data!"

Oh wait they will

It's actually not a bad suggestion

    #define BUF_INITIAL 1024
    char buf[BUF_INITIAL];
    
    int main(int argc, char** argv)
    {
        char* p;
        ...
        p = len > BUF_INITIAL ? malloc(len) : buf;
        ...
        write(1, p, len);
        ...
    }

djb's allocator alloc() does this. He preallocates a 4 KB static buffer before hitting system malloc(). Avoiding the overhead of malloc() is pretty important for the performance of systems like qmail that fork many small processes.

Many small processes? How many?

4 KB * (many) could be frightening.

If your system has 32,000 processes and you allocate 4 KB to each of them... that's 128 MB. I'm not crapping my pants at that figure because even the oldest machine in my office, an old Thinkpad, has 2 GB of RAM.

That's ridiculous. The overhead of fork is about a bajillion times higher than malloc.

Are you sure? I'd bet it's greater, I'd not bet that it's an order of magnitude better, fork() has been optimized a lot more than brk().

With a COW fork() I bet it's smaller. I smell a test coming on, but alas it's late here and I'm going to bed.

I'm also guessing that 4k was chosen because a malloc() of 1 page is faster than a malloc() of >1 page. Of course that's with the assumption that the systems use a 4k page size.

alloca sucks. The error handling is "haha, you overflowed your stack." (VLA sucks for the same reason.)

Isn't that what stdio is for (fopen()/fread()/fwrite()/etc)? It performs buffering for you.

Stdio would add unnecessary overhead. And why bother with stdio (or its Plan 9 equivalent) when it's so easy to do the buffering faster yourself?

Overhead such as...? The only one I can think of is the mutex lock/unlock, but there are the _unlocked variants for those.

"And why bother with stdio (or its Plan 9 equivalent) when it's so easy to do the buffering faster yourself?"

For the same reason you'd use any abstraction: to keep your program simpler. Even if you consider it "so easy" there are plenty of opportunities there for off-by-one errors and buffer overflows that would disappear if you used stdio instead.

Please point out the plentiful opportunities for off-by-one errors in Plan 9's echo.c. If you send me a copy of Plan 9's echo.c using stdio I will run it and benchmark it for you.

Remember that echo.c evolved long before UNIX has shared libraries. So avoiding stdio completely might mean the size of your binary drops from 50K to 5K. If you're on a PDP-11 with 256KB of RAM (being shared by several users) this would make a big difference.

My earliest UNIX programming was in the mid-80s on a machine with a (luxurious!) 1.5MB of RAM. I definitely remember avoiding stdio when writing small utilities that I wanted to start up and run fast.

Also remember that back in those days echo was not a shell builtin. (Hell, back then the testing operator '[' wasn't even a builtin. Some UNIXes like OS/X still have a vestigial /bin/[ executable!) Programs like echo that ran from shell scripts constantly had to be coded to start up as fast as possible.

> Some UNIXes like OS/X still have a vestigial /bin/[ executable!

This includes linux, but it's in /usr/bin/[

This was actually the reason for FreeBSD's more complicated version. Revision 106835:

    Put echo on a diet, removing unnecessary use of stdio and getopt.
    
    Before...
    -r-xr-xr-x  1 root  wheel  58636 Oct 28 05:16 /bin/echo
    After...
    -rwxr-xr-x  1 root  wheel  12824 Nov 12 17:39 /usr/obj/usr/src/bin/echo/echo

http://svnweb.freebsd.org/base?view=revision&revision=10...

If speed is important, why the copying of the data? Only 'slightly' non-portable:

  int main (int argc, const char * argv[])
  {
    for( int i = 1; i < argc; ++i)
    {
        ((char *)argv[i])[-1] = ' ';
    }
    size_t len = strlen(argv[argc - 1]);
    ((char *)argv[argc - 1])[len] = '\n';
    write(1, argv[1], argv[argc - 1] - argv[1] + len + 1);
    return EXIT_SUCCESS;
  }

It has to do with the way plan9 guarantees atomic writes of small buffers. A single write() call will correspond to a single read() call if the buffer is big enough to receive it.

Actually, most of what the GNU is doing is recognising some escape sequences (including characters in hex form).

It seems like "-n" flag appeared in UNIX v7: http://www.bsdlover.cn/study/UnixTree/V7/usr/src/cmd/echo.c....

Plan 9's echo is used to talk to drivers from shell scripts, some of which expect to receive messages in a single buffer. That's why they went out of their way to make echo use a single write.

Reply on news.ycombinator.com