What If I Don't Actually Like My Users? (2008) | Hacker News Reader

What If I Don't Actually Like My Users? (2008)

ozlabs.org

117 pointsJoshTriplett51 comments

I once had to use a C API where you had to guess the size of of the buffer of data you were going to get back, and keep guessing until your number was compatible with the amount of data. So I did what any good CS person would do, and started with a small, reasonable number, then doubled it until the API was happy. Was this documented? No. Everything but how you were supposed to specify the buffer size was documented. However, this code worked in production, and AFAIK, it's still running in production to this day.

To be fair, without knowing what that API does, I could see a situation where it itself doesn't know how long the data would be. Parsing free-form char-terminated data, especially over the network, is one example that comes to mind, and another is decompressing compressed data in a self-delimited stream; in general, data whose length cannot be determined apriori. Furthermore, the API probably doesn't want to allocate and buffer the data itself since it could be very long, nor does it want to impose a fixed limit since the application should decide that.

To be fair, without knowing what that API does, I could see a situation where it itself doesn't know how long the data would be.

This was to get a message out of the API. At the point one is making the call, it does know how long the message is. (Having already gotten a callback.) It looks like the author forgot to write the function that would return the length, then improvised.

Furthermore, the API probably doesn't want to allocate and buffer the data itself since it could be very long, nor does it want to impose a fixed limit since the application should decide that.

This was probably the case.

Lots of win32 API calls work this way ><

IIRC, those generally take a pointer to the buffer length, and return an error code and populate the parameter if it's too small.

I remember doing similar things when using ReadProcessMemory https://msdn.microsoft.com/en-us/library/windows/desktop/ms6...

Like which one? Pretty much every single one I know, you pass a buffer and a pointer to a buffer size; the first time you call with buffer NULL and the required buffer size is put in the buffer size; then you allocate the buffer and call again with the proper buffer allocated.

QueryDosDevice? You're right that mostly length is passed back, but esp in libs you get this problem you pass memory you own, some underlying function fails with "buffer too small", but buffer length was not in/out in the interface. Places where this can happen are not always obvious from reading the msdn.

Win32 actually gives you a way of getting lengths most of the time, like this example:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

... but the corresponding POSIX one doesn't:

http://pubs.opengroup.org/onlinepubs/009695399/functions/get...

http://insanecoding.blogspot.ca/2007/11/pathmax-simply-isnt....

That's actually semi-polite of the API. You can hide packet reassembly behind a facade in that way.

Note the "Hard to Misuse" list is at http://ozlabs.org/~rusty/index.cgi/tech/2008-03-30.html

> Here's my favorite (now fixed) example, from the glibc snprintf man page:

Not fixed is that the ISO C wide character version of the function (swprintf) specifies a gratuitously different return value convention from from that of snprintf.

I think git usually sits around a -6 or -7.

> because I know I'm not as smart as I think I am.

They should print that into every mirror manufactured.

Reminds me of a comment included in code that I wrote a couple weeks back:

  // ... Function strcmp() returns 0 on match, therefore, !strcmp() is TRUE on match.

I deliberated with my inner voice for a minute on whether to include a comment or not. Ended up including the above thinking 18 !strcmp() statements looked strangely illogical in context even after just writing it.

I always go with strcmp(...) == 0 instead of !strcmp(...).

This better reinforces my mental model of it (like many comparison functions) having 3 possible return values: <0, =0, or >0. !strcmp(...) may be terser, but it is illogical in that logical not isn't an operator. I dig using it on booleans, or "booleanish" values (null vs non-null pointers), but this conceptual tri-state isn't even "booleanish". But C didn't have a proper bool type for awhile, and C++ inherited it's baggage, so they'll compile it.

Meanwhile C# doesn't even think my "booleanish" tests are logical enough to allow to compile - no null reference tests with "operator!" for me.

> -10. It's impossible to get right.

char * gets(char * s) ?

Absolutely. Also any interface in C or POSIX specified to return a pointer to local static memory, such as "char * basename(char * path)" (and dirname), or "char * ptsname(int fd)", or "char * asctime(const struct tm * tm);".

Don't most implementations at least use TLS for their implementations to preserve a modicum of sanity there?

Some of them do, though not all. And that also doesn't fix problems with signals and similar.

But for all of those functions, glibc and some other C libraries provide reentrant variants like "basename_r" and "ptsname_r".

There's also the write call that succeeds as soon as it can write a single byte, regardless of the byte count passed in.

For example, https://en.m.wikipedia.org/wiki/Write_(system_call)#POSIX_us... warns "The write function returns the number of bytes successfully written into the array, which may at times be less than the specified nbytes"

Yet, the example following it (https://en.m.wikipedia.org/wiki/Write_(system_call)#Usage_Ex...) doesn't handle that case or error codes such as EINTR.

I remember reading a text describing how to properly use that call, but cannot find it anymore. Faulty examples like the one above are easy to find, though. For example, http://stackoverflow.com/questions/14417806/are-posix-read-a... and http://stackoverflow.com/questions/10650861/atomicity-of-wri... are good questions, but no answer points out that, even if write were atomic, it would not do you any good, as you may have to call it multiple times.

It probably is not that difficult, but I think it would take me a day to wrap this function in something that doesn't explode in your face, and be confident that I did it correctly, and even then, I would wake up in bed wondering whether I handled all edge cases correctly.

gnulib has a decent set of safety wrappers around read(2)/write(2).

safe_rw() http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=li...

This handles a single read/write, handles EINTER, and returns the amount that was successfully read/written.

full_rw() http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=li...

A wrapper around safe_rw() that handles advancing the pointer through the buffer, calling safe_rw() until the requested COUNT bytes have been sent/received.

In most cases, I recommend avoiding all of this and using the existing fopen/fread/fwrite/etc buffered I/O interface.

I can't say this is true in all cases, but I believe that fopen()/fread()/frwrite()/fclose() all cover this for you. You can get the file number for an opened FILE * block thru something like fileno() or filenum() or something.

The f() calls all buffer for you.

The problem is that write() is entangled in the ioctl() universe in many implementations of 'C' and that gets funky fast.

Beyond* that, reading and writing may involve making your own furniture to hide the ugly from dealing with peripherals.

You can also use fdopen to get a FILE* from a file descriptor, and then use the buffering and read/write logic of the C library, rather than having to write your own write_in_full function.

gets can be fixed by specifying some constant like GETS_MAX in some header file such as <limits.h>, along with the promise that gets doesn't write any more than GETS_MAX characters, including the null terminator, to its target buffer, and that the null terminator is always included. Then programs can just include the header and declare arrays of at least that size if they pass them to gets. Presto; no overflow.

I heard this idea was toyed with by the ISO C committee. Evidently, D. Gwyn proposed BUFSIZ to be re-used for this purpose.

If you're going to modify the code anyway, to include the header and set the buffer size, why not just use fgets, which takes a buffer size?

Current versions of glibc actually mark gets as deprecated (so you'll get a warning if you use it), and don't provide it at all if building for C11 or newer.

Good questions.

Since I don't use gets, I have to imagine myself fixing some legacy code.

Firstly, I imagine that this code already defines the arrays, so all I have to do is edit the definition to use this use this fictitious GETS_MAX constant, for which I have to include one simple header (which might already be included for something else). No actual logic has changed. (On the other hand, the adjustment to the size of the array to GETS_MAX could be a problematic issue; it is not without risk of something breaking.)

To use fgets, I have to edit the calls to add the size parameter. Moreover, I have to scan the buffer for the presence of a newline character and overwrite it with a null. gets does that and the code may be relying on it (using "relying" ironically, with regard to gets).

Not completely impossible (on machines with MMUs). You could intentionally allocate a guard page to signal a buffer overrun, but then you are getting outside things defined by the C standard.

> -8. The compiler will warn if you get it right.

> bind() socket library call ... takes a struct sockaddr [when you are usually using a struct sockaddr_in]

Except the compiler is correct. If you want to use a polymorphic type in a generic call that requires the parent type, you should include a cast.

From the example in my man page for bind(2):

    int main(int argc, char *argv[])
    {
        int sfd;
        struct sockaddr_un my_addr;
    
        /* ... */
          
        if (bind(sfd, (struct sockaddr *) &my_addr,
                   sizeof(struct sockaddr_un)) == -1)
            handle_error("bind");

        /* ,,, */

I don't think the complaint is that the compiler won't accept a struct sockaddr_in* where the API expects a struct sockaddr* . I think the complaint is that the API is designed in such a way that obtaining the required struct sockaddr* involves casting between incompatible types. There are more robust ways of doing type erasure, even in C.

> I've been coding in C for about 20 years, and about five years ago I spent an hour chasing a case where I'd done if (strcmp(arg, "foo")) instead of if (!strcmp(arg, "foo")).

This felt completely intuitive to me the first time I saw it: a comparison of characters can result in "less", "greater", or "equal" much like a subtraction would, and if this is expressed as an integer, < 0, > 0, and == 0 are a very sensible set of possible alternatives, and of those the only one that makes most sense as "equality" is the == 0, which is idiomatically expressed as !. I think if(strcmp(...)), if(!strcmp(...)), and ditto for memcmp() and related "3-value" comparison idioms might be one of those things that divide programmers into two groups: one that instantly sees the elegance of this design and thinks it's so natural and obvious, and one that gets eternally confused by it.

Visually it is confusing. The goal is to test for equality but there is the "!" (not) operator in there, so my brian translates it quickly to "not equal"

That is why I always write == 0 on comparison even if it is longer. It removes a negative and makes it clear.

> The goal is to test for equality

If it helps, you should think of strcmp() as returning a difference. strcmp() means "string comparison", and to compare is to find differences. Thus the "not" means "no difference", just as the idiom "if(!p)" means "there is no (object pointed to by) p".

That is why I always write == 0 on comparison even if it is longer. It removes a negative and makes it clear.

Seeing "== 0" for that in C for me is actually like seeing the equivalent of "== false" in languages that have booleans: the extra comparison causes a bit of pondering at whether it was really what the programmer intended. The if(...) is already an implicit comparison with 0/nonzero. I've seen things like this too, presumably written by someone who wanted to make the comparison really explicit, but just ends up causing more confusion IMHO:

    if ((strcmp(a, b) == 0) != false) {
         /* does this code get executed when the strings are equal or different? */
    }

I'm not actually advocating for this change, but at least to me this would intuitively make sense if the function were named instead after "string contrast".

Well to each its own, or however the saying goes. Because strcmp() returns a difference, as you said, i.e. a number comparing it a number "== 0" makes most sense to me.

> "not" means "no difference

I'd rather see less "not"s, it is easier to reason about. "not" means logical expressions to me, strcmp returns numbers, I would rather keep ! away.

> "if(!p)" means "there is no (object pointed to by) p".

Ok, now we have pointer in play. strcmp really does return numbers. Why bring pointers or logic operators, doesn't that confuse things? It does for me.

> ((strcmp(a, b) == 0) != false)

I have never seen that. And would never advocate for that, why are we doing it that way again?

>> ((strcmp(a, b) == 0) != false)

>I have never seen that. And would never advocate for that, why are we doing it that way again?

I think a more reasonable example would be (with some reasonably namespaced define, blah blah blah):

    strcmp(a, b) == CMP_EQUAL;

So you're trying to imply your logical inversion prevents bad coding habits?

What happens when people actually do start using !strcmp() and then add an additional != false anyway?

Your example doesn't show that it's hard to understand what ==0 means, it shows that some people don't understand booleans, which is no less true of your method than the other.

Elegantly solving the wrong problem is not elegant, though.

strcmp is a comparison. Comparisons are intuitively boolean unless there's a clear reason to be otherwise, and "false=0, true=nonzero" is well-known. Therefore, "strcmp('foo', 'foo')" shouldn't return 0 any more than "5 == 5" should return 0.

If the function were called "strsub" or "strdiff" and was described as taking the difference of two strings, then this might be an elegant solution, but "comparison" means something specific in a programming context, and this isn't it.

In an assembly context, cmp means 'set the flags as though you subtracted but don't do the subtraction'. This particular misdesign is from before C.

I blame Ben Franklin for guessing wrong about the sign of the charge of the electron.

(I keed, I keed )

Comparisons are intuitively boolean

Are comparisons in general boolean? I think specific operators are, but that's because they are asking a particular boolean question. A general "compare X and Y" doesn't seem like it should ever be "true" or "false" (because what would those mean? Different --- in what way, less, greater, or both --- or equal?)

As the sibling comment alludes to, comparisons are really a form of subtraction to the hardware, and C makes this fact very obvious.

When you get access to a time machine, go back in time and have this argument with the poor grad student who committed this sin :)

In assembler, comparisons generally have a result of zero when things are the same because hardware works that way. If the difference between two voltages is zero, they "null" and are the same.

In a pidgin, non-real assembler I'm making up:

MOV AX,A_VARIABLE

MOV BX,B_VARIABLE

CMP AX,BX ;; or maybe AND AX,BX or XOR AX,BX because flags

JNE THEY_ARE_NOT_EQUAL

JMP THEY_ARE_EQUAL

...

THEY_ARE_EQUAL:

...

THEY_ARE_NOT_EQUAL:

In 'C' if I am really concerned about this, I will:

const bool TheThingMatches = (strcmp(a,b)==0);

Edit: It ate my line breaks.

> Comparisons are intuitively boolean unless there's a clear reason to be otherwise

To support sorting. The result of a "comparison" does not imply the testing for equality or equivalence. I can "compare" lots of things, and my results are not strictly boolean. A "comparison" often implies one thing is "better" or "worse" or the "same" as another thing.

>one of those things that divide programmers into two groups: one that instantly sees the elegance of this design and thinks it's so natural and obvious, and one that gets eternally confused by it.

Believe it or not, there are people who are capable of figuring out C's cute little "round trip through the integers" trick, but who would rather not deal with that bullshit every time they want to compare two values of non-numerical type. It's not elegant, it's not natural, it's only "obvious" to those of us who have had the standard workarounds for C's inexpressiveness drilled into our skulls, and if it were any different, we wouldn't bother with comparison operators in the first place!

I mean, I get that it's fun to squeeze useful semantics out of terse code, especially in an environment as constrained as C. But please don't act like the rest of us are stupid just because what you regard as elegant, we regard as more of a clever hack to be put up with for want of a simple equality operator.

compare two values of non-numerical type

everything is "numerical" or can be interpreted as such. It's not C-specific at all, it's just a fact of how computers fundamentally operate. I suspect that might be why this issue is so divisive: if you started low-level with hardware, Asm and the like, then C does feel natural and obvious. If you started high-level with some other HLL, one that abstracts and hides this "essence", it won't.

Pointers are another example of something that is obvious to some but a huge mass of confusion and frustration to others. I've taught programming (mostly Asm and C), and this is quite common. You're not stupid, but you just aren't in the right perspective to understand it.

Those who are strong advocates of functional programming and wonder why others don't see the elegance probably think the same way about me. ;-)

>Pointers are another example of something that is obvious to some but a huge mass of confusion and frustration to others. I've taught programming (mostly Asm and C), and this is quite common. You're not stupid, but you just aren't in the right perspective to understand it.

I understand pointers. I understand integer-based encoding. I'm a firmware programmer, I started with FORTRAN and C-with-classes-style C++, I get how computers work. And do you know what makes computers work? Abstractions. Integers are just an abstraction on top of bit vectors are just an abstraction on top of memory are just an abstraction on top of flip-flops, on top of gates, transistors, digital circuitry, analog circuitry, the laws of electromagnetism -- the only reason we are able to construct something as complex as a pocket calculator and have it work at all is clear, mathematical abstraction. The fact that, in 2016, using such a fundamental abstraction as equality of non-numerical values requires anything beyond the two characters "==" is patently absurd.

My "frustration" is not a result of having trouble grasping first-year C.S. concepts. Nor is it the result of the mild incovenience of specifically having to use "strcmp" instead of "==". It's the result of working in an industry where the value of anything beyond first-year C.S. concepts is completely unrecognized, because it's not "how computers fundamentally operate." Both low-level and high-level computational abstractions are useful, and trying to accomplish anything interesting in a system which completely eschews one class of abstraction in favor of the other, whether it's C or Haskell, is hell.

But hey, it beats the hell out of Javascript :)

I don't think he was calling you stupid, podner :)

I just think "ah, it's a yes we have no bananas" and stop thinking about it :) It's nonzero cognitive load but it's not much.

I've dealt with enough active-low/inverting hardware such that "on" = 0 that it actually doesn't bother me at all. Assembly has that all through it at times.

Compared to reading code where people treat cell [1] as the first significant cell of an array ( usually translated from FORTRAN ) , this is a picnic.

For another take on my other post: Would you expect "1 == 1" to return 0? Would you expect "3 == 7" to return -4? Why or why not?

Even if so, it should have been a separate type. The "elegance" of this design can be complemented by having a better interface. My strawman example:

    /* This is for making a type that can be compared but not calculated.
     * Use it like `(const void *) _StrcmpResult[i]` for i = 0, 1 or 2.
     * i = 0 indicates less, 1 indicates equal, and 2 indicates greater. */
    extern const char _StrcmpResult[2];
    const void *_Strcmp(const char *a, const char *b);
    #define strcmp(a, op, b) (_Strcmp((a), (b)) op (const void*) &_StrcmpResult[1])

Now you are almost unable to incorrectly use this interface: `strcmp(a, <, b)` and `strcmp(a, !=, b)` etc. works, `strcmp(a, -, b)` does not compile (because we've casted to `const void*`), `strcmp(a, =, b)` also does not compile (because `_Strcmp()` is not a lvalue). Other incorrect uses will likely result in compile errors, though in possibly more obscure ways.

In 1989, I added streq(a,b), which was essentially !strcmp to my companies standard library because people kept making this error.

I tallied about a -11 using the Youtube Player API today.

Reply on news.ycombinator.com