Comment by dexen | Hacker News Reader

Comment by dexen | Hacker News Reader

Whether memset(), or any other function, gets optimized away by GCC should depend on function attributes (1) -- more exactly on the `pure'; possibly some others. However, GCC (tested with 4.7.1) somehow considers memset() pure regardles of declaration. The default declaration is:

  $ echo '#include <string.h>' | gcc -E - | grep memset
  extern void *memset (void *__s, int __c, size_t __n) __attribute__ ((__nothrow__ , __leaf__)) __attribute__ ((__nonnull__ (1)));

When replaced by hand with declaration lacking any attributes, it still gets optimized away.

    	/* will be optimized away for unclear reasons */
    extern void *memset (void *__s, int __c, size_t __n);

Contrast that to behavior of any user-defined function:

    	/* may be optimized away */
    extern void * memxxx(void *__s, int __c, size_t __n) __attribute__ ((pure));
    	/* should not be optimized away */
    extern void * memyyy(void *__s, int __c, size_t __n);

IMHO GCC's special handling of memset() is broken...

(1) http://www.cs.auckland.ac.nz/references/c/gcc4.7/Function-At...

(edit: I just noticed that qznc's earlier top-level comment makes this same overall point, even down to the specifics about future optimization and volatile. That comment is also shorter, so it is probably provides more "bang for the buck" to read it instead of mine.)

I am pretty certain that memset is not conceptually being handled especially different: it is simply being inlined. If you have a memset of a variable to 0, gcc's compiled output might involve just a single mov instruction and if it is a small array, it might be a few mov instructions: if it is a lot of memory being cleared, it might involve a call.

However, as it is inlined, the semantics are pretty clear for how it should work (and attributes like pure will be deduced, as they would for any function you include in your actual code; you don't need to mark it as such <- edit: to be clear, though, that I really don't think that memset is "pure"... it is pretty much the exact opposite of "pure"): it is equivalent to assigning a variable a value, and if that variable's value will not ever be used, then the assignment will not happen.

Instead, if the developer really really really insists that that assignment happen even if, from the perspective of the C language standard there is no legitimate side effect of that operation, the correct thing to do is to temporarily qualify the pointer with volatile and then do whatever it is you wanted (such as wiping it).

Sadly, this article does not even in a single place mention "volatile", so I'm not certain that the author understood how this works. Instead, he states that a "solution" is to instead put a zero in the first element and then copy it to subsequent elements... something that could easily be detected by abstract interpretation, and which a sufficiently smart compiler would be correct in also optimizing away.

It's not about volatile; marking the array as `volatile' caused warning, and the call is still optimized away:

    a.c:14:2: warning: passing argument 1 of ‘memset’ discards ‘volatile’ qualifier from pointer target type [enabled by default]

To take `volatile' into account, the `memset()' would have to to have `volatile void * s' as first argument in its prototype.

Curiously enough, making a custom function with volatile pointer argument and `pure' attribute still causes GCC to optimize it away. I guess such combination makes no sense anyway.

    extern void * memaaa(volatile void *__s, int __c, size_t __n) __attribute__ ((__pure__));

Right. I did not claim that once you added the volatile qualifier that you could still use the existing implementation of memset: if you simply cast the pointer to volatile and then pass it to the existing version of memset, you will certainly get a diagnostic indicating that you lost the qualifier, as memset itself does not have volatile as part of its type definition.

However, the author's attempts to redefine memset without using volatile rely on gcc limitations: in the first "trivial" case that it does not use abstract interpretation while looking for constant variables (in this case, it should be easy to fold that loop down to "oh, that rvalue is always a constant 0), and in the second that it does not do extensive optimization across compilation units.

As for the other thing you noticed: while I would imagine that the gcc-specific "I am the developer and want you to do something very special and important" __attribute__ would override any notion that it might have regarding "volatile", they are really orthogonal concepts: with just that prototype the compiler wouldn't even know if that function does anything at all with the memory you passed... the function might just do something with the pointer value and not indirect it at all; therefore, one would pretty much demand that "pure" have the semantics that gcc is giving it here.

If you don't compile with "-fno-builtin" or "-fno-builtin-memset", gcc treats memset() as a builtin function when optimization is enabled. I'm guessing it overrides memset()'s prototype with that of __builtin_memset(), but I don't know enough about gcc's internals to be sure.

If you compile with "-fno-builtin-memset", gcc uses whatever prototype is in the preprocessed source code and generates calls to memset() as appropriate. However, if you provide your own memset() implementation in the same compilation unit (or in a different unit w/ LTO) that doesn't play volatile or asm("") games, then the compiler still might be able to deduce that the memset() call can be optimized away.

> IMHO GCC's special handling of memset() is broken

I appreciate how you've distinguished your opinion from objective facts. E.g. according to spec, it's not broken, but it's a perfectly valid opinion to hold that the spec is broken too. :-)

Reply on news.ycombinator.com