Hacker News Reader

Comment by halayli | Hacker News Reader

memset is not a special case. Compiler designers are very much against special casing, and rightfully so.

Compilers do have an intrinsic equivalent but that has nothing to do with whether it gets optimized out or kept.

Following SSA rules, if you are writing to a non-volatile memory and not reading from it for the remaining duration of its lifetime then it's dead code.

use memset_s was a suggestion if your goal is to scrub memory because it guarantees the write will happen. The guarantee comes from the C standard:

> Unlike memset, any call to the memset_s function shall be evaluated strictly according to the rules of the abstract machine as described in (5.1.2.3). That is, any call to the memset_s function shall assume that the memory indicated by s and n may be accessible in the future and thus must contain the values indicated by c

Which essentially means treat the memory as-if it's volatile.

View on HN

Treating "the memory as-if it's volatile" would not block such an optimization. Specifically, zeroing a volatile or atomic that lives on the stack immediately before returning from a function whose scope it lives in is freely optimized away, because it is not observable.

Many people mentally model atomic as akin to volatile, about which they also frequently harbor superstitions.

I knew a FreeBSD core committer who insisted (and convinced our boss!) that volatile was entirely sufficient to serialize interactions between threads.

Someone else expected a line with a volatile or atomic operation never to be optimized away, and was appalled that it happened. Assignment to a volatile stack object whose address has not escaped and is not used will, in fact, be optimized away, regardless of the "clear language of the Standard".

Often, not performing what would appear to be a pointless optimization would block a more substantial optimization. Compiler writers optimize aggressively in order to break through to the substantial ones, which are most usually dead-code elimination that can then propagate backward and upward.

If you really need for a volatile object and operations on it not to be optimized away, you need to allow a pointer to it to escape to somewhere the compiler cannot see. Have a function in a separate ".o" file, and pass the pointer to that. Or, better, use a compiler intrinsic or other facility provided specifically for the purpose, such as memset_s or explicit_bzero.

> Specifically, zeroing a volatile [...] that lives on the stack immediately before returning from a function whose scope it lives in is freely optimized away

> Assignment to a volatile stack object whose address has not escaped and is not used will, in fact, be optimized away

I don't believe either of these statements. https://godbolt.org/z/8cqEdY9oE is a simple case where this doesn't happen. Can you post a Godbolt link or something that demonstrates when it does?

Keep in mind that the standard says "Accesses to volatile objects are evaluated strictly according to the rules of the abstract machine." In particular, it doesn't say that accesses to non-volatile objects through volatile-qualified pointers will be handled in any special way. (This looks like it's going to be changed in C2x, but it hasn't been changed as of C17.) Also, it says "If an attempt is made to refer to an object defined with a volatile-qualified type through use of an lvalue with non-volatile-qualified type, the behavior is undefined."

Any particular compiler and command-line setting where it is not optimized away tells you nothing about what that or some other compiler may do in a different release or command-line setting.

Talk to any core compiler implementer, and they will say that the optimization is allowed. No compiler performs all permitted optimizations, in any release, but the set of those each does do, varies.

> Any particular compiler and command-line setting where it is not optimized away tells you nothing about what that or some other compiler may do in a different release or command-line setting.

Can you provide a different program, compiler/version, and command line where it will be optimized away?

> Talk to any core compiler implementer, and they will say that the optimization is allowed.

Are you just guessing that they'll say this, or have they actually said this? If the latter, can you link to where they did?

> No compiler performs all permitted optimizations, in any release, but the set of those each does do, varies.

My Godbolt link is a really obvious, easy place that the optimization you claim is possible could be performed. I think the fact that none of the 4 major compilers optimized it away is decent evidence that it isn't allowed.

> have they actually said this? If the latter, can you link to where they did?

They have said it face-to-face, in person. It is possible that, with enough pushback, they have since disabled the optimization. (They all work at large corporations, nowadays, and must obey management direction.) Or, more likely, disabled certain cases of it. I will ask again.

For documentary evidence, you may consult the definition of C11 memset_s, which is specified not to permit the optimization. It is hard to imagine such a requirement being perceived as needed, absent cases where calls to memset have actually been elided.

I will also experiment further with Godbolt. You might have failed to tickle the compilers in just the right way.

> They have said it face-to-face, in person. [...] I will ask again.

If you do ask again, can you do so somewhere like a public mailing list, so I can read their response myself?

> For documentary evidence, you may consult the definition of C11 memset_s, which is specified not to permit the optimization. It is hard to imagine such a requirement being perceived as needed, absent cases where calls to memset have actually been elided.

The point of memset_s is to prohibit the optimization when the object isn't volatile.

Are you allowed to cast the void * to volatile void *, pass that to memset and except the same result, or is the void *, volatile void*, void * conversion eliminated aswell?

You can wave whichever dead chickens you like over the keyboard, all to the same effect. The compiler ensures only semantics that are observable.

If you pass a pointer to a volatile or atomic object to another function the compiler cannot see into, the compiler knows that any specified operations might be observable, so must act accordingly. Then, operations on that object will not be optimized away. Or, anyway, not until someone builds the program using Link Time Optimization ("-flto") and the called function body becomes visible.

I'm confused. Is the memset to the temporary volatile void* observable or not, and is the temporary possibly eliminated?

Casting the pointer passed to memset has no effect. The compiler knows what the storage pointed-to is: a temporary object. The compiler knows that memset does not allow a pointer to the object to escape from view. So, it can eliminate the call.

Calling C11's memset_s, or MS's SecureZeroMemory, or FreeBSD's explicit_bzero, has the effect, to the compiler, of making the object observable, so that operations on it will not be optimized away. There is no way to express this locally within the language; you must rely on facilities provided by a Standard or your platform, or on the dodgy linkage trick cited elsewhere.

volatile is a type qualifier and can be casted.

However, memset's declaration takes a void* which prevents you from passing a volatile void*. casting or not won't change anything in this case because what matters is what memset takes and not what the pointer's fully qualified type is prior to calling memset.

From the compiler's perspective, you're passing a non-volatile pointer to memset which makes it a candidate for elimination if the pointer's lifetime ends after memset call.

As of C17, adding a volatile qualifier like that doesn't do anything useful. The standard currently only requires that accesses to volatile objects be handled specially, not accesses to nonvolatile objects through volatile-qualified pointers. A lot of people, including Linus Torvalds [1], are unhappy that this is how it works, and it looks like DR 476 may finally change it in C2x.

[1]: https://lore.kernel.org/lkml/Pine.LNX.4.64.0607060856080.124...

The default "memset" should just do what the programmer intended, and there should be a "memset_kinda_maybe" for the current interpretation. I think it's totally busted that users need to guess what way the compiler is going to maul their program.

99% of C/C++ programmers are not bored language lawyers with time to memorize thousand page specs. They use the language because they're pragmatic. Messing up peoples code with incredibly over-eager optimization is the opposite of pragmatism. I'd rather spend a day writing inline assembly for a hot path than two days dealing with the compiler breaking my code and making users data insecure becuase "usually" its ok to ignore a memset.

Frankly, just learn the language properly… Compiler writers can’t throw their hands up in the air and do some non-deterministic/non-specified thingy. That would be terrible.

I know the language perfectly fine; that doesn't stop me from recognizing how broken this all is. I can dance around these horrible land mines, I'm just saying take the landmines out because people are getting hurt.

Reply on news.ycombinator.com