Hacker News Reader

Comment by sharpneli | Hacker News Reader

I is not wrong optimization. The guy is explicitly breaking the C specification by doing that.

The correct way of accessing a variable like that is trough an union. Here is an example http://pastebin.com/8V3whcL3

Alternatively one can make both of them as volatile. That will basically force the compiler do all the loads and stores.

I really hate it when people write code that breaks the spec and then complains to the compiler maker. And now they actually changed the compiler to work with the code that is simply wrong.

Just as what happened with glibc memcpy (works like memmove due to people not knowing what they are doing and then crying for the libraries to change their behaviour to match their own incompetence).

View on HN

> The correct way of accessing a variable like that is trough an union.

It's funny you should say that, because people have been taking the exact opposite viewpoint for years: “union do not allow type-punning because it was only clarified in TC3 of C99 that they do. memcpy has always been the correct way to type-pun.”

The bug reporter is referring to the rules described in 6.5:6 and 6.5:7 of C11 (the same rules are included in C99, but interestingly not in C90, meaning that “gcc -std=c90” should go theoretically easy with the type-based optimizations. It doesn't). Here is a link: http://port70.net/~nsz/c/c11/n1570.html#6.5p6

And an excerpt for your convenience:

If a value is copied into an object having no declared type using memcpy or memmove, or […], then the effective type of the modified object for that access and for subsequent accesses that do not modify the value is the effective type of the object from which the value is copied, if it has one

How do you interpret the above paragraph?

I've been under the impression that unions don't truly allow type punning, but given that the practice was widespread (e.g., it was used in the code Protocol Buffers generated a few years ago), it was a pretty safe bet compilers would support it as an extension.

I was also under the impression that the One True Way to type pun was to cast to char*, which is allowed to alias, and then cast again to the type you actually want.

I was also under the impression that the One True Way to type pun was to cast to char, which is allowed to alias, and then cast again to the type you actually want.*

That clearly doesn't work, but I can see from a naive perspective why some think that. It feels like a nice hack, but a cast to char* doesn't "remove" the original type.

To be clear: char* can alias any type, but only char* can alias char*.

Casting through chard doesn't help: how do you get to a specific pointer value doesn't really matter. The C aliasing rules are only about dereferencing pointers.

What is allowed, as an explicit exception to the aliasing rules, is reading an object representation by dereferencing a char pointer. This is why memcpy is the approved way to do type punning, but you do have to copy.

> How do you interpret the above paragraph?

It says that...

   int a = 5;
   char *b = memcpy(malloc(sizeof(a)), &a, sizeof(a));

...should fail. The memcpy(3) src is &a, a pointer-to-int; the memcpy(3) dest is a pointer-to-void (i.e. an untyped pointer) from malloc(3). Thus, memcpy(3)'s return type should be a pointer-to-int.

It's being assigned a pointer-to-void, yes?

It is, yes, according to the typespec of memcpy(3). But according to the standard, it shouldn't be; it should be returning a [whatever-type-src-was-passed-in-as]. I have doubt that there's any way in C to specify variable return type like that—but the C standard, AFAIK, doesn't require that memcpy(3)—or anything else from libc—be implementable in C. It can be, as Lisp would put it, a "special form."

To get an implicit pointer-to-char-typed buffer out of memcpy(3), you should have to attach the malloc(3)-returned buffer to a pointer-to-char-typed variable (or give it an explicit cast) to give it a type, and then pass it in as dest.

The guy is explicitly breaking the C specification by doing that.

You assert this, but clearly there are people in the bug thread who believe otherwise. In the thread, the submitter references a portion of the C11 spec that he feels defends his usage:

  C11, 6.5p6: If a value is stored into an object having no declared type 
  through an lvalue having a type that is not a character type, then the 
  type of the lvalue becomes the effective type of the object for that 
  access and for subsequent accesses that do not modify the stored value.

I'm not familiar enough with the spec to know the context, nor would I defend this code as good practice, but superficially this seems to be evidence that this is a bug. Is there another specification that can you point to that this "explicitly" breaks?

C11, 6.5p7: An object shall have its stored value accessed only by an lvalue expression that has one of the following types:

— a type compatible with the effective type of the object, — a qualified version of a type compatible with the effective type of the object, — a type that is the signed or unsigned type corresponding to the effective type of the object, — a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object, — an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or — a character type ---------------------

If we interprete 6.5p6 like the submitter did then the whole p7 becomes meaningless for any mallocs. In that case everything always aliases everything ever when it's trough a malloced pointer. Which I hardly imagine is the point of the former paragraph. If so they have fundamentally changed the whole function of malloc. Which makes it odd as in C99 (The p6 wording was introduced there. It was not there in C89) they explicitly allowed unions to alias, just due to this issue. Would have been unnecessary if malloced things always alias.

The wording is not clear however. It would be nice to see the standards committee to publish a comment about this. This is also not the first thing in the spec that is unclear.

6.5:6 is certainly not saying that malloced memory always aliases with “everything”. Quite the contrary, it says that malloced memory cannot be used for type-punning. The following works and is idiomatic:

uint32_t u;

float f = …;

memcpy(&u, &f, sizeof u);

/* use u as a uint32_t in computations */

You cannot do the same thing with malloced memory: if u had been malloced memory instead of a variable, it would have been illegal to access that memory with an lvalue of type uint32_t in subsequent computations. 6.5:6 is certainly no license to use malloc'ed memory any which way.

There is a non-normative note in 6.5:6 "Allocated objects have no declared type."

Sure that is not strictly part of the standard. But helps in interpreting it. I'd say it quite clearly refers to malloced objects. Malloced object is an object with no declared type.

If you look at the bug report it uses malloced object as the example. And accesses it using two pointers of different types.

I think you are misinterpreting Pascal's example. He's saying that because 'u' is a variable with a declared type (uint32_t), it can still be used as that type even after it is written to with another type. He then claims that were it an "allocated object" with no declared type, the effective type would change to the type of the most recent write (float), and thus it could no longer be used as the original type (uint32_t).

I'm not sure he's right, though. The quoted part of the standard says "stored into an object having no declared type through an lvalue having having a type that is not a character type". I would think that "through an lvalue" is referring only to the case given in the bug report, where the store is expressed as "*ptr = val", and not to the case where ptr is used as an argument to memcpy(). Am I wrong?

You're right. I did misinterpret it.

However I also think you're right. Memcpy is effectively the same as through a character type in the end.

Uh, thanks for pointing that out. I'm familiar with allocated memory and effective type, but I never considered it from this perspective.

This could be quite a nasty surprise when the only change in the code is storage duration.

What you're missing is that storing to an object doesn't "access it's stored value".

Once you've stored to the malloced memory through an int-typed lvalue, it's effective type becomes int, and you can't access it (ie. read it) through a double-typed lvalue. But you can then write to it through a double-typed lvalue, because that doesn't access it's stored value - it overwrites it, and changes its effective type to double.

The specification defines the word "access" as:

access 〈execution-time action〉 to read or modify the value of an object

I would interpret that so that writing does modify the value.

There is a lot of code written and released that doesn't follow the standard to the letter. New compilers with new optimizations that require strict language adherence otherwise they generate broken programs cause issues for the developers but, most importantly, the users.

The memcpy/memmove debacle could have been avoided entirely if memcpy performed a simple check about what was being copied where and changed the direction of the copy based on that. Simple, cheap, and an entire class of bugs can be avoided.

People should still be aware of the behavior of their development system but the system shouldn't punish when a slightly wrong choice is made.

Optimizations are great but software that continues to function is even better.

It's a strain on portability.

Instead of just having a C or C++ program you actually have MSVC++ program (this is actually really common) or GCC program or clang program.

Sure one can argue that the C standard is too lax. But these kind of errors come simply from the fact that people learn C by trial and error, which doesn't work with that language.

Personally as long as the compilers still allow one to enable the optimizations it's not that bad in the end. Portability issues for badly written programs remain however.

Reply on news.ycombinator.com