Hacker News Reader

Comment by drdrey | Hacker News Reader

Can someone ELI5 what the issue is with type punning and union aliasing?

Suppose you want to look at the bits of a floating point number, as though it were an integer. The type punning way to do that would be to access a double in memory through an integer pointer:

    double d = 3.14;
    int64_t *p = (int64_t *) &d; // !!!
    int64_t n = *p;

The union aliasing way would be to makeset the double field in a union and read the integer field back:

    typedef union {
        double d;
        int64_t n;
    } int_or_double;

    int_or_double u;
    u.d = 3.14
    int64_t n = u.n

According to the standard, the union aliasing approach is allowed, but the type punning one is undefined behavior.

---------

The standard team decided to forbid type punning to allow compilers to implement more optimizations. They still allow union aliasing because they felt there should still be a way to do low-level type casts if for those that absolutely want to.

No, type punning via unions is a gcc extension.

The standard only allows reading a value as the type it was written with or as char, and an access as char is only good for copying. If a char access used different bit order, that would be OK according to the standard because you couldn't tell unless you violated the standard.

It seems every compiler will tolerate a memcpy for type punning, even though this isn't required.

Does this mean that memcpy() is really more of a language construct than a library function, since it cannot be implemented correctly in fully defined standard C?

Yes. Check here for a discussion: https://blog.regehr.org/archives/959

Everything defined in the standard is in principle a language construct: whether it is builtin in the compiler or an actual function is an implementation detail. For many compilers, memcpy is both: they have builtin handling for it but fall back to the library implementation when there is no optimization opportunities.

Having said that, it is possible to implement memcopy in standard C: char pointers have an explicit special exemption to the aliasing rule so you are allowed to inspect objects as array of chars.

memcpy is legal and fine. To borrow the example from above:

  double d = 3.14;
  uint64_t i;
  memcpy(&i, &d, sizeof(d));

is perfectly kosher.

> is perfectly kosher

Aren't exact-bitwidth integer types optional even with C99 (let alone preceding versions of standard)?

They are optional, but only in the sense that if you can't exactly represent e.g. uint64_t, there can't be a uint64_t type. So the example above is kosher in the sense that it's standards compliant, even if it isn't perfectly portable (for reasons unrelated to memset type punning; that you're using uint64_t in the first place).

If an implementation can't represent uint64_t exactly, it can still have a uint_least64_t type of whatever width works.

I certainly agree with what you said as you said it. However, as usual, it is the implementation that kills it.

a) I believe sizeof (double) * 8 == 64 is not enforced by the standard either. One could argue that it is hard to find anything else these days, but (for a contrived example) I believe nothing prevents you from making your own port of GCC for some purpose, and make 'double' be 'float'. (From my memories, it is even far enough from rocket science.)

b) It already happened to me personally, to work with code where people have implemented their own uintXX types (because of the lack of the standard ones), and then over time these became narrower/vider.

So if we would like to have it portable, I think, nothing beats:

union { double d; char cc [sizeof (double)]; };

and there we are, back to what is in the standard.

I mostly agree, but with b) IMO the problem was that you worked in an environment that was already not concerned with the standard. I'm not sure if it's OK by the standard for a user to define types named e.g. int32_t that don't comply to the standard restrictions.

sizeof (double) can be guaranteed to be 8 if __STDC_IEC_559__ is defined, so checking that is an additional hurdle if you want to use memcpy for punning. That said, I think it can be assumed that you're writing non-portable code for a specific set of platforms already if you are punning floats to integers, because the representation is implementation dependent in the first place. Probably you also know the environments in which it will be compiled as well, so the example is a bit contrived.

Very much agree that unions are the easiest and standard way to do type punning. There's so many misconceptions about this somehow being unclear in the standard. Sure, it does reflect poorly on the quality of the standard that Linux contributors don't understand it and that Linus Torvalds actively ignores it, but if you know what you're looking for it's often not that hard to find a definite answer.

C99 and C11 allow type punning via unions (although apparently that only gets mentioned in a footnote)

https://stackoverflow.com/questions/11639947/

AFAIK even using union for the above purpose can be seen as "undefined behavior" when reading standard, and that's why the standard is especially wrong, as it makes some typical desired (and needed) outcomes impossible to implement, and at the moment clang still has problems even with the union variant whereas gcc allows you to use union by passing the compiler switch which effectively has the semantics "we promise we won't listen to the standard in this case."

That's why Linus writes:

"So the commit message that talks about how horrible union aliasing is is pushing a story that is simply wrong."

The commit message with which he disagreed obviously explicitly wrote something like (paraphrasing) "using union is undefined behavior by standard, so we are changing the code to not use it" etc.

Linus also wrote: "I'm not talking about the changes themselves - I can live with them. But the _rationale_ is pure and utter garbage, and dangerously so."

The rationale was (note, I don't have exact quote, I'm paraphrasing, if somebody has the original please give a link) that using unions for that is bad as per standard. Which is what the standard says but which is for practical purposes simply wrong. The standard is wrong an should not be followed, that was main Linus' argument.

What part of the standard should I read to come to the conclusion that type punning with unions is undefined behavior? C99 and C11 both explicitly mention that it's not, but maybe you're talking C89/C90?

IMO there are 4 highly relevant paragraphs in C99:

6.2.6.1: When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values.

6.2.6.1: When a value is stored in a member of an object of union type, the bytes of the object representation that do not correspond to that member but do correspond to other members take unspecified values

6.5.2.3 DR283) If the member used to access the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called "type punning").

6.7.2.1: A pointer to a union object, suitably converted, points to each of its members (or if a member is a bit-field, then to the unit in which it resides), and vice versa.

None of this implies that unions can't be used for type punning. What it does imply is that reading from a larger member of the union than you last wrote to will leave you with an unspecified value.

I think the clang and gcc contributors consider all this in combination with “strict aliasing” and “type” consistency attitude effectively if you just wrote a double to a union member, you can’t expect that reading integer from the same union has any meaning at all so they optimized that away “per standard”. To come to that, it’s not what is explicitly written when you do a keyword search for “union” but what is not said that it “must work” in spite of other “must work” and “undefined” standard statements. But ask them for the details, I’m just following the topic and have no active involvement.

> if you just wrote a double to a union member, you can’t expect that reading integer from the same union has any meaning at all so they optimized that away “per standard”.

gcc 4.9.2 for sure does not (-O3 -fstrict-aliasing). Not sure what clang does, but I'd be surprised if it ignores the standard. The standard makes it pretty clear what meaning it will have. Sure, gcc and clang maintainers may seem to read the standards like a lawyer looking to void a contract, but in this case it is actually very clear what they should do.

> what is not said that it “must work” in spite of other “must work” and “undefined” standard statements.

The standard makes it clear how to deal with the semantics: "In the abstract machine, all expressions are evaluated as specified by the semantics. An actual implementation need not evaluate part of an expression if it can deduce that its value is not used and that no needed side effects are produced (including any caused by calling a function or accessing a volatile object)." Only constraints directly use "shall"/"shall not" type language, but the expressions should still be evaluated as specified by the semantics.

Googling I've found for example this:

“GNU extensions to standard C++ (and to C90) do explicitly allow type-punning with unions. Other compilers that don't support GNU extensions may also support union type-punning, but it's not part of the base language standard.”

Also, apparently in C99 one relevant statement was:

“The value of a union member other than the last one stored into [is unspecified]”

https://stackoverflow.com/questions/25664848/unions-and-type...

Note: In case it's not obvious, I, personally, believe that, at least, using unions for type-punning has to exist and should be explicitly supported in the standards too. And that the compiler contributors should not produce practically unusable results even when some standard is wrong (for common use) or unclear. But the quotes like the above give a hint on which basis some try to cling to bad approaches in this special case. In other cases it's probably even harder to avoid unwanted results.

Several compiler optimizations center around proving which variables don't change. Allowing type punning makes it harder for the compiler to do this.

Reply on news.ycombinator.com