Hacker News Reader

Comment by JonChesterfield | Hacker News Reader

I expected this to be an aliasing violation and was not disappointed. Dereferencing a uint8_t through a uint64_t is undefined behaviour whatever the target architecture is. That it works for the author on x64 just means the compiler didn't break it this time around.

In particular, an 8 byte aligned uint8_t also can't be dereferenced via a uint64_t. It isn't UB because the alignment is wrong, it's UB by definition and also the alignment might be wrong.

This is a woeful state of affairs propagated by an ill founded and widespread confidence that the C and C++ standards probably aren't as hostile to type aliasing as they actually are.

View on HN

If uint8_t is a char type (and invariably it is), then it is fully defined.

Similarly, it is perfectly safe to cast a correctly aligned uint8_t* to uint64_t* and dereference it as long as you originally stored an uint64_t into it (or the compiler can't prove you didn't). Remember aliasing UB is always on derefencing with the wrong dynamic type, not on casting pointers.

You are correct in principle, but the code in the original post definitely exhibits strict aliasing violations. Not only that, it didn't need to.

The original code computing the hash by casting uint8_t* to uint64_t* for wide leads is generally wrong, but the memcpy one is fine.

I'm surprised the gcc and clang people have never noticed that uint8_t is an integer type, and therefore does not need to be a character type.

Well, if we were starting from scratch {u,}int8_t should definitely not be character type; instead we would have byte_t (plus an explicitly sized variant, octect_t?) as the general universally aliasable byte type, and all of these would be guaranteed to be distinct types.

But today, uint8_t is in practice always a char type. You can static_assert if you want to be completely sure. Or just use std::byte.

There's an old gcc bug report about whether uint8_t should have the magic aliasing properties of char or not. I don't remember what they concluded.

You can cast any pointer you like to any other pointer. When you dereference it, if the declared type is not the type of the underlying object, bad times for you.

An interface which takes a uint8_t* and immediately invokes UB if you pass it anything other than a pointer to a uint64_t can indeed be written, but one should expect people to pass it things other than a cast uint64_t*.

Further, the magic aliasing char doesn't work in both directions. If this code was changed to take a char instead of a uint8_t it would be exactly as undefined as it is currently. You can aim a char* into a uint64_t and deref the char, but you can't aim a uint64_t* into a char and deref the uint64_t.*

They ended up making uint8_t a typedef for unsigned char rather than a distinct type (which is permitted but not required by the standard) so the semantics are now clear.

Edit: That's what I remember but I can't find evidence of it. The docs say this is ABI dependent so perhaps the above statement is true on some specific ABI (if I haven't misremembered it entirely). But I can't track down wheere a relevant ABI would be documented.

https://gcc.gnu.org/onlinedocs/gcc-13.2.0/gcc/Architecture-i...

Edit 2: this header has a typedef of uint8_t to unsigned char, but it's part of glibc rather than gcc

https://android.googlesource.com/platform/prebuilts/gcc/linu...

I thought if uint8_t exists, it logically must be the same as unsigned char? You put some of the statements of the standards together and that's their combined consequence.

No it doesn't have to be, and it wasn't. That was the whole problem. They were both unsigned integer types with 8 bit widths but they were distinct and, in particular, uint8_t didn't have the aliasing exception that unsigned char does.

No, the relationship is directional. You can reinterpret a pointer to an integral type to a pointer to unsigned char and read the underlying bytes. The other way around is UB.

What I mean is that this is perfectly valid code:

   uint64_t x=0;
   uint8_t * p = (uint8_t*)&x;
   uint64_t y = *(uint64_t*)p;

It is not the casting that is the problem, it is the dereferencing. And as long as that matches the dynamic type, it is ok.

> This is a woeful state of affairs propagated by an ill founded and widespread confidence that the C and C++ standards probably aren't as hostile to type aliasing as they actually are.

Yep. Honestly, while I get the historical reasons for strict aliasing, I wish it hadn't been created. Just make -fno-strict-aliasing the default and normalize the abundant use of restrict, maybe shorten it to res or rest just like we use const instead of typing out constant. These implicit rules that usually, but not always, do what you want can be a bigger footgun than having to do things explicitly.

> just like we use const instead of typing out constant.

Those aren't constants†, they're immutable variables which are quite different. Because of the "as if" rule and provenance in some cases you get the same benefit as with a real constant, but that's not what this is.

† In C or C++. In some other languages const does, logically, get you an actual constant.

Unless you think that the abbreviation const stands for some other word than constant, I'm not sure how that's relevant here?

Because these aren't actually constants it would actually be even worse if the keyword was "constant" rather than "const". Jargon that's identical to ordinary words but is just plain wrong if taken that way makes it harder to be understood.

In a language where the const keyword gets you an actual constant this condition wouldn't arise.

How historical is the importance of strict aliasing? AFAIU, strict aliasing survives not because of alignment issues when type punning, but because optimizing compilers heavily rely on non-aliasing to guarantee semantic correctness when reordering code, especially in and around loops. And the performance benefit of reordering has grown exponentially in lock-step with the emergence and increasing sophistication of superscalar architectures, pipelining, and cache hierarchies.

Strict aliasing, IOW, is as a practical matter mostly about when and to what extent compilers (and sometimes processors) can optimize code through automatic reordering, much like wrt parallelism and memory barriers. In this sense, the role of strict aliasing is more important than it ever has been, and while technically "merely" a matter of performance (correctness could be obtained by completely abstaining from reordering), the performance of modern processors plummets precipitously when implicit hardware parallelism can't be fully leveraged.

It's a hack from the days of separate compilation. Like lots of the "no diagnostic required" sucks to be you UB which can only be diagnosed at whole program link time. That is, the semantics and usability of modern C++ are compromised by the hardware capabilities of the machines on which C was first developed.

You're right that disabling strict aliasing removes some optimization opportunities for the compiler, but you can always get them back by adding restrict here and there. So it's really about what the default behavior should be. I'm thinking that it probably would've been better for the safe and intuitive behavior to be the default and for the extra optimization opportunities to be opt-in. A set of rules that usually does the right thing, but not always, can lull programmers into a false sense of security.

Also, I'm not sure how true your argument is in practice. Modern compilers have other, more sophisticated ways to prove that things don't alias. A number of projects compile with -fno-strict-aliasing without using restrict very often, for example the Linux kernel, and they don't seem to suffer much from it. Linus has this to say about aliasing in kernel code:

> In x86, I doubt _any_ amount of alias analysis makes a huge difference (as long as the compiler at least doesn't think that local variable spills can alias with anything else). Not enough registers, and generally pretty aggressively OoO (with alias analysis in hardware) makes for a much less sensitive platform.

Well, yes, if you're playing this kind of game, the compiler cannot type-check what you're doing (or sanity-check it in any other way). You're outside the area covered by the standard. You're coding without a net.

But C and C++ are explicitly intended for this kind of situation. You know enough about how types are represented in memory to fiddle with their internals, and you have sufficient reason to do so? Go for it. Just don't expect the type checking to save you, because it won't.

I'd be totally happy with C or C++ here if the attitude to type checking was "you've done weird things so you're on your own, be careful". Things like mutating a vtable could be fair game, just do it carefully.

However that's not the setup. C++ in particular, and increasingly C copying from it, optimises assuming you never do anything weird with pointers. This makes application code that doesn't do such things faster in a separate compilation world. It makes system code that does do such things undefined, forcing either compiler flags (fno-strict-aliasing and friends, aka writing in a different language that looks like the nominal one) or taking care around separate compilation to make sure the compiler never sees both sides of the boundary at the same time.

If I had the skill, I would want to create my own C compiler that tried not to optimize things like this, e.g. don't optimize away infinite loops, don't optimize away signed overflow checks or do incorrect algebraic simplifications because signed overflow is undefined behavior, and by default act as though -fno-strict-aliasing were on. Add options for trapping on overflow or optimizing away overflow checks, signed or unsigned. Just do register allocation and minor constant folding (but again, don't act like signed overflow doesn't exist, and don't do constant folding across pointer writes if its address has been taken).

There exist open source C compilers that aren't GCC or Clang, such as TCC and Zig, but to my knowledge they all produce really, really bad assembly. There existed proprietary C compilers in the 1980s and 1990s that were written by one or two people that still produced solid assembly, so it's possible, just no one has done it, so Clang and GCC are the only free and open source options for decent assembly for C. Those two also have the downside of being C++ compilers, making them more complex than they need to be.

Maybe another option would be to improve an already existing open source C compiler's codegen than to write a new one from scratch again, assuming they'd agree with the above ideals.

It's on my to-do list for C. There's some mess around the preprocessor and parsing typedefs but it's not too horrendous. The guix bootstrap had a compiler for a largish subset of C written in scheme, that's probably a good start point if you don't want to go from scratch.

I like C as a notation for programming computers. I don't like the slight misstep -> UB -> everything is ruined philosophy of WG14.

(edit: it should be possible to emit LLVM IR with the right metadata to avoid the optimiser mangling it. In practice there would be a long tail of accidentally assuming C++ semantics that aren't justified by the IR, where bug fixes for those would be valid upstream. If one wanted to take on the probably thankless task of trying to remove ISO C/C++ assumptions from llvm.)

Rust already encountered lots of bugs when they tried to add all the aliasing info they had to the IR output.

> Things like mutating a vtable could be fair game, just do it carefully. However that's not the setup.

You can mutate a vtable without UB. A method may call the destructor on its this pointer and use placement-new to create a new object in place. The fact that any method of a class may do this combined with the fact that the placement-new object might be a more derived version of the destroyed object (so an existing Foo* continues to be valid) means that compiler can't cache vtable lookups for consecutive method calls, making nearly any optimization of virtual function calls impossible because you don't know the type or called function, unless you see the object being constructed (when the vptr is assigned) and inline each called function in turn.

(At some point C++ added a rule that basically reads "you're allowed to cache the vptr, if the accesses were written using the same pointer variable name". This doesn't work well for optimizing compilers because they'll quickly fold two equal values into a single variable in their internal languages and lose track of whether the user wrote two distinct variable names or not.)

I think we're getting to the point where C++ users should define two classes:

    class Integer { 
        // uses unsigned integers for all operations
        // but converts to signed when you need output
    };
    class Pointer {
        // uses void* and memcpy for all loads or stores
    };

Put all of the correct operator overloading on those to get the ergonomics we had in the 90s.

Then we can safely ignore all the undefined behavior which has been so eagerly embraced by the compiler writers, and we can go back to treating the hardware as though it is sane. (Which the hardware is desperately trying to fake being anyways)

After that, we can create some sane container types which don't offer up dangling references or pointers, and Bob's your uncle.

If my signed integers overflow, that is wrong. I want to be able to detect if it ever happens, not have that be defined as perfectly valid wrapping behavior. If it's undefined, it's perfectly legal for the compiler to make overflow trap, so I can catch it, or confirm it never happens when running tests our debugging. `-ftrapv` or ``-fsanitize=undefined` do this.

To each their own. I want my 64 bit integers to do what the CPU does, and I could bore you with useful examples where two's complement wrap around is exactly the right behavior.

But more importantly, I don't ever want to worry about stumbling into undefined behavior, and compiler extensions like -ftrapv or -fwrapv are not part of the standard. I can't count on those being there when I switch compilers, and if I write a library for others, I can't count on the application developer using the flags I need.

Reply on news.ycombinator.com