Why oh why does "undefined behavior in C" come up constantly here? I wrote C every day for decades. I was aware there's such a thing as undefined behavior. I can't remember it ever being even a miniscule factor in my daily work. What's changed?
If you get Godbolt to run the program and show the output, you'll see it change with GCC version.
Obviously what is "sensible" and what is "surprising" is subjective.
int foo(FILE *f) {
int val;
fread(&val, sizeof(val), 1, f);
return val;
}
I'm left feeling like it is impossible to implement fread() in C and that the standard library's API is no longer considered a good example of how to construct an API.
I guess the original motivation for it being UB to cast is to do with alignment constraints on some hardware. For example, a machine might be able to read an int8_t from any alignment but might require 4-byte alignment for int32_t. If so, then casting a non-aligned int8_t* to int32_t* and deferencing it would indeed fail on that hardware.
But some would argue that it is not right that GCC 7 breaks such code on a machine where that problem doesn't exist.
People should write these bits in assembly (as little as possible, but doing the casting and things), and then keep their C code well-defined using these low-level library routines.
The key difference is that the alignment (usually) cannot be determined at compile time, so the compiler would still have to generate code that worked if the alignment was correct at run time. I think this is what many people either expect or would like the compiler to do.
With that change to the spec, I could continue to write my code in portable C, and I'd only get an abort on the target with this alignment constraint.
What if on some other machine it doesn't cause the program to abort but instead returns a broken result? How do you cover that in the spec?
> But some would argue that it is not right that GCC 7 breaks such code on a machine where that problem doesn't exist.
Right, so let's write these low-level things in C and not worry about the case of 'where the problem doesn't exist'.
OK, how about, "dereferencing a pointer that isn't correctly aligned doesn't work on some hardware". I see that this is a hard area for the spec to do well. But the current situation isn't viable. I can't tell whether I can safely use fread for heavens sake.
I feel like dereferencing an incorrectly aligned pointer could be a runtime error, just like divide by zero or accessing memory that doesn't exist. Maybe that would be an improvement from the current status. Although I'm not sure the spec _could_ be changed like this at this point in time.
By checking the alignment on every access? That'd be far too expensive if the machine you're on doesn't trap.
Editing here because it looks like we've hit the max reply depth. This wouldn't be like UB in the same way that code that could generate an out-of-bounds memory accesses is not UB. The critical thing is that the compiler should assume that the hardware error does not occur.
Yeah, but good luck selling that... for the reasons you've identified it'd be a nightmare to run C on it.
You can cast void* to something else and it will work, but you can't expect to get a random void* and presume it has some valid well defined data for you to read. You must know where it came from and what's been done to it.
As a rule of thumb if you just write dumb bytes it's char*. If it's anything else you must know what you're doing and there are ways to perform type punning that's within the spec.
Can you expand on that? I don't understand.
Edit: I see the answer in one of your other comments:
> If len and array are both of the same type then the compiler must assume they can alias (or one of them is char*). One has to explicitly state with restrict that they cannot alias.
I'd happily take that hit. I can put restrict in the hot paths of my code. I'm tempted to say I want restrict to be the default, but I haven't thought that through.
I know. I think the reason that this an area that causes lots of debate is because the majority of C programmers from 20 years ago would not have thought the memcpy or unions approach was a better idea than casting and dereferencing void* . The casting approach used to "just work" unless you violated the alignment requirements on your platform, and that was a problem 20 year-ago C programmers were happy to take on.
For next time this discussion comes up, I'll try to think of a good example of when the void* approach yields easier to maintain code :-)
BTW, you can also use -fno-strict-aliasing to make the C compiler sort-of work like it did 20 years ago. I believe this is what the Linux Kernel does but I couldn't be certain that it is for exactly this reason.
If you're certain there is no violation (such as with malloc), then go ahead and do it. -fno-strict-aliasing is unrelated to this issue. It removes the strict aliasing assumption from the compiler (another source of potential issues if not understood), but void* (and char*) are explicitly allowed to alias to anything regardless.
I modified the code to ensure that the data is 8-byte aligned (I think I've done this correctly). It still goes "wrong". https://godbolt.org/z/Pnxnb8
In either case, adding -fno-strict-aliasing fixes it.
uint64_t *a = malloc(8);
You're starting with uint32_t pointers, which then go through void pointers and end up being accessed through pointers to uint64_t. This breaks strict aliasing since void* can not be used to bypass it.
If you had started with uint64_t, you could go through void* and back to uint64_t without any issues.
And in the case of malloc, I don't know what type the data was "originally" before it returned it to me. I guess I have to treat malloc as a special case where I can cast its void* to anything safely. That would seem morally wrong.
You need to have a model of how the compiler/optimizer works. In your example, it's important to know what objects you're working with and what the effective types of these objects are. Focus on the objects, not on the pointer types that you use to access them. Normally, the effective type of an object is its declared type. Allocated objects however, have no declared type according to the standard [1]. They get their effective type on first access:
"If a value is stored into an object having no declared type through an lvalue having a type that is not a character type, then the type of the lvalue becomes the effective type of the object for that access and for subsequent accesses that do not modify the stored value."
This means that you don't care how malloc works internally (unless you're writing one), and casting/accessing its result - which is guaranteed to be properly aligned - does not break strict aliasing.
But in your example, the strict aliasing violations are crystal clear. You declare uint32_t objects, therefore the compiler knows and can track the effective types of these objects. Later, you're trying to access the values of these objects through different types which is when the strict aliasing violation occurs.
> So is the rule that it is OK to cast a pointer back to its "original" type and dereference it?
Since you're accessing the same object and there's no mismatch between its effective type and the type you're using, you're ok.
> If so, I feel this rule breaks the abstraction mechanisms C provides. eg now when I'm implementing that xor512b() function, I need to know about the invisible "original" type of the arguments.
void* is not really an abstraction and should not be used as such (or to avoid thinking about designing a proper interface). You can make use of the type system by declaring proper types and being consistent, or you can use char* as a byte-level interface. Usually, when you see void* in an interface, it's clear what the implications are and how you should proceed with it. If it's not, you're looking at bad design.
[1] http://www.iso-9899.info/n1570.html Footnote 87) Allocated objects have no declared type.
The fread implementation sees dest as a void pointer and will access it going through char*, which is explicitly allowed.
Compilers incorporate inferences that can be made by assuming UB doesn't occur into their optimization because it improves code generation of correct programs.
I wouldn't say it's a minuscule factor in my daily work, as OP put it, but it's not something to agonize about if you stay mindful of avoiding common undefined behavior pitfalls. It would be very helpful if compilers were better about warning you when you're venturing into UB, but that's a different topic.
1: You can even get the shirt: https://teespring.com/shop/undefined-behavior-shirt
Many people say this but nobody "literally" believes it.
It's a way of expressing the righteous dominance of one person or group's attitude towards reasonableness over another's. In a way that forestalls argument.
"Literally anything" could literally mean global thermonuclear war.
- More applications are talking to the internet and being exposed to malicious input. There's also more money to be made in finding exploits, and more people and governments looking. At the same time, fuzzing techniques are getting dramatically better.
- Applications are more likely to run across different architectures (mobile) and different compiler versions (faster pace of compiler development), leading to more opportunities for "something weird" to happen. Multithreading is also more common.
- Memory-safe languages have taken a larger market share, and the popular attitude towards memory safety issues is changing from "bugs are a fact of life" to "this is a drawback of specific languages".
For the most part, avoiding UB is common sense when you grasp the basic abstract C memory model - types of storage and their lifetime, avoiding out of bounds array access, keeping track of the lifetime of all allocations, etc.
Do all C programmers know what strict aliasing means?
How about pointer provenance?
Have they memorized integer promotion rules?
These are all potential sources of undefined behavior that can lead to catastrophic results.
I bet most experienced C programmers with decades under their belt would fail one or more of these tests.