I’m amused by the people who ask for optimization and then complain about it. Or the “it did what I said, not what I meant” crowd.
But, officially, undefined behavior is always undefined, not just at higher optimization levels.
But, officially, undefined behavior is always undefined, not just at higher optimization levels.
Additionally in C UB is often not intentional nor trivial to detect in your codebase since it may be the interaction of two pieces of code that are not anywhere obviously close to each other. There comes a point where faster but broken code isn't better it's just broken.
I’m a little more familiar with the C++ committee than the C committee. The C++ committee prefers to declare something an error than to declare it undefined behavior. They only declare something undefined when they believe it would be unreasonably difficult for the compiler to detect the problem (e.g., using multiple, incompatible, definitions for an inline function; which can happen if you have a macro expanding differently in different parts of the code, or you have multiple definitions for the function, but only one definition is ever visible to the compiler at any moment in time).
I’m pretty sure the “signed overflow is undefined” rule is something of a special case: it should be easy to detect when source code doesn’t have a hard coded upper bound, but giving an error or warning in all cases will create too many false positives, and declaring that it wraps on overflow has been deemed unacceptable by the committee.
Most instances of UB are ones that the compiler could not detect. Your compiler isn't going to detect that your integer overflows, it assumes that it won't. Shit blows up if that assumption was wrong.
If you want to detect UB, you should run ubsan and the like. The compiler is not running it.
Programmers do not, in general, want to do the amount of work it takes to safely add two numbers together. You need something "weird". Like an error path for every addition. Or a type system for ranges, where you actually have to specify ranges (because adding two bytes together that you casually left as 0-255 will create a new ranged value 0-510; multiplication is even more fun). Or some other exotic idea. So we just let them overflow and the chips fall where they may.
It absolutely can't if it doesn't know what the values are going to be.
> the compiler will have to flag every integer addition as UB
No, I disagree. If it has no knowledge of my runtime values, it can't flag addition as UB because it isn't. It's UB only if I'm using values that would overflow, and the compiler in general can't know that. If I've done my program right, it will never overflow. There is no UB, any flag would be just absolutely wrong. The compiler couldn't detect UB. There's nothing to flag.
If it knows the values, then of course it can detect it and flag it:
x.c:3:18: warning: integer overflow in expression of type ‘int’ results in ‘-2’ [-Woverflow]
3 | int a=2147483647+2147483647;
I agree that overflow-checking is needlessly sucky in C but that's not "the problem." I usually make sure my arithmetic is safe but the compiler won't know it.
This is a deep misconception, and not at all how most undefined behavior is related to optimization.
Trivial example: Compilers assume that your variables aren't written to randomly from other threads. Without this assumption, virtually no optimization would be possible. Therefore, data races are UB - they violate a hard assumption of the compiler. But at no point did the compiler say "oh, you wrote to this variable without synchronization! Now I'll show you, hehehe!".
This is the same for e.g. removed overflow checks. Compilers can optimize many loops only if they assume signed integer overflow never happens. So, because compilers should be able to assume this, it is UB if it happens in your code. The same logic that deduces "this loop cannot wrap around" deduces "this if condition [an overflow check] cannot ever be true".
But it's easier for a programmer who got their UB-reliant checks optimized out to attribute malice to the compiler than to understand optimization fundamentals, and thus we get people complaining to high heaven and back.
No, the OP wrote specifically that "the compiler detects [UB]" and then does optimizations exploiting that. Not "detects potential UB". The former is a common misconception, the latter is basically a truism.
Tell me how these use UB: integer constant folding, unused load removal, optimal use of GP registers to minimize stack spilling
>sane assumptions
"sane" here is doing a lot of work. Assuming overflow won't happen is not a sane assumption, assuming some other thread won't randomly write into your stack memory is.
int a = 10;
int var1;
int b = 20;
foo(&var1);
int c = a + b;
void foo(int* x) { *(x-1) = 0; }
Same idea as above.
> Assuming overflow won't happen is not a sane assumption
If the alternative is loops being much slower when you request optimizations, then maybe it is. Consult your compiler manuals if this default irks you.
My original point was that compilers don't specifically abuse UB to sleight you. You're trying hard to draw a line in the sand between different degrees of UB so that you can label one side as "not sane", with (what appears to be) the sole intent of concluding malice on the compiler author side. Please accept that the related tradeoffs have been discussed, and that a default was chosen (if you are curious for a recent instance of this, see [0]).
If you are ok with leaving performance on the table by disabling overflow being UB, you are literally free to do that. You're also free to lobby the standard bodies to alter their carefully considered choices. But you're not going to get anywhere by labeling any of this "insane".
[0]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p090...
Oh come on, this is like the most UB-centric thing in the entire compiler. The compiler uses UB to know that it has enumerated all uses of a memory location, as any pointer that didn't get its value from the data dependency tree that compiler used had to use UB to compute the address. (Be it from buffer overflow, use after end-of-lifetime, random integer converted to pointer, etc.)
Everybody agrees that compilers should only do sane optimizations. Nobody agrees on which subset is sane.
It isn't; that sentence is even a type error. UB is not a property of compiler output. It's a property of compiler input.
The use of registers to store variables between uses (when those variables never have their address taken) relies on the lack of any defined behavior which would allow the program to determine this is being done. The fact you can't find this in the spec is precisely because it is not defined.
The compiler sees a variable x that's allocated on the stack. It looks at all the uses of the address of x, and sees that they are all loads and stores. It therefore decides to promote x into a register variable.
Where's the UB, you ask. Well, since the address of x was never leaked, it is UB to compute another pointer to that address (say, via an integer-to-pointer conversion). The compiler never checked for that possibility to affirm that it was UB; it knew that any code that did that would be UB and simply ignored the possibility.
This makes arithmetic overflow a very poor vehicle for UB because it's unusual in that you can't really take advantage of the UB without pointing to the specific operation that caused the UB to occur. This is why I believe that arithmetic overflow UB is gratuitous, and people objecting to UB because of what happens specifically with arithmetic overflow go on to make suggestions that are completely untenable because they don't have familiarity with how most UB works.
The problem causing this is that C loops are overspecified. If there was a “for…in” loop that didn’t make you declare and increment a variable, then manual increments could trap on overflow without causing so many problems.
If you can turn the loop from upward to downward then you do have a bound for it, and then you can tell if it overflows or not.
Also why is signed overflow a problem and not unsigned?
Surely you want unsigned loop optimized also?
Typically not a problem for i=0...n loops but eg `for (i=0;i<n;i+=2)` could overflow.
> UB problems happen only when compilers detect it, understand it is an opportunity for optimization, because they are allowed to do anything, then do that optimization.
Perhaps I'm misunderstanding the comment, but I took "detect it, understand it is an opportunity for optimization, because they are allowed to do anything, ..." to mean "detect that undefined behavior occurs and then ..."
Instead, it should be "detect that an optimization can be applied and that the optimized code will do the right thing in all cases that undefined behavior does not occur, and apply the optimization."
I've met programmers who seem to believe that compilers intentionally apply weird transformations when they detect undefined behavior. But really, the compilers are applying sensible transformations for well-defined code and ignoring whether those transformations are sensible for code that isn't well-defined.
That's because the compiler doesn't detect the situation. If it knew, it could, say, remove all code leading to that point - it's legal and the resulting code would be faster.
>Compilers can optimize many loops only if they assume signed integer overflow never happens.
I don't think this happens. Most loops iterate over arrays, and at least in C tradition unsigned integers are used as array index with no signed integers in sight.
void func(struct x *aptr, struct x *bptr) {
x->a = 5;
y->a = 10;
x->a = 10;
}
The compiler is not really "detecting that your program has undefined behavior". Instead, it is assuming that undefined behavior doesn't happen, and optimizes your program using that assumption. It's unclear what "detecting undefined behavior" would mean here... what kind of diagnostic the compiler should emit, or how to suppress the diagnostic. It's clear that this simple code is pervasive, and the preferred approach for dealing with it is somewhat outside the scope of a normal C compiler... something like static analysis, runtime instrumentation, or some kind of formal methods.
I'm not sure this is a good example, because there's no way for you to read them from the same thread (and reading from another thread would be a race). Reading them from a signal handler would yield unspecified values (unless you're using lock-free atomic types, in which case we're probably not worried about optimizing these assignments).
> When the processing of the abstract machine is interrupted by receipt of a signal, the values of objects that are neither lock-free atomic objects nor of type volatile sig_atomic_t are unspecified, as is the state of the floating-point environment.
void function(struct x *ptr) {
ptr->x = 1;
if (ptr == NULL) {
some_big_chunk_of_code();
}
}
"Warning: dead store detected (file):line clobbered by (file):line. Use keyword volatile to always store or remove (file):line."
This way the optimization is promoted from the resulting binary to the source code, and bugs / typos / etc can be corrected.
Telling the developers to optimize these would be equivalent to telling them to start inlining their code (or creating variants of every function that might be inlined, to optimize it for the specific context where it will be inlined).
The above code isn't necessarily code as the programmer wrote it, but the code after it appears after macroexpansion and various optimization passes. If you were writing C++, the code may be templated.
Compilers justify register allocation by assuming stack can't be smashed, contrary to hardware reality. Because smashing the stack is UB. That's what C UB is for.
Think of it like trading one cycle-accurate emulator for a better, more tightly-coded cycle-accurate emulator. Or a plain loop-and-switch bytecode interpreter for a threaded-code bytecode interpreter. The way your code is running on the underlying machine changes, but the semantics of your code relative to the abstract machine the code itself interacts with should not change.
> Compilers justify register allocation by assuming stack can't be smashed, contrary to hardware reality.
...which should be entirely fine, as the existence of a stack isn't part of the exposed semantics of the C abstract machine. Values can be "on the stack" — and you can get pointers to them — but nothing in the C abstract machine says that you should be able to smash the stack. There's nothing in the C standard itself describing a physically-laid-out stack in memory. (There are ABI calling conventions defined in the standard, but these are annotations for target-uarch codegen, not facts about the C abstract machine.) There could just as well be a uarch with only the ability to allocate things "on the stack" by putting them in a heap — in fact, this is basically how you'd have to do things if you wrote a C compiler to target the JVM as a uarch — and a C compiler written for such a uarch would still be perfectly compliant with the C standard.
If C were an interpreted language, the fact that the stack can be smashed would be referred to as a "flaw in the implementation of the runtime" — the runtime allowing the semantics of the underlying uarch to leak through to the abstract-machine abstraction — rather than "a flaw in the interpreter" per se; and you'd then expect a better runtime implementation to fix the problem, by e.g. making all pointers to stack values actually be hardware capabilities, or making each stack frame into its own memory segment, or something crazy like that.
As a compiled language — and especially one that has encapsulation-breaking holes in its abstract machine, like the C inline-assembly syntax — you can't exactly expect a runtime shim to fix up the underlying uarch to conform to the C abstract machine. But you could at least expect the C compiler to not let you do anything that invokes those cases where it's exposing potentially-divergent uarch semantics rather than a normalized C-abstract-machine semantics. As if, where there should be a shim "fixing up" a badly-behaved uarch, you'd instead hit a NotImplementedException in the compiler, resulting in a compilation abort.
There are a huge number of languages where you pass strings along with their length. There are also plenty of languages where you know (either via compile-time analysis and an insistence on writing code in an analyzable way, as in Rust, or via mild runtime overhead, as in Swift or Haskell or Java) that the memory remains still valid when you reference it. But those languages are not a subset of C. They either need more information about a program than a C program contains, or they disallow operations that the C abstract machine can perform.
1. Compile a safe language to C (whether a new language or an existing one)
2. Formal analysis of C, or of some practical subset of C, to prove the absence of undefined behaviour
Work has been done on both approaches.
ZZ compiles to C. [0] Dafny can compile to C++, but it seems that's not its primary target. [1][2]
There are several projects on formal analysis of C. [3][4][5][6]
[0] https://github.com/zetzit/zz
[1] https://github.com/dafny-lang/dafny
[2] https://dafny-lang.github.io/dafny/
[4] https://www.microsoft.com/en-us/research/project/vcc-a-verif...
(They look cool and I'm enjoying reading about them, but for the things I do for my day job where C is one of the obvious options, I'm not sure they're better options, but usually one of the languages above is a better option.)
Dafny has a slightly different emphasis, as it's about formal verification, so it's more competing with Ada SPARK than with mainstream languages.
I can't comment on the general ergonomics on ZZ and Dafny as I've not tried them out, but presumably they're going to be well behind a language like Java with its huge ecosystem of IDEs etc. Neither seems particularly mature, either, so I wouldn't bet the farm on them.
> Is it better to write ZZ or Dafny code than to write Java, Python, Haskell, Rust, or Swift code?
If you've got the choice and you're just trying to do 'normal day job programming' I imagine that yes you'd be much better off just using Java. If you want a language that's quite like C or C++ but much better regarding safety, there are Rust, Zig, and the too-often-overlooked Ada language.
How integers act at the extremes of their values, for example, is implememtation-defined — when you’re using the integer types (rather than using IEEE754 floats for everything to get well-defined formal semantics), you’re implicitly telling the compiler that you don’t care what happens at the boundaries, and that you want whichever behavior the target uarch’s ISA entails.
Think of it like exposing an opaque native FFI type in a managed-runtime language. The semantics of that type aren’t “up to” the runtime; all the runtime does is ensure that only a safe set of interaction primitives on the type are exposed into the managed runtime. (“Safe” in the sense of not corrupting the runtime’s own abstract machine semantics; not in the sense of making any guarantees about the operations leaving the value in a well-defined state per the underlying machine.) in C, integers — and the operations on them — are exactly such “opaque foreign types.” As are pointers, for that matter. (And this is also why casting between integers and pointers is UB — since the internal representations of each aren’t part of the semantic model of the C abstract machine, it cannot make any guarantee about their interconvertability.)
C compilers shouldn’t abort on implementation-defined behavior. Having that behavior be implementation-defined is the well-defined semantics that the C abstract machine applies to such things.
It’s only cases where there’s no clear semantics even per implementation, that the C compiler should swerve and say “I don’t know what that should mean; so I give up.”
Reading from a pointer past the bounds of the object being pointed to is undefined, not implementation-defined. How do you propose to write a size_t strlen(const char *s) that successfully compiles with your proposed compiler?
Sure, the "Why are you deleting my checks for if *this is null" is a little silly - but there are definitely sharp edges where UB conflicts with actually useful things you might want to do. Did you know seqlocks are undefined (benign race conditions)? Ever ran into padding concerns playing poorly with trying to do atomic CAS?
It's not unreasonable for the standard to say 'padding is undefined', 'data-races are undefined' - but having no way to say "hey, trust me, please un-poison this thing you don't like" is pretty unfortunate.
It wouldn't be undefined behavior if you used atomic variables--data races involving atomics aren't undefined behavior.
Unless things have changed a lot in the past two years neither LLVM or GCC do that much optimisation around atomics so this comes with disastrous performance implications as well as overhead battling the standard.
In practice (UB aside), this is basically fine in the context of a read, compute, CAS loop. Those bytes do have some value in the machine and if that memory isn't written they won't mysteriously change. It's playing games with the optimisers and UB however. You might be able to get around this by first initialising the bytes to zero, then in-place copy constructing whatever you want? I wouldn't bet anything serious on that being defined though.
I think the spec is written this way to codify how compilers are allowed to do liveness analysis... a local variable is marked as live when it is initialized, and if a variable is not live, its storage (registers or stack) may be used for other variables or temporary values. You then get UB because, for example, two comparisons may result in contradictory outcomes, because the value has been overwritten with garbage between the two comparisons. Or in a more modern compiler, you read from an uninitialized local variable, and the compiler trims the CFG.
Here's an example of what I'm thinking of:
int x, y;
// point A
if (condition) {
x = 5;
} else {
y = 10;
}
// point B