Falsehoods programmers believe about undefined behavior
predr.ag
predr.ag
Unfortunately this article doesn't refer to the best tool to root out UB - UBSan. If you write C or C++, you should have a UBSan+ASan build that is configured to terminate with an error when it encounters violations. You'll save yourself a lot of hair-pulling later. Then you don't have to wonder about the nebulous topics like what "line" the UB "happened" on.
For some compilers, when they add an optimization that capitalizes on an aspect of UB, they will also add a corresponding warning. So make sure you have warnings enabled -- and in most cases you'll want -Werror because generally no one scrutinizes the build output unless there's a failure.
So the only way to run them is too use an emulator, and that means I cannot sanitize drivers.
Besides that we have Zephyr stack and heap overflow detection, but it is not quite as accurate as ASan, and does not handle cases UBSan does.
This post is already fairly long (~10min read) and I didn't want to make it any longer. It's also already getting flamed in predictable ways ("nonsense! UB has to be reachable to be a problem!") so I didn't want to add more ways to get flamed for it (:
(I'm the post's author.)
> That's the whole list. No, I didn't forget any items. Yes, seriously.
I’m a little disappointed there isn’t an empty <ol> or <ul> in the middle there. I imagine screen readers would announce something like “list, zero items”.
I wish there were a best practices doc + a matching linter for accessibility things like this.
> I wish there were a best practices doc + a matching linter for accessibility things like this.
This sounds like it was meant to be a fun tweak/joke for screen readers but it's accessible without.
So it's not practical to automate a lot of accessibility checks like this one because it depends on context and there's subjectivity to it, similar to visual design and text readability. The best way to get more of a feel for this is to try a popular screen reader for your OS, do the tutorial, and try it on the page yourself.
In other words, UB!
True, if the language spec is the only thing in the universe you care about. But there are better and worse things compilers can do in the presence of UB, and the big one is warning you about it when it possibly can.
There's another big discussion on the front page about signed integer overflow in particular, and gcc screwing the developer over because it was UB when they thought it was just implementation defined:
Integer overflow is harder ofc.
No, the developer thought "UB" produced an integer with an undefined (but still valid integral?) value. They misunderstood what UB is, not whether or not something was UB.
x86 defines what specific instructions do, nothing so crass as "signed integer overflow" - that's part of why the language standard has to define it as such.
I'm not gonna wage a holy war against optimizing compilers, but let's not pretend that this is a language standard issue.
I don't understand what you mean. The x86 standard defines what happens if an `add` instruction is performed and the result overflows, possibly affecting the sign bit (what happens is that the OF flag is set, and the target register is set to the two's complement representation of the sum modulo 2^bitlength).
This is for the same sort of reason that unsigned overflow does have a well defined meaning in C, and will always do exactly that even for targets that do not natively do that (if necessary it would emulate that behaviour in software).
It kinda has an implied obligation to not screw with its users. And doing the same optimizations to fold bloated Cpp templates properly messes with my C code.
I am not an expert on compiler optimization, but having reasoned through a few cases where a compiler is _unable_ to correctly make an obvious-seeming optimization, I tend to think that compiler writers have good reasons for the things they want to be able to assume.
If there were good reason for wanting to assume this, I would expect someone to have presented it; responses I hear just take the form of a single (usually contrived) example where more performant code is generated, then insisting that all other factors (e.g. being predictable/unsurprising, acting like the underlying architecture, mitigating bugs) are irrelevant. Just as GP did.
If there weren't good reasons for making the assumptions that modern compilers do, I'd have expected users of C and C++ to have converged on a more conservative compiler, at least in some significant number.
The problem is that this is really hard to do without producing either too much or too little warnings, as undefined behaviour is usually input-triggered; and the compiler usually doesn't know all constraints on the arguments that the programmer knows about.
For example, the function f() in the article of the other discussion is perfectly well-defined for any argument x <= 0x402010, as in that case no overflow happens. The compiler doesn't know whether all callers ensure the argument is valid (e.g. this might be a library function, or the invariants might not be expressable in a way the compiler can understand). This is a lose-lose situation for the compiler: it can warn, but people will find the unnecessary warnings annoying and turn them off, while if it doesn't warn people will complain about perceivedly bad optimizations.
Of course there's a point to be made that the real problem here is that the C standard makes signed integer overflow undefined behaviour, but that's not gcc's fault and can be easily worked around with -fno-strict-overflow or -Wstrict-overflow.
This could still be greatly alleaviated with the use of asserts: A compiler could default to the most conservative set of assumptions for an input value - e.g. that a parameter of a global function assumes the full value range of its type - unless the programmer provides more assumptions through an assert statement.
In debug builds, you could compile those asserts to actual runtime checks, so the program can be appropriately tested. Then, in production builds, the asserts would simply be skipped.
The second improvement would be warnings about "nonensical" statements (from the compiler's POV) that will be optimised away. E.g., many of the most egregious examples of UB are from code where the programmer inserted a bounds check to protect against UB at another location - and then the compiler optimized away that same bounds check as a "nasal demon" action, caused by the UB it was supposed to protect against. In those cases, it would be useful to warn "hey, this bounds check can never fail because that other statement implies that either we're in bounds or there is UB. You might want to check for UB here."
And that proof, the compiler then decided to use to remove the code, ignoring the red flag that the programmer undoubtedly had tried (and failed!) to achieve something with it. I would have preferred a warning here. I don't turn off "condition is always true"-type checks.
GCC should add -Wundefined-behaviour flag that will unconditionally print "warning: this program might or not have UB" as the first diagnostic.
Seems pretty viable.
“An iteration statement that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.”
Rationale for including that: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1528.htm (basically, without it compilers can’t optimize loops that terminate as well as with it)
Hence for(;;) std::this_thread::sleep_for(1s) is not UB.
C is even more lenient.
The other important thing to realize is that UB is generally a runtime condition and cannot be detected statically, in the general case. That’s usually why it’s been made UB in the first place.
Compilers assume your code doesnt contain UB. Example:
If(x == 0) printf(“hello”); y /= x;
The compiler can assume x is not zero because it assumes the user will not cause UB, therfore the if statement and printf can be removed.
14. Okay, but if the line with UB is unreachable (dead) code, then it's as if the UB wasn't there.
15. If the line with UB is unreachable code, then the program won't crash because of the UB.
16. If the line with UB is unreachable code, then the program will at least stop running somehow and at some point.
This is nonsense. The line with UB has to be reached for the execution to be undefined.
"Anything at all can happen; the Standard imposes no requirements. The program may fail to compile, or it may execute incorrectly (either crashing or silently generating incorrect results), or it may fortuitously do exactly what the programmer intended."
"If any step in a program’s execution has undefined behavior, then the entire execution is without meaning. This is important: it’s not that evaluating (1<<32) has an unpredictable result, but rather that the entire execution of a program that evaluates this expression is meaningless. Also, it’s not that the execution is meaningful up to the point where undefined behavior happens: the bad effects can actually precede the undefined operation."
"If any step in a program’s execution has undefined behavior, then the entire execution is without meaning..."
Yes. So UB is a property of a given execition, not a static property of the program itself.
I don't argue that you can reason about the behavior of any part of the execution, once it evaluates operations with UB.
If a given execution would eventually lead to UB, then it retroactively it cannot be reasoned about.
This time traveling behavior is what is confusing people in thinking that if any execution is UB then all possible executions are UB.
Many people in this thread are confused and think that unreachable UB (either statically or dynamically unreachable) compromises the entire program. This is not true, but your comment helps me better understand how people reached this conclusion.
Reachable UB (either provably reached at compile time or dynamically reached at runtime) can retroactively invalidate the correctness of previous statements. But this does not mean that unreachable UB compromises the correctness of well-defined executions.
int main(int argc, char **argv) {
int a = atoi(argv[1]);
int b = atoi(argv[2]);
return a + b;
}
So if you have UB in your future path of execution, the compiler might just do whatever _right now_.
You are right that once the execution is undefined, you can't reason about the behavior. This includes ordering of side effects, or side effects od operations occurring before the line with UB.
But uttering UB in dead code making execution UB is utter nonsense. Otherwise any execution of any program that just utters __builtin_unreachable() would be undefined.
However, even when processors execute code speculatively, they guarantee that no observable effects of executing that code will happen. So, if you have something like `if false {*NULL=7/0}`, the processor might attempt to execute that line speculatively, but it will revert any change it made, and it will not trap regardless of any other flags.
Check the footnote at item 14, it links to a page which shows one example of how a line with UB can be reached (basically: the optimizer can reorder the code so that parts of the line with UB will run much earlier than you'd expect).
Yes UB in live code can have surprising effects, including executing dead code.
I think the main reason for this is an if(testSomeState()) { //code with UB } block being optimized to just //code with UB.
Functions which are unreachable, maybe I agree, they can't be executed (without UB somewhere in reachable code). I'm not sure that's true though - if you have UB in your hardware (not your code), you can still execute that code. This is why (at least I was taught) never to leave dead code around. There's this better thing for that called source control you may have heard of...
IF you have perfect hardware, and IF you have perfect logic/compiler and IF you have no UB in your program, then maybe you can guarantee that dead code is really dead, but I'm not really sure you want to heavily define dead code like that.
Just remove it. Especially if it has UB in it to cause more fun bugs...
¹or is guaranteed to happen in the future
> Right now, we have the fundamental principle that dead code cannot affect program behavior.
For example, if a program looks like this:
void foo(int* const p) {
if (p!=NULL) {
bar(p);
}
*p = 1; //this line would be UB if p could be NULL
//so we are free to assume p is not NULL in this scope
}
void foo(int* const p) {
bar(p);
*p = 1;
}
This is a good example of how UB on a later line can have an impact on earlier lines, or even in other places in the code.
Note that it's not executions which trigger UB, it is source code programs that do.
Edit: fixed const pointer syntax.
It is true that when foo() is called with a NULL argument, the UB will be realized before bar() has had a chance to run (EDIT: this doesn't actually appear to be true for the given code sample, neither Clang nor GCC elide this NULL check: https://godbolt.org/z/v9v74c3jd, which makes sense as bar() could call exit()).
> Note that it's not executions which trigger UB, it is source code programs that do.
For C, I think this interpretation is directly contradicted by DR #109: https://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_109.html
> Furthermore, if every possible execution of a given program would result in undefined behavior, the given program is not strictly conforming. A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior. Because foo might never be called, the example given must be successfully translated by a conforming implementation.
The point of those numbered lines was to refer to how people normally think about UB. Someone seeing that function may be tempted to say that it's not UB to call with a NULL value as long as the dereference doesn't happen (say, bar() could call exit() or it could run an infinite loop). What you're saying is exactly what the article is saying: the fact that a line with UB is unreachable in practice doesn't mean that it can't affect your program.
> For C, I think this interpretation is directly contradicted by DR #109: https://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_109.html
The standard talks about executions in the sense of theoretical executions by the C abstract machine, not actual executions of the compiled binary. I thought the other comment was rendering to the latter notion of execution, but perhaps I was wrong and it was also talking about the former.
An infinite loop is UB for precisely this reason: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1528.htm So an infinite loop cannot prevent the invocation of UB, because an infinite loop is itself UB.
I would expect that a call to exit() inside bar() would prevent eliding of the NULL check. If the program calls exit() before the dereference, it does not invoke UB and therefore should run correctly. I tried your example and Clang and GCC do not actually elide the NULL check in this case, probably for this reason: https://godbolt.org/z/cnWPz3o59
> The standard talks about executions in the sense of theoretical executions by the C abstract machine, not actual executions of the compiled binary. I thought the other comment was rendering to the latter notion of execution, but perhaps I was wrong and it was also talking about the former.
I don't see how this would change the analysis. Running the binary triggers actual executions of both the abstract machine and the physical binary.
Still gcc doesn't optimize the check away so it is possible there is some other line of reasoning that prevents the transformation.
My first answer was going to be that the transformation is not allowed, but then I thought about it more.
I couldn't get gcc to break it either though. Either it misses the optimization or tries to be conservative with reordering UB with side effects.
I'm afraid it's not nonsense, that's why it's a common misunderstanding ;)
> Right now, we have the fundamental principle that dead code cannot affect program behavior.
Why are people fighting this?
And yes, Rust is much more reasonable about it.
If you are not satisfied, you can read the article's references (there are 2 for unreacheable code). Or any article about UB here for the last 20 years.
void bar(int* const p, int n) {
for (int i=0; i<n; ++i) {
int m = *p
// other code (without UB)
}
}
void bar(int* const p, int n) {
int m = *p
for (int i=0; i<n; ++i) {
// other code (without UB)
}
}
Edit: Oops, in the previous version I used a write instead of a read on *p, making the optimization invalid.
If you had *p == 0 before calling bar(p, 0) then *p had better not be 1 after!
If the compiler has some special knowledge of the target architecture which makes the second form behave as if the first form was executed (such as if NULL is a valid address on the target architecture), it may make the transformation, but that still wouldn't cause undefined behavior, because it must behave as if it had the first form.
For example, imagine a C programmer writes a function that adds two `int` values. Hypothetically, with some inputs, the addition may overflow. That would be undefined behavior, so the compiler assumes the function is never called with such inputs, and optimizes accordingly. The programmer knows this, and assures the function is never called with those inputs. Since the only assumption the compiler made was that the function wasn't called with input which would cause the addition to overflow, and the programmer assured that assumption is correct, the function always behaves correctly.
Consider if the same function's inputs were provided at runtime from stdin. The behavior of the whole program is undefined if invalid inputs are given, but if only valid inputs are given, the program must behave correctly.
So we could say, unreachable UB is fine, but if you do reach it, then it doesn't matter when you reached it.
The implicit assumption is no UB in live code, otherwise the UB in dead code is a red herring.
So the UB is never reached, by the definition of dead code.
Unreachable UB is fine.
If this wasn't the intended message of point 14, then it's phrased wrong.
This one came as a surprise to me. Is this really (unconditionally) true? And if so, why? Seems trivial to me to ensure that UB that will never under any circumstance execute will not affect the functioning of an otherwise correct program.
Or, here is a better example:
for(int i=0; i > n;) {
}
*p=1;
Then, since the loop terminates, p will be dereferenced, so it can be assumed that p is never NULL, so any NULL checks for p in this context can be ellided.
This remains valid even if you call this function with an n such that *p=1 is unreachable.
> Furthermore, if every possible execution of a given program would result in undefined behavior, the given program is not strictly conforming. A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior. Because foo might never be called, the example given must be successfully translated by a conforming implementation.
To hammer it in, the consider the following:
if(p) *p = 0;
*p is undefined if p is NULL. But as long as that statement is never reached when p is NULL that is not a problem. That is trivially true in this case, but the same applies even for more complex relations between the conditional and the undefinedness of the statement.
Or put another way, in
int * p = 0;
if(fermats_last_theorem_is_wrong()) {
p = malloc(sizeof(int));
*p = 0;
}
if(p) {
puts("fermats last theorem is wrong\n");
}
if(fermats_last_theorem_is_wrong()) {
*p == 1;
}
if(p) {
puts("fermats last theorem is wrong\n");
}
In fact, something having undefined behavior and the compiler being allowed to assume that it is unreachable are effectively the same thing. Which is why the spec for std::unreachable() and the compiler-specific intrinsics that preceded it is that it invokes undefined behavior and not more complex wording.
Please correct my misunderstanding...
if (p is not NULL) {
dereference p
}
No it won't. Otherwise all precondition checks would be meaningless. The compiler can only assume that it isn't null when fermats_last_theorem_is_wrong() is true. If the compiler cannot reason about fermats_last_theorem_is_wrong() then it can't just ignore it.
> for(int i=0; i > n;) { } *p=1;
> Here the compiler will assume that the loop terminates, since non-terminating loops without side-effects are UB in C++.
Infinite loops are UB precisely because otherwise the compiler would NOT be able to reason about this. So yes, here the comiler can assume that p is a valid pointer - but only because it can also assume that *p=1 is NOT dead. This falls entirely under the umbrella of behavior not being defined at all after you execute undefined behavior (the infinite loop).
> This remains valid even if you call this function with an n such that *p=1 is unreachable.
Only because the loop itself is undefined behavior in that case.
An infinite loop is not UB. It is well defined. Just the definition of a terminating loop is a bit funny.
> An iteration statement whose controlling expression is not a constant expression, that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression, may be assumed by the implementation to terminate.
Plus:
> An omitted controlling expression is replaced by a nonzero constant, which is a constant expression.
The compiler may only use as-if rule here if it can deduce the value is constant or limited. It can (but does not have to) assume the loop will end due to the conditional and lack of the other features. Which is still defined behavior. (since C11 at least, probably earlier)
From C11:
> An iteration statement whose controlling expression is not a constant expression,156) that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.157)
> 157)This is intended to allow compiler transformations such as removal of empty loops even when termination cannot be proven.
This makes infinite loops UB unless they fall under one of these exceptions (constant controlling expression, performs I/O, accesses volatile objects or atomics).
It only allows the compiler to assume loops with non-constant expressions can terminate. (And a bunch of other exceptions where it cannot.) A true infinite loop has a constant expression as the conditional and cannot be optimized away based on this rule.
What this rule allows the compiler to do is to optimize away loops where number of iterations are known ahead of time if it has a non-constant expression condition.
That is basically what I said: loops can be assumed to terminate, except for an enumerated set of exceptions where infinite loops are allowed.
You seem to be defining "true infinite loop" to mean "while(1)" or "for(;;)" exclusively. But "for (int i = 0; i < 1;)" is also an infinite loop, and that one is UB since the controlling expression is not constant.
> What this rule allows the compiler to do is to optimize away loops where number of iterations are known ahead of time if it has a non-constant expression condition.
If the number of iterations are known ahead of time, then it is not an infinite loop so I don't see how the rule would apply.
The rationale for making infinite loops UB is described here: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1528.htm
Specifically, it allows for transformations like turning:
for (p = q; p != 0; p = p -> next) {
++count;
}
for (p = q; p != 0; p = p -> next) {
++count2;
}
for (p = q; p != 0; p = p -> next) {
++count;
++count2;
}for (;;) { }; cannot be assumed to have finite number of executions. Therefore, it cannot be optimized away to finite execution.
You're showing potentially infinite loops with non-constant expression as condition.
for (; i != 0; ) {} can be assumed to have finite number of executions and can be optimized unless i is constant expression.
Yes, because a constant controlling expression is one of the enumerated exceptions to the rule against infinite loops.
> You're showing potentially infinite loops with non-constant expression as condition.
"for (int i = 0; i < 1;)" is an infinite loop, full stop. It is not potentially infinite, it is infinite under all possible executions. It is UB because it does not fall under one of the enumerated exceptions where an infinite loop is allowed.
I think we are saying the same thing, the only difference is that you seem to be defining "infinite loop" to only include loops with constant controlling expressions, whereas I am defining it to mean "any loop that does not terminate."
The example with the loop though is more to point out that our native understanding of "unreachable" may be different than the compiler's. The same problem could occur without invoking UB if we un-conditionally generate a SIGKILL to our own process and then dereference a NULL pointer - the compiler doesn't know what SIGKILL does, so it doesn't know the UB is unreachable in practice, so it may reorder etc.
It is the reverse. If the compiler doesn't know what raise(SIGKILL) (or really, any other function it cannot see through) does, it has to assume that it can potentially never terminate, abort or otherwise change the control flow, so it can't safely reorder across it. And even if it does return, it could still inspect memory and observer the violation of the as-if rule.
That's why for example pthread_mutex_lock/unlock worked correctly for the most part even before C++ got a proper concurrent memory model.
The compiler can't know that an arbitrary part of a function is unreachable if, for instance, that code path is controlled by a parameter or global state since that equates to solving the halting problem.
https://blog.llvm.org/2011/05/what-every-c-programmer-should...
DR #109 specifically states that a compiler may not fail to translate a strictly conforming program just because some possible executions could trigger UB: https://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_109.html
Edit: downvoters, please explain your disagreement. I cited a primary document in support of my position.
It's just that the generated code may not do exactly what you expect it to do because the presence of undefined behavior allowed the compiler to make assumptions which may surprise you.
edit: It seems that I am incorrect
> A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior.
This text specifically allows for the case that a program is strictly conforming even if there is a possible execution that invokes UB. If a program is strictly conforming, it must produce the correct behavior.
I would challenge you to show an example where GCC or Clang will break the correctness of a program on account of UB that is not reached during program execution.
Here is an example where GCC and Clang specifically respects the correctness of a program as long as it does not reach the UB: https://godbolt.org/z/befWah77W
At the very least I am unable to construct a counter-example so I concede this point
> The Rust compiler has a few assumptions that it makes about the behavior of all code. Violations of those assumptions are referred to as Undefined Behavior.
Then it says
> In other words, even just constructing, for example, an invalid bool, is Undefined Behavior—no matter whether that bool is ever actually “used” by the program.
The blog post is talking about how just creating an invalid/trap/niche representation for a value is UB. The act of creating the value is the UB here, not the usage. So, in this case the UB is most definetely reachable and executing (The act of creating the invalid value).
The compiler has to respect DR 106. The linker, however, is allowed to write garbage into code.
Most people driving cars with an active recall are unaffected by the problem the recall is supposed to address. They can drive on blissfully unaware that something could go wrong. They might even enthusiastically recommend that others purchase the car they drive.
For a few people, they'll experience a failure, possibly a dangerous one.
And this is the root of the problem. The fundamental concept of using a language prone to UB exposes all programs to a small risk. In isolation, the risk is small. In aggregate, the problem is real.
Should we try our best to minimize the risks? Absolutely. But we are talking programs here. We shouldn't measure every program with the same ruler. Not all programs need to be MISRA compliant when they don't need to.
My Reddit app crashes several times a day, and I guess there is no immediate danger.
> My Reddit app crashes several times a day, and I guess there is no immediate danger.
That's because your app is heavily sandboxed and probably (on Android, at least) running in a VM which enforces memory safety and isolation from other apps.
If each crash had, say, a 1% likelihood of eating a large portion of your data (or leaking pictures or whatever) you'd probably care a lot more about it.
The problem highlighted by the parent poster is that all these little risks add up and often end up being catastrophic (e.g. heartbleed) because so much of what we run is interconnected via the network (or just data in files).
I tried to cover as much ground in the post as possible, but the post is already a 10min read and covering unspecified / impl-defined behavior would have made it a 20min read instead :)
If, in your description of the three buckets, you called the middle one something else like "Platform-dependent behaviour" or whatever, I think that would have been less weird. You'd still only need one short paragraph to say it's compiler/OS/hardware dependent, covers implementation-defined and unspecified, and is not the focus of the article.
C needs a mechanism to declare in a source file that certain behaviour must be defined, where that behaviour is undefined in the C standard but the target architecture behaves the defined way. Then, when the source is compiled on the target architecture it works as expected, but when compiled on something else it refuses to compile.
For example, unsigned ints. It'd be great to be able to declare "I define int to be 32 bits and to overflow like a sensible 32-bit int - treat this as defined behaviour, and refuse to compile on a system where this isn't true." The number of platforms where this would fail to compile would be extremely small, and at least they would give a sensible compile error on systems that are weird.
(I'm the post's author.)
`-fwrapv` is an attempt toward doing such a thing for signed integer overflow specifically. I bet it's a pain to maintain because it significantly increases the surface area of the compiler that needs to be tested.
I don't believe doing this for more cases is feasible at the level of the compiler. Things like CHERI where the hardware and OS help are much more viable, in my book: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
As far as I can tell there is no reason LLVM couldn't compile a document like "On x86 division by zero throws a #DE exception. On ARM division by zero returns zero. On x86 shifting by more than 32 bits shifts by (cnt mod 32) bits. On ARM shifting by more than 32 bits clears all bits. [...]" that specifies all undefined behavior. The current "nasal demons" behavior is preferred in the name of performance on some SPEC benchmarks that nobody was asking for, screwing OS writers and anyone who expected sane language semantics.
As for your example, unsigned overflow is already defined in C and C++ and you can force the compiler error for non-32-bits using a static_assert.
The purpose of C is to be easy to implement on new systems. You're never going to get a lot of traction adding anything to C proper.
It is ok, the accelerometer readings won't go outside the range of an int.
What's nice about Ada is you have choices, and the ecosystem facilitates creating and using such tooling.
And in a way, I think you did say something deep. Web developers often have no idea that problems like this were solved a long time ago, and it lowers their expectations for what should be possible with the tools of their industry.
- Compilers and language designs could not improve on this situation without sacrificing performance or portability.
It is not significantly better than running your test cases with with UBsan but it is a nice feature.
Leveraging what the compiler gives you is like that saying "the best camera is the one you have on you".
I love constexpr because it allows me to write compile time unit tests inline with the library code, with static_assert. The implied UB check is gravy.
It's still super useful though, and more common language UBs are caught.
Consider that something like (++*ptr1) + (++*ptr2) may or may not be UB, depending where the pointer points to. The required additional bookkeeping might make compile time evaluation unpractically slow.
Alternatively the sequencing could be always defined between any two subexpressions within constexpr context.
The purpose of this post and other "falsehoods" posts is to shine a light on the existence of a misconception or set of misconceptions. By that point, there are usually plenty of other high-quality posts about "why / how / when" etc. but for a variety of social dynamics reasons those posts by themselves did not have the wide distribution and readership they deserved. After shining a light on the misconception, interested readers are generally able to continue the learning process both through the sparked discussion and through doing their own searching.
Here's a specific example of such a post with even more details: https://blog.llvm.org/2011/05/what-every-c-programmer-should...
If an UB adopt X behavior one time, a reproducible build will take same X behavior the next time.
Of course I am not negating the fact that this undefined in the first instance nor condoning the UB usage.
It isn't a bug in the compiler if the compiler is non-deterministic in the presence of UB. Such behavior does not violate any guarantee provided by the compiler or language spec.
In fact, it's still not a bug if the compiler is non-deterministic, full stop. Many (most?) compilers have non-deterministic behavior due to a variety of factors. To get a sense of what's involved in getting determinism, look up some articles on why it's difficult to get reproducible builds working. It's a lot harder than it might seem at first.
"Sometimes gcc will opt to use a randomized model to guess branch probabilities, when none are available from either profiling feedback (-fprofile-arcs) or __builtin_expect. This means that different runs of the compiler on the same program may produce different object code." (from the gcc manual)
The bunch of amateur bards trying to mythicize undefined behavior really need to find something better to do with their and our time…
The truth is that practice diverges from theory, and, in practice, it is actually useful to talk about UB as a thing that happens and not as some mystical aura the compiler can sense or not. Leading to, in practice, almost all of these points being, if not flat-out wrong, at least unhelpful or misleading to the average programmer. For example:
> 14. Okay, but if the line with UB is unreachable (dead) code, then it's as if the UB wasn't there.[1]
The linked post ([1]) doesn't even showcase this phenomenon, it shows something completely different, where ignoring a 3 transmuted into a bool causes more UB. The problem was that you already had UB to begin with when transmuting a 3 into a bool. I want to see a godbolt output where the claimed:
if (false) {
run_ub();
}
> 28. At least it won't completely wipe the drive.[2]
This is just unhelpful for anyone trying to learn more about UB. UB can call parts of your code that would otherwise be dead, which is a useful piece of information to know. And if you literally write "rm -rf /" in these dead sections, then yeah, I guess UB is technically wiping your drive, but stating it like that just gets in the way of anyone trying to learn more about UB and just perpetrates this unhelpful theory that UB can literally do anything it wants, such as...
> 30. At least it won't start playing Doom if the program didn't already have the Doom source code in it.
Really?
> 31-32. If a UB-containing program "worked fine" previously, recompiling the program without any code changes will still produce a binary that "works fine." Recompiling without code changes and with the same compiler and flags will produce a binary that still "works fine." Recompiling as above + on the same machine will produce a binary that still "works fine."
In practice, compilers are generally deterministic. I see no reason this should be the case. Don't link me a sadistic compiler that randomizes code output.
Finally,
> The moment your program contains UB, all bets are off.
is blatantly untrue. All guarantees are off, but I can bet what most programs with UB do, and it is still useful to be able to reason about what programs with UB do. For example, I can tell you what:
int main(int argc, char \*argv) {
int *p = NULL;
*p = 4;
}
You guys are engineers right? Software engineers? Why don't you do what all engineers are supposed to do and that is use experience and intuition to know when theory applies in practice, and when it doesn't?
[1]: https://www.ralfj.de/blog/2020/07/15/unused-data.html
[2]: https://kristerw.blogspot.com/2017/09/why-undefined-behavior...
Welcome to kernel code. I believe one could make it play DOOM by clever malice in this case.
> When a C or C++ program triggers undefined behavior, anything is allowed to happen in the program execution. And by anything, I really mean anything: The program can crash with an error message, it can silently corrupt data, it can morph into a colorful video game, or it can even give the right result.
-- https://www.nayuki.io/page/undefined-behavior-in-c-and-cplus...
> This is just unhelpful for anyone trying to learn more about UB. [...] And if you literally write "rm -rf /" in these dead sections, then yeah, I guess UB is technically wiping your drive, but stating it like that just gets in the way of anyone trying to learn more about UB and just perpetrates this unhelpful theory that UB can literally do anything it wants
The example is contrived, yes. But even if the source code doesn't literally contain "rm -rf /", if the program contains a buffer overflow vulnerability, then someone can craft an input to inject arbitrary code/commands into the program execution and still effect a deletion anyway. So yes, UB really does mean anything can happen.
and
> The moment your program contains UB, all bets are off.
Does this include that a compiler, when it can prove that a program it compiles contains UB, erase your disk as part of the compilation process? i.e. without you ever running the compiled program.
No, but technically compiler would not violate specification by compiling program that on running will scan your disk for secret data, publish it online, apply ransomware to your disk and deliberately destroy your SSD.
void main(){};
If yes, then compiler can compile program doing literally anything* and be in compliance with standards as defined.
*including hacking Pentagon to order drone strike on location where it was compiled.
Or summoning demons.
Or deleting your file system through a system call.
So, yes, the C specification allows a standard-conforming compiler to erase your disk during compilation. It even allows it to do that if your code doesn't contain UB.
In practice, it means all bets are off, and yes, depending on your code, you may lose some data.
UB is (usually) input dependent as well. So shouldn't this be phrased as
> If the compiler produced a binary that on some input leads to UB and does X, is that a compiler bug?
I think this is their point. :)
… "what the compiler guarantees in the presence of UB"
I cannot say I have ever written anything that ran into or caused undefined behavior. And I have programmed for a long time.
Is this just an interesting topic or do people actively push the compiler to see what it will do? Maybe I am a boring/simple programmer. :)
But how would you know? :)
I suspect every C programmer invokes undefined behavior in their first 50 hours of writing C.
For other cases (e.g. aliasing, alignment), I'd agree that one is rather safe as long as the dangerous tools (e.g. reinterpret_cast) are not used.
If the compiler removes an important check or branch like in the post, and it isn't noticed in development chances are the code is dead, or not that important. And I'm not saying this to downplay the risks. A big risk is that a particular instance of UB is exploited only by a newer version of the compiler, after the code is already finished. But in general, the language remains a remarkably productive one that lets me get very close to where I want to be, very quickly.
Is it possible to know for sure that it's all correct and will continue to be correct with newer compilers? Not 100%, but maybe it's still a good tradeoff in some environments.
Assuming you're working with C or C++, you've never:
- Dereferenced a pointer? - written a loop with a signed index? - Accessed an index in an array or a vector? - Used a variable? - Used an optimising compiler?
Compilers use undefined behaviour in all of the above scenarios as much as possible, and to do what you likely intended. That's not to say _all_ UB is as reasonable, but a significant amount of what you write relies on some implicit assumptions about UB for a compiler to output reasonable code.
If during that long time you were writing C or C++, it's very likely that you did but you may not have attributed the bugs you were fixing to Undefined Behavior. You may have considered them either to be your bugs or "compiler bugs that you worked around."
Back when I worked on VAX/Alpha (I am old) I absolutely had to get down and read assembly/binary to figure out the compiler was doing what I thought. And in some cases had to work around compiler bugs.
But for most of the more recent things I have written I just have not run into these types of errors. I am really boring or I write code that is boring ha.
It is fascinating that people study this stuff and interesting to read about, I just do not run into it.
Perhaps this is true, or perhaps you are just unaware of the undefined behavior you have relied on.
What I can say for sure, is that all of the non-trivial C++ programs written by a large team which I have worked on in my career have invoked some undefined behavior. Usually, the compiler does do "the right thing" despite the error, but sometimes it blows up in our face, and causes incredibly hard to debug bugs, such as a pointer appearing to simultaneously point to two different addresses, because an array was declared as two different sizes in two different translation units.
Even when I spot the instances of undefined behavior, it's often hard to convince other developers that, although this seems to behave correctly now, it could break in unexpected, hard to debug ways at any time in the future.
That interpretation of the specification language is clearly bogus as that means more or less all C and C++ programs can be compiled to a single return instruction - when compiler devs introduced this new new interpretation of UB, it turned out they had to then go through and add a bunch of exceptions for basics like memset, memcpy, and memmove could work without falling afoul of their invented nonsense.
> That interpretation of the specification language is clearly bogus as that means more or less all C and C++ programs can be compiled to a single return instruction - when compiler devs introduced this new new interpretation of UB, it turned out they had to then go through and add a bunch of exceptions for basics like memset, memcpy, and memmove could work without falling afoul of their invented nonsense.
I've never heard of anything like this, have you any sources for this happening?
I think aliasing rules messes up memcpy if you are dogmatic?
> I think aliasing rules messes up memcpy if you are dogmatic?
I'm _not_ dogmatic or a language lawyer and aliasing is a total pain in the ass, but that's a whole different topic to compilers getting fundamentals wrong because of language lawyering.
[0]: https://stackoverflow.com/questions/49298704/how-reason-abou...
memmove fails as it requires pointer comparisons between separate allocations, which is UB.
Comparing unrelated pointers is unspecified, not undefined. It's also an implementation detail of memmove. It can use a magical total order comparison of pointers that is otherwise unavailable to users of the language.
memset and memcpy read and write padding bytes, which is UB. If you try to say memset/cpy is a "primitive operation", then you're still stuck with the "correct" struct initialization (memset, bzero) being UB if you did something like:
void f(bool condition) {
if (condition) {
struct structwithpadding thing;
bzero(&thing, sizeof(thing));
} else {
printf("wat\n");
}
}
void f(bool condition) {
printf("wat\n");
}
memmove falls to UB much requires pointer comparisons between pointers to different objects, which is UB.
Compiler designers just chose the notion of "obviously user wouldn't invoke the footgun, so any code that would be causes by footgun can be removed"
Since, in the process of optimization, optimizing compilers make conclusions about parts of a program based on other parts of the program which may be executed later, undefined behavior in one part of a program causes invalid conclusions about other parts which may be executed earlier. Therefore undefined behavior cannot be contained to just parts of a program executed after a particular part, if the compiler tries to optimize parts of the program based on parts that may be executed later.