GCC undefined behaviors are getting wild | Hacker News Reader

GCC undefined behaviors are getting wild

blog.pkh.me

156 pointssigna11349 comments

As others are pointing out, the C standard does allow this. There is no safe way to check for undefined behavior (UB) after it has happened, because the whole program is immediately invalidated.

This has caused a Linux kernel exploit in the past [1], with GCC removing a null pointer check after a pointer had been dereferenced. Null pointer dereferences are UB, thus GCC was allowed to remove the following check against null. In the kernel, accessing a null ptr is technically fine, so the Linux kernel is now compiled with -fno-delete-null-pointer-checks, extending the list of differences between standard C and Linux kernel C.

[1]: https://lwn.net/Articles/342330/

> because the whole program is immediately invalidated.

The problem is the program isn't invalidated, it's compiled and run.

The malicious compiler introducing security bugs from Ken Thompson's "Reflections on Trusting Trust" is real, and it's the C standard.

I will grant that trying to detect UB at runtime may impose serious performance penalties, since it's very hard to do arithmetic without risking it. But at compile time? If a situation has been statically determined to invoke UB that should be a compile time error.

Also, if an optimizer determines that an entire statement has no effect, that should be at least a warning. (C lack C#'s concept of a "code analysis hint" which have individually configurable severity levels).

> If a situation has been statically determined to invoke UB that should be a compile time error.

That's simply not how the compiler works.

There is (presumably, I haven't actually looked) no boolean function in GCC called is_undefined_behavior(). It's just that each optimization part of the compiler can (and does) assume that UB doesn't happen, and results like the article's are then essentially emergent behavior.

C++ bans undefined behavior in constexpr, so you can force GCC to prove that code has no undefined behavior by sprinkling it in declarations where applicable:

https://shafik.github.io/c++/undefined%20behavior/2019/05/11...

Constant-evaluated expressions with undefined behavior are ill-formed but constexpr annotated functions which may in some invocations result in undefined behavior are not.

It is undefined behaviour if I write GCC --hlep

Does that mean it's acceptable for GCC to reformat my hard drive?

Just because something is UD doesn't give anyone a license to do crazy things.

If I misspell --help I expect the program to do something reasonable. If I invoke UD I still expect the program to do something reasonable.

Removing checks for an overflow because overflows 'can't happen' is just crazy.

UD is supposed to allow C to be implemented on different architectures if you don't know whether it will overflow to INT_MIN it makes sense to leave the implementation open. If I, the user knows what happens when an int overflows then I should be able to make use of that and guard against it myself. A compiler undermining that is a bug and user hostile.

> It is undefined behaviour if I write GCC --hlep

No, it's not, and I don't know why you'd think so. UB is a concept applying to C programs, not GCC invocations.

> UD is supposed to allow C to be implemented on different architectures if you don't know whether it will overflow to INT_MIN it makes sense to leave the implementation open. If I, the user knows what happens when an int overflows then I should be able to make use of that and guard against it myself.

I think you're confusing UB with unspecified and implementation defined behavior. It's fine if you think something shouldn't be UB, but you have to go lobbying the C standard for that. Compiler writers aren't to blame here.

This has come up before, because, in some technical sense, the C standard does indeed not define what a "gcc" is, so "gcc --help" is undefined behavior according to the C standard, because the C standard does not define the behavior. By the same token, instrument flight rules are undefined behavior.

A slightly less textualist approach to language recognizes that when we talk about C and UB, we mean behavior, which is undefined, of operations otherwise defined by the C standard.

I think this is confusing undefined behavior with behavior of something that is undefined. And either way, the C standard explicitly applies to C programs, so even this cute "textualist" interpretation would be wrong, IMO.

Do you know what a metaphor is?

No GCC --hlep isn't in the c standard.

But it is a simple example to illustrate how programs react when it receives something that isn't in the spec. GCC could do anything with Gcc --hlep just like it could do anything with INT_MAX + 1. That doesnt mean that all options open to it are reasonable.

If I typed in GCC --hlep I would be reasonably pissed that it deleted my hard drive. You pointing out that GCC never made any claims about what would happen if I did that doesn't make it ok.

If you come across UD, there's reasonable and unreasonable ways to deal with that. Reformatting your hard drive which is presumably allowed by the C standard isn't reasonable. I would contend that removing checks is also unreasonable.

>No, it's not, and I don't know why you'd think so. UB is a concept applying to C programs, not GCC invocations

What should happen when I invoke --hlep then? The program could give an error, could warn that it's an unrecognised flag. Could ask you if you meant --help. Infer you mean help and give you that, or it could give you a choo Choo train running across the screen. Or it could reformat your hard drive. Just because it isn't specifically listed as UD doesn't mean it's not. If it isn't defined then it's undefined. The question is what is the reasonable thing to do when someone types --hlep. I hope we can agree reformating your hard drive isn't the most reasonable thing to do.

>I think you're confusing UB with unspecified and implementation defined behavior

Am I? What's the reason for not defining integer overflow? Yes unspecified behaviour could be used to allow portability, but so can undefined.

>It's fine if you think something shouldn't be UB, but you have to go lobbying the C standard for that. Compiler writers aren't to blame here.

I'm not saying it shouldn't be UB. I'm saying there's reasonable and unreasonable things to do when you encounter UB. In the article the author took reasonable steps to protect themselves and the compiler undermined that. That isn't reasonable. In exactly the same way that --hlep shouldn't lead to my hard drive getting reformatted.

C gives you enough rope to hang yourself. It isn't required for GCC to tie the noose and stick your head in it though.

I think you're confusing UB with unspecified and implementation defined behavior

> What should happen when I invoke --hlep then? The program could give an error, could warn that it's an unrecognised flag. Could ask you if you meant --help. Infer you mean help and give you that, or it could give you a choo Choo train running across the screen. Or it could reformat your hard drive. Just because it isn't specifically listed as UD doesn't mean it's not. If it isn't defined then it's undefined. The question is what is the reasonable thing to do when someone types --hlep. I hope we can agree reformating your hard drive isn't the most reasonable thing to do.

I honestly don't understand the point of this paragraph.

> Am I? What's the reason for not defining integer overflow? Yes unspecified behaviour could be used to allow portability, but so can undefined.

Yes, you are confused about that. UB is precisely the kind of behavior where the C standard deemed it unsuitable to define as implementation defined or whatever, and it usually has really good reasons to do so. You could look them up instead of asking rhetorically.

> I'm not saying it shouldn't be UB. I'm saying there's reasonable and unreasonable things to do when you encounter UB. In the article the author took reasonable steps to protect themselves and the compiler undermined that. That isn't reasonable. In exactly the same way that --hlep shouldn't lead to my hard drive getting reformatted.

Again, you seem to fundamentally misunderstand how compilers work in this case. They largely don't "encounter" UB; It's optimization passes are coded with the assumption that UB can't happen. The ability to do that is fundamentally the point of UB. Situations like in the article are not a specific act of the compiler to screw you in particular, but an emergent result.

Additionally, I think you you're also confusing Undefined Behavior with 'behavior of something that is undefined'. These are not the same things.

> It's fine if you think something shouldn't be UB, but you have to go lobbying the C standard for that. Compiler writers aren't to blame here.

I'm glad I don't live in your country, where the C standard has been incorporated into law, making it illegal for compiler writers to do things that are helpful to programmers and end users, but aren't required by the standard.

> UD is supposed to allow C to be implemented on different architectures

No, that's wrong. Implementation-Defined Behavior is supposed to allow C to be implemented on different architectures. In those cases, the implementation must define the behavior itself, and stick with it. UB, on the other hand, exists for compiler authors to optimize.

If you want to be mad at someone, be mad at the C standard for defining so much stuff as UB instead of implementation-defined behavior. Integer overflow should really be implementation-defined instead.

>No, that's wrong. Implementation-Defined Behavior is supposed to allow C to be implemented on different architectures.

Is it? We're talking about integer overflow here.

I wasn't in the meetings when writing all the c standards. I'm not convinced this is purely an optimisation thing though.

I would guess the story is more.

Interested party X: "can integer overflow do X?"

Party Y: "no because our processor doesn't work like that.

Party Z: "and it breaks K and R"

Party X: "how about implementation defined?"

Party A: "but our compiler targets 5 different processors"

Party B: "plus that precludes certain optimisations"

Not only to optimize but to write safety tools. If you defined all the behavior, and then someone used some rare behavior like integer overflow by accident, it'd be harder to detect that since you have to assume it was intentional.

> UB, on the other hand, exists for compiler authors to optimize.

Was this really the original reason why there's UB in the C standard, or has this been retconned by 'malicious compiler authors'? ;)

It is the original reason. For example, register allocation is possible because stack smashing is UB.

But then again, UB doesn't mean the compiler author can't treat it as implementation-defined and do something reasonable.

You're getting it backward. UB doesn't immediately stop compilation only due to implementation defined backward compatibility, just because you don't want to break compilation of existing programs each time the compiler converges to the C spec and identified an implementation of undefined behavior.

And since you want some cross-compiler compatibility, you also import's third parties implementation defined UB.

This is not some conceptual reasonable decision, the proper way would be to throw out compilation on each UB behavior. The reality is that the proper way would be too harsh on existing codebase, making people use a less strict compiler or not updating version, which are non-desirable effects for compilers writers.

And sanitizers that throw warnings on undefined behavior do indeed exist.

>UB, on the other hand, exists for compiler authors to optimize.

s/exists for/has been exploited by/g

The worst part is the optimizations aren't even that significant. (I recall a blog post of somebody testing this but I can't find it rn)

It is undefined behaviour if I write GCC --hlep

Well no, it's a compilation error, you need at the very least a semicolon after hlep and from there on it depends on what GCC is. If it's a function you need parentheses around --hlep, if it's a type you need to remove the --, if it's a variable you need to put a semicolon after it,...

Because GCC is all-caps I'm guessing it's a macro, so here's an example of how you could write it (though it won't be UB): https://godbolt.org/z/dYMddrTjj

I'm not sure if you're supporting my pov by showing the absurdity of the other position???

Yeah sure, if my phone auto incorrects gcc to GCC then that is technically meaningless so you're completely free to interpret my comment how you want.

..... Although..... GCC stands for GNU Compiler Collection so it can be reasonably capitalised, so maybe then, rather than saying anything goes we should do something reasonable because then you aren't left saying something really stupid if you're wrong???

Parent point is when the standard talks about UB it refers about translating C code. So parent cheekly interpreted your comment about command line flags (which are outside the remit of the standard) as code instead. I thought it was fitting.

The example here doesn't have compile-time known undefined behavior though; as-is, the program is well-formed assuming you give it safe arguments (which is a valid assumption in plenty of scenarios), and the check in question is even kept to an extent. Actual compile-time UB is usually reported. (also, even if the compiler didn't utilize UB and kept wrapping integer semantics, the code would still be partly broken were it instead, say, "x * 0x1f0 / 0xffff", as the multiplication could overflow to 0)

The problem with making the compiler give warnings on dead code elimination (which is what deleting things after UB really boils down to) is that it just happens so much, due to macros, inlining, or anything where you may check the same condition once it has already been asserted (by a previous check, or by construction). So you'd need some way to trace back whether the dead-ness comes directly from user-written UB (as opposed to compiler-introduced UB, which a compiler can do if it doesn't change the resulting behavior; or user-intended dead code, which is gonna be extremely subjective) which is a lot more complicated. And dead code elimination isn't even the only way UB is used by a compiler.

> also, even if the compiler didn't utilize UB and kept wrapping integer semantics, the code would still be partly broken were it instead, say, "x * 0x1f0 / 0xffff", as the multiplication could overflow to 0

That's the most important point! You simply cannot detect overflow when multiplying integers in C after the fact. This is not GCC's fault.

I agree that some of the optimizations exploiting UB are too aggressive, but the article presents a really bad example.

> If a situation has been statically determined to invoke UB that should be a compile time error.

But you typically can’t prove that. There’s lots of code where you could prove it might happen at runtime for some inputs, but proving that such inputs occur would, at least, require whole-program analysis. The moment a program reads outside data at runtime, chances are it becomes impossible.

If you want to ban all code that might invoke it it boils down to requiring programmers to think about adding checks around every addition, multiplication, subtraction, etc. in their code, and add them to most of them. Programmers then would want the compiler to include such checks for them, and C would no longer be C.

If, as you seem to say, you want to ban a subset that’s easily provable, I think enabling all warnings already does that. See for example https://clang.llvm.org/docs/DiagnosticsReference.html#wargum..., https://clang.llvm.org/docs/DiagnosticsReference.html#warray..., https://clang.llvm.org/docs/DiagnosticsReference.html#winteg... , https://clang.llvm.org/docs/DiagnosticsReference.html#wcompa...

C will accept every valid program, at the cost of also accepting some invalid programs. Rust will reject every invalid program, at the cost of also rejecting some valid ones.

("unsafe" (aka "trust me" mode) means that's not quite true, and so do some of the warnings and errors that you can enable on a C compiler, but it's close enough)

> But you typically can’t prove that. There’s lots of code where you could prove it might happen at runtime for some inputs, but proving that such inputs occur would, at least, require whole-program analysis. The moment a program reads outside data at runtime, chances are it becomes impossible.

No, I specifically ruled out doing that in my comment.

I was referring to the situation where a null check was deleted because the compiler found UB through static analysis.

(Or specifically, placing a null check after a possibly-null usage. It is wrong to assume that after possibly-null usage the possibly-null variable is definitely-null.)

As I recall, the compiler didn't know it had found undefined behaviour. An optimisation pass saw "this pointer is deferenced", and from that inferred that if execution continued, the pointer can't be null.

If the pointer can't be null, then code that only executes when it is null is dead code that can be pruned.

Voila, null check removed. And most relevantly, it didn't at any point know "this is undefined behaviour". At worst it assumed that dereferencing a null would mean it wouldn't keep executing.

It removed redundant check then, why not warn about that? gcc -Wpedantic even warns about empty statements fcol.

The compiler didn't find UB. What it saw was a pointer dereference, followed by some code later on that checked if the pointer was null.

Various optimisation phases in compilers try to establish the possible values (or ranges) of variables, and later phases can then use this to improve calculations and comparisons. It's very generic, and useful in many circumstances. For example, if the compiler can see that an integer variable 'i' can only take the values 0-5, it could optimise away a later check of 'i<10'.

In this specific case, the compiler reasoned that the pointer variable could not be zero, and so checks for it being zero were pointless.

Yes - and the original post here is the same:

    if (x < 0)
        return 0;

The compiler now knows x's possible range is non-negative.

    int32_t i = x * 0x1ff / 0xffff;

A non-negative multiplied and divided by positive numbers means that i's possible range is also non-negative (this is where the undefinedness of integer overflow comes in - x * 0x1ff can't have a negative result without overflow occurring).

    if (i >= 0 && i < sizeof(tab)) {

The first conditional is trivially true now, because of our established bounds on i, so it can just be replaced with "true". This is what causes the code to behave contrary to the OP's expectations: with his execution environment in the overflow case we can end up with a negative value in i.

It is probably more precise to say “if the pointer is null, then it doesn’t matter what I do here, so I am permitted to eliminate this” than to say that it can’t be null here. (It can’t be both null and defined behavior.)

I'm not sure that's right. The compiler isn't tracking undefined behaviour, it is tracking possible values. It just happens that one specific input into determining these values is the fact "a valid program can't dereference a null pointer", so if the source code ever dereferences a pointer, the compiler is free to reason that the pointer cannot therefore be null.

In essence, the compiler is allowed to assume that your code is valid and will only do valid things.

Consider function inlining, or use of a macro to for some generic code. For safety, we include a null check in the inlined code. But then we call it from a site where the variable is known to not be null.

The compiler hasn't found UB through static analysis, it has found a redundant null check.

> I was referring to the situation where a null check was deleted because the compiler found UB through static analysis.

You can say that but in practice -Onone is fairly close to what you're asking for already. Most people are 100% unwilling to live with that performance tradeoff. We know that because almost no one builds production software without optimizations enabled.

The compiler is not intelligent. It just tries to make deductions that let it optimize programs to run faster. 99.999% of the time when it removes a "useless" null check (aka branch that has to be predicted and eat up branch prediction buffer space and bloats up the number of instructions) it really is useless. The compiler can't tell the difference between the useless ones and security critical ones because all of them look the same and are illegal by the rules of the language.

Even if you mandate that null checks can't be removed that doesn't fix all the other situations where inserting the relevant safety checks have huge perf costs or where making something safe reduces to the halting problem.

FWIW I agree that the committee should undertake an effort to convert UB to implementation-defined where possible... for example just mandate twos complement integer representations and make signed integer overflow ID.

To illustrate the complexity: most loops end up using an int which is 32-bit on most 64-bit platforms so if you require signed integer wrapping that slows down all loops because the compiler must insert artificial checks to make the 64-bit register perform 32-bit wrapping and we can't change the size of int at this point.

To accomodate trapping implementations you'd have to make it "implementation-defined or an implementation-defined signal is raised" which it happens is exactly the wording for when an out-of-range value is assigned to a signed type. In practice it means you have to avoid it in your code anyway because "an implementation-defined signal is raised" means "your program may abort and you can't stop it".

But again, the compiler did not find UB through static analysis. The compiler inferred that the pointer could not be null and removed a redundant check.

For example you would you not expect a compiler to remove a redundant bound check if it can infer that an index can't be out of range?

The compiler made a dangerous assumption that the standard permits ("the author surely has guaranteed, through means I can't analyze, that this pointer will never be null").

Then it encountered evidence explicitly contradicting that assumption (a meaningless null check), and it handled it not by changing its assumption, but by quietly removing the evidence.

> For example you would you not expect a compiler to remove a redundant bound check if it can infer that an index can't be out of range?

If it can infer it from actually good evidence, sure. But using "a pointer was dereferenced" as evidence "this pointer is safe to dereference" is comically bad evidence that only the C standard could come up with.

> using "a pointer was dereferenced" as evidence "this pointer is safe to dereference" is comically bad evidence

Do you think the compiler would be right to remove the second check here?

   if (!x) std::abort();
   if (!x) return;
   ... = *x;

What about changing std::abort with the following?

   [[noreturn]] void my_abort();

How's that different form a check after dereferencing a pointer? In both cases the check can be removed because dataflow or control flow analysis.

What if my_abort returns instead? Or another thread changes x after the fact?

You typically can't prove it, but if and when you can prove it, you should definitively warn about it or even refuse to compile.

Things like that meaningless null check mentioned, can definitively be found statically (the meaningless arithmetic sanity check in OP's example, I'm not so sure, at least not with C's types).

So, how much effort should the standard require a compiler to make for “if and when you can prove it”? You can’t, for example, reasonably require a compiler to know whether Fermat’s theorem is true if that’s needed to prove it.

There are languages that specify what a compiler has to do (e.g. Java w.r.t. “definite assignment” (https://docs.oracle.com/javase/specs/jls/se9/html/jls-16.htm...)), and thus require compilers to reject some programs that otherwise would be valid and run without any issues, but C chose to not do that, so compilers are free to not do anything there.

Everyone wants to drag nontermination into this, but in the OP's example, the compiler already had proof that a the conditional would never evaluate to true. What you can or can't prove in the bigger picture isn't so interesting when we already have the proof we need right now.

It's just that it used this proof to remove the conditional evaluation (and the branch) instead of warning the user that he was making a nonsensical if statement.

So to the question of "when can we hope to do it" the answer is, "not in all cases, sure, but certainly in this case".

> The problem is the program isn't invalidated, it's compiled and run.

Anything can happen with undefined behaviour, including exactly what you would expect to happen for five years, and then everything breaks.

Compiling and running as if nothing is amiss is exactly how UB is allowed to look like.

> Compiling and running as if nothing is amiss is exactly how UB is allowed to look like.

Yes, and this is a "billion-dollar mistake" that's responsible for an ongoing flow of CVEs.

(the proposal to replace "undefined" with "implementation-defined" may be the only way of fixing this, and that gets slightly easier to do as the number of actively maintained C implementations shrinks)

Create a Defined-C dialect.

You can already do that to some extent. There's tons of compiler flags that make C more defined. Eg both clang and gcc support `-fno-strict-overflow` to define signed integer overflow as wraparound according to two's complement.

> and it's the C standard.

No, it's puerile interpretations of the C standard learned from the comp.lang.c newsgroup and such places rather than from engineering work.

>Linux kernel is now compiled with -fno-delete-null-pointer-checks

Like many other large C codebases, it also uses -fno-strict-aliasing and -fno-strict-overflow (which is a synonym for "-fwrapv -fwrapv-pointer").

-fwrapv introduces runtime bugs on purpose! The last thing you want is an unexpected situation where n is an integer and n+1 is somehow less than n. And of course that bug has good chances of leading to UB elsewhere, such as a bad subscript. If you want to protect from UB on int overflow, -ftrapv (not -fwrapv) is the only sane approach. Then at least you'll throw an exception, similar to range checking subscripts.

It is sad that we don't get hardware assistance for that trap on any widespread cpu, at least that I know of.

I can easily test if n+1 is < n with fwrapv.

Without you have to do convoluted things like rearranging the expression to unnatural forms (move the addition to the right but invert to subtraction, etc), special case INT_MAX/INT_MIN, and so on - which you then have to hope the compiler is smart enough to optimize, which it often isn't (oh how ironic).

It's not to protect from UB, it's to protect from the optimiser deleting your bounds checks.

On x86 you can put an INTO instruction after each arithmetic operation to trap if the overflow flag is set.

We've got a few components written in C that I'm (partially) responsible for. It's mostly maintenance, but for reasons like this I run that code with -O0 in production, and add all those kinds of flags.

I'd be curious to know how much production code today that's written in C is that performance critical, i.e. depends on all those bonkers exploits of UB for optimizations. The Linux kernel seems to do fine without this.

I'm fairly confident in declaring the answer to your question: None.

Most programs rarely issue all the instructions that a CPU can handle simultaneously, they are stuck waiting on memory or linear dependencies. An extra compile-out-able conditional typically doesn't touch memory and is off the linear dependency path, which makes it virtually free.

So the actual real-world overhead ends up at less than 1%, but in most cases something that is indistinguishable from 0.

If you care that much about 1% you are probably already writing the most performance critical parts in Assembly anyway.

> If you care that much about 1% you are probably already writing the most performance critical parts in Assembly anyway.

I call this hotspot fallacy and it is a common one. This assumes there is relatively small performance critical parts that can be rewritten in assembly. Yes, sometimes there is a hotspot, but by no means always. A lot of people caring about 1% is running gigabytes binary on datacenter scale computer without hotspots.

Thanks for that lwn article.

I had read it a long time ago, and had since forgotten the source. I've spent a few hours trying to find it in bug-trackers. really glad to have the link now, thanks!

The C standard doesn't really matter. Standards don't compile or run code. Only thing that matters is what the compilers do. "Linux kernel C" is a vastly superior language simply because it attempts to force the compiler to define what used to be undefined.

This -fno-delete-null-pointer-checks flag is just yet another fix for insane compiler behavior and it's not the first time I've seen them do it. I've read about the Linux kernel's troube with strict aliasing and honestly I don't blame them for turning it off so they could do their type punning in peace. Wouldn't be surprised if they also had lots more flags like -fwrapv and whatnot.

I don't believe that it does. If the invalid arithmetic proceeds without crashing, and produces a value in the int32_t i variable, then that issue is settled. The subsequent statement should behave according to accessing that value.

"Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message)."

Ignoring the situation completely means exactly that: completely. The situation is not being ignored completely if the compilation of something which follows is predicated upon the earlier situation being free of undefined behavior.

OK, so since the situation is not being ignored completely, and translation or execution is not terminated with a diagnostic message, it must be that this is an example of "behaving in a documented manner characteristic of the implementation". Well, what is the characteristic; where is it documented? That part of the UB definition refers to documented extensions; this doesn't look like one.

What is "characteristic of the implementation" is in fact that when you multiply two signed integers together with overflow, that you get a particular result. A predictable result characteristic of how that machine performs the multiplication. If the intent is to provide a documented, characteristics behavior, that would be the thing to document: you get the machine multiplication, like in assembly language.

> I don't believe that it does. If the invalid arithmetic proceeds without crashing, and produces a value in the int32_t i variable, then that issue is settled. The subsequent statement should behave according to accessing that value.

You may dislike it, but that is not how UB in C and C++ works. See [1] for a guide to UB in C/C++ that may already have been posted elsewhere here.

It is a common misconception that UB on a particular operation means "undefined result", but that is not the case. UB means there are no constraints whatsoever on the behavior of the program after UB, often referred to as "may delete all your files". See [2] for a real-world demo doing that.

[1] https://blog.regehr.org/archives/213

[2] https://kristerw.blogspot.com/2017/09/why-undefined-behavior...

I should clarify that I believed all that in 1990-something; I've arrived at a more mature professional opinion in the nearly three decades since.

> If the invalid arithmetic proceeds without crashing, and produces a value in the int32_t i variable, then that issue is settled. The subsequent statement should behave according to accessing that value.

The C standard imposes no such constraint on undefined behaviour, neither is it the case that real compilers always behave as if it did.

hxhxhrra has already shown this, but here's another good blog post on this kind of thing: https://markshroyer.com/2012/06/c-both-true-and-false/

Even if this solution cannot be used for the Linux kernel, for user programs written in C the undefined behavior should always be converted into defined behavior by using compilation options like "-fsanitize=undefined -fsanitize-undefined-trap-on-error".

>the C standard does allow this

The C standard also allows doing it even when there is no UB. C gives implementations a ton of freedom.

Somewhat unfortunately this is valid behavior according to the standard. Having to go through walls of text in a standard to prevent the compiler from deleting your security checks because of how you multiplied two integers seems a bit silly.

Having said that I’m of split feelings here. I work in a very performance sensitive industry and on one hand welcomes the ability of compilers to use the knowledge that certain things “can’t happen” to optimize, without me having to always do these optimizations be hand.

On the other hand, there seem to be so many cases like this one where the “undefined behavior code deleter goes brrrrrr” really overextends its usefulness. The “lalalalala standard say I can do this can’t hear you” finger in your ears attitude from compiler maintainers doesn’t help at all either.

I understand that the way much of this works, propagating “poison/impossible values”, can hide the root cause, so you can’t just say “please do the good undefined behavior optimizations but not the bad, so there’s no easy answer. The outcome in the blog post doesn’t feel like a local optimum though, and it’s not the only place I’ve felt that your options are “potentially slow code” or “pray you were perfect enough to not have your program deleted”

The problem here is that the thing that "can't happen" isn't actually something that can't happen, it's something that isn't allowed to happen according to a many-hundred-page document that approximately nobody reads. It's not something that can be optimised because the compiler can prove it cannot happen, it is allowed to be optimised because the standard says "dear programmer, if you ever make this happen, god help you".

I think this view is slightly unfair. I think of UB as the compiler saying "when you promised this thing wouldn't happen, I took you at your word. If bad things happen because you lied, they're your fault, not mine."

Lying requires intent. This was a mistake, something that humans are well-known for making, and if the compiler is designed to assume otherwise, it borders on useless in the real world.

A compiler can’t know why you fucked up, it can’t even know that you fucked up, because UBs are just ways for it to infer and propagate constraints.

If an optimising C compiler can’t rely on UBs not happening, its potential is severely cut down due to the dearth of useful information provided by C’s type system.

> A compiler can’t know why you fucked up, it can’t even know that you fucked up, because UBs are just ways for it to infer and propagate constraints.

To be honest, that's just how compiler writers interpret UB these days.

It's perfectly possible (in principle) to use lots of more sophisticated static and dynamic analysis to recover much of what C compiler just assume. You don't have to restrict yourself to what C's type system provides.

(For an example of what's possible, have a look at all the great techniques employed to make JavaScript as fast as possible. They have basically no static types to work with at all.)

> For an example of what's possible, have a look at all the great techniques employed to make JavaScript as fast as possible. They have basically no static types to work with at all.

I’m sure people will be very happy with a C JIT. That’s definitely what they use C for.

JIT-ed code is full of runtime type and range assertions which bail if the compiler’s assumptions are incorrect.

C has always considered that the programmer knows what they are doing. Programs are assumed correct unless proven invalid.

This is -- or at least was -- a feature, not a bug. You can implement any valid program, but you can also implement some invalid programs.

I know the OP mentioned Rust, but it's a valid comparison: if you don't invoke "unsafe" then all your behaviour is well-defined. But the trade-off is that Rust will only let you implement a subset of valid programs unless you invoke "unsafe", which might be better termed "assumed correct".

That's C for you. If you want something saner, use Rust or Haskell or Python or even Java or Go or.. almost any other language that's not C or C++.

These days the whole point of C is this Faustian pact with the devil of speed for sanity.

Writing Rust and Haskell for sanity is not something I would agree with. Maybe for language characteristics but reading those make me jump out of the window.

I have the same issue with Java and Go. (Which I brought up as well.)

Yet, they still compare favourably with C in this regard. Almost anything does.

I think Rust doesn't allow integer overflow either, unless you specifically use the wrapping_* operations. Probably the same kind of thing will also happen to Rust.

Rust either allows or traps on integer overflow depending on the mode. Either way it’s defined behaviour.

And yet C is still the dominant language. Undefined behavior is actually the reason why: any defined behavior is expensive to implement in the compiler and possibly incurs a cost at runtime. The language design intentionally trades programmer’s sanity for ease of implementation.

Not sure that's actually the reason in practice?

For a real prominent counterexample: the Linux kernel is intentionally programmed in a C dialect (defined by a myriad of GCC compiler flags) that removes a lot of UB.

If they craved the Faustian bargain of UB for speed, they could immediately move in that direction by dropping some GCC options.

Legacy reasons, most embedded devs and UNIX clones won't use anything else.

In many other domains, other languages have taken their place, and this will keep on going, even if it takes a couple of generations, or goverment cybersecurity mandatates to make it happen.

> The language design intentionally trades programmer’s sanity for ease of implementation.

There's nothing "easy" at all about UB-exploiting performance optimizations in modern C compilers, and "ease of implementation" is absolutely not why those optimization passes have been included. In fact, the easiest thing for the compiler to do, when it sees an int * int operation, is to emit an IMUL assembly instruction (or the equivalent for your CPU architecture) and not worry about deleting overflow checking code. Which is what C compilers did before the extent of UB exploitation became excessive.

I agree on the top compilers but there are dozens if not hundreds architectures with their own proprietary C compilers maintained by a dinosaur and a couple intern dino chicks if they’re lucky. I postulate any other language wouldn’t be implemented or would be defanged to C-level of (non)safety anyway in a way similar to mrustc.

I don't think this is a fair comparison as this is all based on implicit inferences by the compiler.

If the programmer had specifically invoked the "__assert_valid_pointer(p)" standard function (which does not exists) to promise the compile that the pointer was valid then it would be fine.

The problem is that there are a lot of places where the compiler makes these assumptions.

This is a good positive model of UB, fulfilling the compiler assumptions.

Does the compiler know? If so, can't they have a flag that doesn't allow UB?

Doesn't allow optimization enabled by this specific UB: Yes Does't allow UB: Hard, because you probably need runtime checks.

"and I'm going to kill you for it even though I could also just slap you"

Even if someone would read all those pages, constraining ourselves to ISO C only, no way that after an year they would still remeber the about 200 UB cases that are documented there.

Which is why everyone should adopt static analysis tooling and enable all the warnings that are related to UB, pointer and casts misuses.

Many think they know better, it is like those that think builders don't need protection gear at a construction site, it is stuff only for the weak.

I think implicitly compiler-added runtime check are a more robust and reliable solution than static analysis. For example for pointer dereferences the compiler should could 0-offset dummy load if the load is not guaranteed to be within a page of the pointer. Or adding abort-on-overflow for math. Or bound checking where possible.

It will have a non-trivial cost, but hopefully aggressive optimizations can remove many of these checks (which ironically it is exactly the kind of optimizations people are complaining about) and compilers provide pragmas to disable them when critical.

In a way sanitizers are getting there, but they are explicitly marked as for non-production use which is a problem.

I agree, but unfortunely that will never happen in most C and C++ circles, just see the heat JF Bastien has been facing for a feature that has been shipping in Windows and Android for the last two years, proven in the battlefield to hardly hinder performance in real use cases.

https://isocpp.org/files/papers/P2723R0.html

Lots of people telling him it will never fly in production, while their Windows and Android phones are using the code that they say isn't good enough.

Zero initialization is also one of those features that seems such a low hanging fruit to implement...

I'm still moderately optimistic. I suspect that many of these checks will end up being enabled by default on compilers shipped by distros, like stack guards and other forms of hardening.

Except this can't happen happens many and many times in practice so maybe it's time the language bureaucrats got off their high horse (but they won't)

It's still braindead and idiotic. Every relevant platform nowadays has well defined overflow for signed ints. A sane C compiler should go with that and base its optimizations on it. GCC has been a pile of garbage in this regard for many years now. Its devs get further removed from reality with every year. Treating signed int overflow as undefined should be hidden behind a flag.

The C/C++ language doesn't provide for a way for the compiler to see that you really meant this one check to take precedence over the implicit promise in another.

The reason why C++ is always relevant here (though C macros and inlining cause similar issues) is that generic programming being close to optimal is a language feature - and one of the ways that's possible is by letting you right reusable code that might be "called" from a context in which some of the checks or conditions just aren't necessary. It's by design that the optimizer gets to... well, optimize that kind of code.

There's a solid case to be made that the details of C's UB weren't well chosen and we should try to update them; but which decades old choices are perfect? Which are easy to change once there's this much legacy software in operation?

Don't forget that some of those UB's were chosen to deal with hardware realities of the day; i.e. that the "same" operation on different hardware would do different things. For example, eliminating signed integer overflow might allow a C compiler to use a signed register that's wider than necessary, which may help on hardware that doesn't have every possible register width, or where there are complex register usage limitations. I'm no hardware geek; I'm sure somebody here knows or real examples where UB allows portability, because that's the point: UB allows people to write portable, performant code - just don't do certain things, and you're fine... which leads us to today's situation, in which UB can feel like a minefield.

> Don't forget that some of those UB's were chosen to deal with hardware realities of the day; i.e. that the "same" operation on different hardware would do different things.

That's an argument for implementation defined behaviour. Not for undefined behaviour, at least not UB in the modern sense.

Having implementation defined behavior would imply non-portability. C compilers have all kinds of ways of exposing platform-specific features, but sneaking those into what looks like standard behavior has its own issues. And even if you accept that, that doesn't deal with the issue of inlining, generics, and macros - you can get different implementation defined behavior even in a single hardware implementation like that.

If that is what you want, compilers have various flags that let you in essence do that. But the next problem with that is (1) that it's possible existing code may be suddenly and unpredictably lose performance, and (2) now you need to provide some other well-defined behavior for those UB cases, and (3) the selling point of generics/macros/inlining may be reduced.

How many relevant UB's are there? I don't know. How much perf would code common lose? I don't know. To be sure, I fully acknowledge that removing UB from the spec may be the right thing to do, but it's also easy enough to find possible problems with that strategy; I'm just pointing out the complexities, which is a lot easier than solving them or knowing which are irrelevant.

> Having implementation defined behavior would imply non-portability.

Not any more than UB does. Ie if you avoid the implementation defined behaviour, your code is perfectly portable.

(And otherwise your code could probe for what the implementation defined behaviour is, and adjust accordingly. No such luck with UB.)

However I do agree with you that it's hard to remove UB from the spec. The spec's failing is in having so much UB in the first place.

The problem is not UB per se -- the problem is that the compiler uses UB to make assumptions that are incorrect.

Removing a comparison because of UB is fucking stupid. The compiler on the one hand assumes that the programmer is diligent enough to consider of every invocation of UB, but on the other hand too stupid to see the check they wrote will always be true.

It's not a good idea.

Checks that are always true _in some context_ are entirely normal and by design if the code can be used in a different context. If your code is reused in a way that let's the optimizer re-optimize the code per-context, then you'll benefit from the compiler's ability to remove dead code or even merely to choose less expensive special case ops. Macros, templates and inlining are some common ways that happens, but platform-specific builds and perhaps others exist too.

For example, imagine you have some SIM wide value, and you want to do something to each word or byte that the SIMD value contains. In today's C, you can just write a bunch of ifs: is width < 2? then... is width < 4? then... etc. The compiler with completely elide those ifs and leave behind only the reachable code - if it can specialize that re-used code for the given context.

Furthermore, today those checks might be implicit via the use of UB. That's perhaps not a great solution looking at the entire ecosystem, but it is the situation we're in. Changing that might be quite a lot of work.

I don't mind that the compiler simplifies expressions and removes code that it can prove is dead.

I just don't like that it derives preconditions from UB.

I like that the compiler removes comparisons that always have the same result.

It means I can write clear code, guard things rather than explain in a comment why the guard isn't needed, and know that the compiler will remove the inefficient code. In general, optimizing compilers mean that taking the clearer option is much less of a performance loss. I like that.

In many of these UB cases, the annoying things is that the compiler removes the safety feature you explicitly added, but there are plenty of alternatives.

The compiler doesn't make assumptions that are incorrect (it is what it is, basically, if it assumes no integer overflows there better not be).

Your application, and the application programmer, instead fail to fulfill those compiler assumptions.

Signed int overflow being UB is one of the most basic UBs of the language, and what allows generating tight code in loops.

This is not new, -fwrapv was introduced in 2003, but it can quite severely impact code quality, if you don’t care, just set that. Then complain that C is slow, because C is a shit language.

> and what allows generating tight code in loops.

How so? How does breaking an if statement the programmer added make the code faster? If they intended the check not to happen/be required, they wouldn't have written it. Let signed int overflow and leave any code that depends on its value alone. So yes maybe make fwrapv the default.

> because C is a shit language.

Well, it's as low level as it can get before reaching assembly, but why not try reducing the number of foot guns? Sometimes you still need C, and that's not going to go away for the foreseeable future.

> How so?

A somewhat common example I've seen is sign extension in loops, where the width of the loop variable is not the same as that of the CPU register [0]. If the compiler can assume that signed integer overflow is UB, then it has a lot more freedom to unroll/vectorize the loop [1] (remove -fwrapv and watch Clang go to town).

Of course, that specific optimization is rendered somewhat moot if the programmer chooses to use a 64-bit loop variable, but that is a slightly different rabbit hole.

> If they intended the check not to happen/be required, they wouldn't have written it.

I feel that's somewhat iffy reasoning - if we trust the programmer so much, why allow the implementation to optimize in the first place? And if not to that extreme, where should the line be drawn?

[0]: https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759...

[1]: https://godbolt.org/z/EMaq1j3Kc

I feel that's somewhat iffy reasoning - if we trust the programmer so much, why allow the implementation to optimize in the first place?

Right. If you want each line of code to be loaded and executed precisely in sequence, exactly as written, then you know where to find Python.

That's one contrived example solved by changing the type of the loop variable. You have to profile your code for hot spots anyway, this loop would be immediately obvious. And if it's not a hot spot, the difference in emitted code is completely pointless.

> How does breaking an if statement the programmer added make the code faster? If they intended the check not to happen/be required, they wouldn't have written it.

See your problem is that you’re

1. not thinking like a compiler

2. and reasoning on an isolated example

The compiler does not “break an if statement”, the compiler uses the UB to limit the range of the input and output, it can then propagate this range analysis to see that the check is dead code, and so removes the dead code.

It’s common for users to write unnecessary or redundant checks, even more so because of inlining, and especially macros.

If you’re carefully checking for null in every function prologue, and the compiler in-line everything and knows the pointer is non-null, all checks are dead and can be removed. Which is what the compiler does. This reduces the amount of branches (and thus the space needed by the branch predictor), and reduces the amount of code meaning the new inlined function could fall below threshold and itself become a candidate for inlining.

Ok sure, I'm using this particular example here, but I've yet to see a good counter example to convince me it's the lesser evil to let that happen.

Also I agree you should not write code like that example and rather move the check up.

But reality is we (at least I) still depend on code written in C, like openssh, and want it to be as safe as possible. Now I can blindly trust the devs know every UB in the C spec in and out, run all the static and dynamic analysis tools in existence, but it would just make me feel even more safe if the compiler would also work with them, not against. Somewhere here in the comments it was claimed that the linux kernel for example already uses -fwrapv and its performance seems absolutely fine to me. And I'd suspect that an OS kernel is already on the more performance critical end of the spectrum regarding stuff written in C that's still in use.

I just find it worrysome that such evidently unsafe optimizations are the default, and not hidden behind some sufficiently scary-sounding flag.

It's not about what your CPU does.

These days undefined overflow for signed integers is mostly used by compilers to be able to assume that eg 'a + 1 > a' is always true, and thus eliminate redundant checks.

(And you wouldn't typically write code like 'a + 1 > a', but you can get either from code generation via macros etc or as a intermediate result from previous optimization passes.)

Basically, the compiler implements integer addition using an operation that doesn't match the semantics of integer addition in the standard, then hallucinates that it did. That is:

1) The compiler sees an expression like "a += b;" where a and b are signed integers.

2) It emits "add rA rB" in x86 assembly (rA/B being the register a/b is currently in).

3) Technically the machine code emitted does not match the semantics of the source code, since it uses wraparound addition, whereas the C standard says that for the operation to be valid, the values of a and b must be such that no overflow would occur. This is fine however, because the implementation has the freedom to do anything on integer overflow, including just punting the problem to hardware as it did in this case.

4) The compiler proceeds with the rest of the code as if the line above would never overflow. My brother in the machine spirit, you chose to translate my program to a form where integer overflow is defined.

The compiler should either a) trap on integer overflow; or b) accept integer overflow. It will be fine if it chooses either a) or b) situationally, i.e. if we have a loop where assuming no overflow is faster, then by all means - add a precondition check and crash the program if it's false, but don't just assume overflow doesn't happen when you explicitly emit code with well-defined overflow semantics.

The bigger problem is there is pretty much no way to guard against this. The moment your program is longer than one page you're screwed. You may think all your functions are fine, but then you call something from some library, the compiler does some inlining and suddenly there's an integer overflow where you didn't expect, leading to your bounds check being deleted.

We need a -fsane-c

Then they can add a #pragma optimize(assumes=no-int-overflow, whatever, etc) to precisely add optimizations when needed and you 'know' its safe.

Everyone wants that, but when asked for a concrete specification they seem to realize that it is harder than it sounds. Look for John Regehr's blog entries about "Friendly C" for an example. The basic problem here is that C is a terrible language. We should just give up on it by now.

I provided the concrete specification. What is hard about it? Get it done already.

This already exists. Don't write standard C, avoid it like the plague. Compile with -fno-strict-overflow -fno-strict-aliasing -fno-delete-null-pointer-checks, like I do, like Linux kernel does, and like everyone sane does.

No, not everyone sane. Rather everyone sane who has been bitten enough by these issues to use such rules. Everyone starts out at -O2, because understanding all the other flags and their implications is super difficult. As long as the insane setting is default, a large percentage of programmers will be using the insane setting. Arguing that they should have flagged their compilations otherwise is about as useful as pointing out that people shouldn't write UB in the first place.

Sure, I 100% support changing the C standard. Until it is done, compiling with flags is infinitely easier than trying to avoid these UBs.

You can get 99% of the way there with -fno-delete-null-pointer-checks -fno-strict-aliasing -fwrapv . Pretty much every program I've worked on uses those flags, as that's the only way to keep your sanity.

-fdwim

Next generation of AI powered compilers will try to interpret code at a more abstract level and infer what the programmer was thinking even if they wrote the wrong thing.

Everything will work perfectly 100% of the time.

The hard things about C are knowing all these footguns you are getting yourself into. If our electrical grid was built like this we had no isolation, no fuses, no RCDs and a constant torrent of electrocutions and fires. It is bad engineering.

... and people defending the status quo because "you just have to know how electricity works" :-)

> On the other hand, there seem to be so many cases like this one where the “undefined behavior code deleter goes brrrrrr” really overextends its usefulness.

I would simply not depend on invoking UB as part of my program's behavior (?).

That's the only way to use C correctly.

Alas, it's nearly impossible for a mere mortal to write any non-trivial C code that doesn't have UB.

"a+b" for signed integer types potentially invokes undefined behavior (overflow). It's a horrendous situation.

Alternatively I would not trust myself with not doing that, which is why I don’t use C.

I don't think I ever claimed that one would or should intentionally invoke UB as part of their program's behavior?

GCC's UBSan catches it,

> runtime error: signed integer overflow: 50000000 * 511 cannot be represented in type 'int'

https://godbolt.org/z/oz9vvj5YM

Another heads up of the dangers of UB optimizations, and why using static analysis is a requirement for C derived languages.

If you aren't using all warnings turned on as errors, disabled implicit conversions and at very least have static analysis on the CI/CD pipeline, you're up for a couple of surprises.

From CppCon 2022, "Purging Undefined Behavior & Intel Assumptions in a Legacy C++ Codebase"

https://www.youtube.com/watch?v=vEtGtphI3lc

Yes absolutely, and this is possible today with only open source software. So money is not a barrier.

The sanitizers (UB, address, memory, threads) are supported by both Clang and GCC [1]. Yes that's up to 4 different builds and tests runs but with an automated C/I this is not a big deal.

The Clang static analyzer, with Z3 enabled as a checker, used through CodeChecker [2] is now very good, so much so that I prefer it to a different commercial product showing too many false alarms. Using it on an embedded GCC cross-compiled code base may still require some workarounds, but nothing too bad and this is improving regularly too.

I wouldn't want to do without this. Switching to Rust may not always be possible, and there are big C and C++ code base that will live a long while. Tools like this help and they should be used.

[1] https://github.com/google/sanitizers/

[2] https://codechecker.readthedocs.io/en/latest/

Definitly, Java, V8, .NET, Android runtimes still have lots of C++ into them, LLVM and GCC depend on C++ and are comparable to Linux kernel in complexity, GPGPU toolchains, .....

So reboting into any safe alternative, is going to take decades, hence why the first step is still trying to advocate for best practices, even if it feels like a Quixotic endevour.

Note that UBSan is a dynamic analysis tool.

Thanks for the correction, one should use those as well on debug builds for integration tests.

Agreed :)

Indeed. But still a good idea to run at least your test-suite with it. And also with address sanitizer and clang's memory sanitizer, etc. Whatever you can find.

The static analysis is you compile with it and see if there's any trap instructions emitted by the compiler.

(The answer is: yes.)

I keep asking this: in my experience sanitizers and other dynamic checkers have always overperformed, while I'm underwhelmed by static analysis. Do people have different experiences?

The biggest issue is that too few use them, most surveys place the number around 10% - 20% in such tooling adoption.

Or you know, just turn off UB. I don't know why C still has this, it was useful when we had truly exotic architectures with sign bits &c, but these days it is doing way more harm than good.

> Or you know, just turn off UB.

You cannot “turn off UB”. The behaviour is undefined in the standard, and nothing the compiler can do will make it defined. There is a profound misunderstanding of what undefined behaviour is in a lot of the comments. It is not a compiler setting. The way to make it defined is to change the standard.

Right, UB is essential part of C and can't be turned off. But it's entirely possible to turn off integer overflow UB by compiling with -fno-strict-overflow, and you should use it.

> Or you know, just turn off UB

I'm sure you will be willing to write the GCC PR that implements the required, surely trivial, analysis.

It is already implemented: -fno-strict-overflow.

For this specific kind of UB yes. I thought parent meant all UB in general.

Since the author and GCC disagree about whether this behaviour is useful, it is likely that insufficient requirements analysis has taken place. Is GCC supposed to behave this way? This depends on what goals it is supposed to reach. The GCC authors would say that the C standard allows such compiler behavior, and what is allowed by the C standard doesn't need to be justified by other means. The article author would argue that usability towards the programmer leads to less bugs and is needed, at least partially, as a justification.

Going a step further, this places the article author outside GCC's main intended user group. It raises the question: Who are GCC's main intended users? And is there a way to more clearly advertise that the article author isn't part of them? This would probably help other potential GCC users to decide whether GCC is the right tool for them at all.

I don't really get the discussion about the C standard and UB in the other threads here. The standard and UB are only a tiny pixel in the big picture.

There really is no disagreement here.

GCC implements a language. The intended users are people programming in that language, which implies some sort of proficiency. The author isn't aware of the pitfalls of said language.

This is not about GCC.

That's just saying, "there is no disagreement because one side is clearly right, and the other is clearly wrong". Even if that were true, which is far from certain in this case, it doesn't preclude a disagreement.

The argument about proficiency has been bruoght up multiple times already -- but only by one of the parties involved in the discussion, which shows that there is disagreement -- and besides that, makes a visit in literally every single discussion about usability.

C is in many ways a low-level assembler (as by extension C++), in this case though iirc both Java and C# has copied the behavior of fixed-size integer overflows because it's pretty much the efficient usage of the cpu multiplication instruction(and addition,subtraction) w/o introducing extra branches.

Should all languages start warning about this or would it introduce too much clutter (much much code in real life doesn't touch upon external input or would otherwise be ok with errornous computations)? In many languages the second order-effects aren't usually dangerous (bounds checks) but for C/C++ the second order effects ARE dangerous.

Know your users. Who are the target users of GCC, and to what extent are they aware that UB can cause the then-branch of an if-statement to be executed even though the condition is actually false? Does GCC, in an "intentional no true scotsman" way, define its target users to be those that are aware of such pitfalls?

In the end, GCC's users must use it, and they are the ones to give hints (ideally, answers) towards how GCC should behave in such a situation, e.g. whether GCC should prioritize optimization or (programmer's) fault tolerance.

GCC users quite often probably uses other compilers (not that it usually stops GNU developers from embrace-extending stuff). Don't know what the primary PS4/PS5 devkit compiler is but at least gamedevs (not an insignificant portion of developers still mainly using C++) often has MSVC(Win/Xbox) or CLang(osX/iOS/Sony?) as their daily driver due to platform choices.

Rust solves it with an explicit ,,wrapping_mul(..)'' call when it's performance critical.

I believe all languages that don't support bignums should do the same.

Oh it is. The fact that you have to remember a (very lengthy) document and every single mention of undefined behavior in it just to be sure that the code that goes out of the compiler will somewhat resemble your mental model of it is, in my opinion, not a reasonable requirement.

It really shouldn't be that difficult to wrap all these assumptions into a ‘if (can_exploit_ub)’. Then you can just pass something like -fno-exploit-ub and everybody's happy.

> It raises the question: Who are GCC's main intended users?

Thank you, for not using the word "begs".

This has nothing to do with undefined behavior. Switch the code to unsigned integers, where overflow is perfectly defined as wrapping, and the result is exactly the same.

The compiler, to avoid the division, compares x * 0x1ff with 512 * 0xffff instead of x * 0x1ff / 0xffff with 512. This comparison is obviously bunk, since it doesn't take the uppermost bits of the multiplication into account. But so is the original comparison!

You see the same thing happen in Rust -- https://rust.godbolt.org/z/xf67rM77T -- the difference being that there's a second language-level bounds-check inserted at the lookup site.

The author took issue with the compiler removing the i >= 0 part. The compiler did so because it could infer it's true given x >= 0, the only way i < 0 could be true is via integer overflow.

I think the output of the Rust compiler is fine because it uses an unsigned comparison ("ja" as opposed to "jl") to implement the "if".

In the example, the crash occurred because the compiler removed the i<512 part because without overflow that condition is always true.

Despite the appearance (the printf says i=62183), the compiler is not removing the check i < 512. That's clear from the decompiled code.

What actually happens is that 50000000*511 is 0x5F2E60F80, which is -219803776 when truncated to 32-bits and treated as signed. At the C abstract machine level this was UB, but at assembly level this is the value that is stored in the register and used for subsequent computations.

The compiler then says "this must be positive because I had already checked 50000000 and it was". So it only performs a signed check -219803776 < 512*511, which passes.

It's only inside the "if" that the code divides -219803776 by 65535. One could plausibly expect the result to be -3353, but the compiler decides to use an unsigned divide (again, it can do so because the input "must" be less than 2^31) and therefore it returns 61283.

But the test that's being removed is i>=0, not i<512.

I based my comment on the description in the article.

I do not understand why the compiler would divide inside the 'if'...

At some point the conclusion may be that compilers are trying to be too smart for everyone's good.

> I based my comment on the description in the article.

I see, indeed the author did not explain entirely what was going on in the optimizer. Though he's correct that overflow is what causes the UB.

> I do not understand why the compiler would divide inside the 'if'...

Why not? A division is expensive, it makes sense to do it only if the result is used. Anyway the problem is not (just) the division, because you would still have problems if the compiler removes the comparison with 0 and also decides to use a signed division -219803776/65535.

Interestingly, there is also a chance that the compiler uses a double-precision intermediate result for the multiplication, using the x86 instructions IMUL and IDIV, because 65535>511 i.e. the final result is always smaller than the input (assuming infinite precision for intermediate results). In that case the compiler "fixes" the overflow entirely for you, but it can only do so exactly because the overflow is undefined behavior.

That does sometimes happen, and with macro expansion the similar "x = a * 10 / 100 -> x = a / 10" optimization also happens, but nobody complains because it fixes bugs in their code... ;)

It doesn’t matter what x is. Without prior undefined behavior, there is no way to justify “if (i >= 0 && i < sizeof(tab))” passing when (as demonstrated by the printf) i is not actually in that range.

Edit: Though, incidentally, the comparison does not work the way it was probably intended to work. In `i < sizeof(tab)`, `i` is converted to `size_t`, so an unsigned comparison is performed, making the `i >= 0` part redundant. But the result is the same as what was intended.

That is not how undefined behavior works in C (or C++).

Effects of UB are not temporal or spacial limited to the place where undefined behavior happens.

The moment you enter a compilation unit (assuming no link optimizations) with a state which at some point will run into undefined behavior all bets are of.

EDIT: Yes, UB can "time travel". Compared to that ignoring an if condition iff the UB code was triggered is harmless. Similar it can also "split realities". E.g. a value produced by UB might at one place have the value 1 and at another place a completely different value. E.g. unsigned int overflow values might for an if condition have one value and for the print statment in the condition another and for the index operation again a different value.

EDIT2: Which is why a lot of people which have proper understanding of C++ and don't have a sunken (learn C++) cost fallacy came to the conclusion that using C++ is a bad choice for most use-case.

> The moment you enter a compilation unit (assuming no link optimizations) with a state which at some point will run into undefined behavior all bets are of. [...] Yes, UB can "time travel"

Close, but not quite. This is a common misconception in the reverse direction.

Abstractly, what UB can do is performing the inverse of the preceding instructions, effectively making the abstract machine run in reverse. However, this is only equivalent to "time-traveling" until you get to the point of the last side effect (where "side effect" here refers to predefined operations in the standard that interact with the external world, such as I/O and volatile accesses), because only everything since that point can be optimized away under the as-if rule without altering the externally visible effects of the program.

As a concrete, practical example, this means the following: if you do fflush(stdout); return INT_MAX + 1; the compiler cannot omit the fflush() call merely because the subsequent statement had undefined behavior. That is, the UB cannot time-travel to before the flush. What the program can do is to write garbage to the file afterward, or attempt to overwrite what you wrote in the file to revert it to its previous state, but the fflush() must still occur before anything wild happens. If nobody observes the in-between state, then the end result can look like time-travel, but if the system blocks on fflush() and the user terminates the program while it's blocked, there is no opportunity for UB.

The program can logically undo the call to fflush, too. Mainly by not dispatching it at all–UB is a global program attribute, at least currently. (People have made proposals to change this, but I don't think they have gone anywhere.)

No, it cannot, and UB is not a global program property. The C standard defines valid program executions according to the behaviors of the abstract machine. UB is a property of an execution of the program given some inputs.

Yes, sorry for not being precise: UB applies to executions. When I said "global" I meant global over that entire execution, so if your path ends up hitting undefined behavior it can go back and logically undo its entire execution, including parts which it shared with a well-defined execution or where you'd generally expect side effects to be placed.

Something I should add here in hindsight is that I've been rather sloppy in this discussion with a few details, and perhaps they're worth clarifying. For example, despite me using them interchangeably, "observable behavior" is not the same thing as "side effects", and you really have to refer to the standard and your implementation to see what constitutes observable behavior. For example, fflush() may in fact be elidable if the compiler can prove the file is unbuffered (and it wouldn't even need UB for that). Similarly, if the compiler can prove fflush() has no observable behavior (i.e. it is guaranteed to return without raising signals, terminating the program, etc.) then it may be able to elide the call in the UB case as well. In practice this isn't usually possible to guarantee given fflush() performs an opaque system call, but it may be more possible in a freestanding implementation than in a hosted one.

Ultimately, my point here wasn't about fflush() or even about the specifics of what exactly constitutes observable behavior in the abstract machine. (I do recall writes to volatile variables was among them, but you'd have to check all of them to be sure.) Rather, my basic point was the fact (tautology?) that any interactions with the external world that affect the program's observable behavior necessarily must be allowed to happen before the program can "know" for certain that the execution path will trigger UB—which by definition isn't possible when one of the intervening operations is an opaque call.

> if you do fflush(stdout); return INT_MAX + 1; the compiler cannot omit the fflush() call merely because the subsequent statement had undefined behavior

False! The expression (INT_MAX + 1) has no side effect (assuming no UB), so according to the rules of the C abstract machine, the compiler is allowed to hoist this calculation above the fflush(). If you run this on a machine that traps on integer overflow (which is allowed behavior), the process could crash before the fflush() is executed. Remember, everyone: With UB, anything can happen.

To hammer it home: UB isn't restricted to a variable having a funny value. Your C program is allowed to play Nethack on startup, if the compiler can prove that a few hours into your program, there would be UB.

Yeah, I was wrong. Mixed up the signed and unsigned cases.

If this kind of optimization is unacceptably risky for your use case, you need a different programming language, not a different C compiler.

"unacceptably risky" except I can't think of anything more basic than asking the computer if a > b and it fails at that?

UB invades your whole program, not specific lines.

However in this case, the culprit wasn't comparison `a > b`, but assignment `a = b`.

In general, addition like 'a + b' also isn't safe in C.

It's not the assignment. It's the multiplication x * 0x1ff.

The compiler has done range analysis and knows that at this point, x is non-negative. The programmer has dilgently ensured that values are such that the multiplication can't overflow, therefore the result of it is also non-negative. That means the later check for i being non-negative is trivially true.

If it's wrong break on compiler time, not on run time

The problem is the compiler going implementation defined on the multiplication/assignment then going all language lawyer on the following line and blaming the user

> In general, addition like 'a + b' also isn't safe in C.

Cool, another reason to retire it

> If it's wrong break on compiler time, not on run time

The number being multiplied isn't known until runtime, so there's not a good way of doing this.

Well, funny that the compiler sees no problem in removing a check (on compile time) of a value that is totally possible to obtain, on runtime

Isn’t the very base issue the program didn’t sanitize the input?

Just blindly accepting user input is universally bad methinks.

Why would “basic” and “correct” overlap in any way?

how does it fail at that? it does exactly what the standard advertises. you need to write standard-compliant code.

It would seem GCC is doing its very best to make C an irrelevant language. They're taking "-pedantic" a bit too seriously.

All optimizing compilers do stuff like this. You yourself ask the compiler to do it when you pass it '-O2'. Its default behavior is, in fact, to not optimize based on the assumption that UB won't happen.

No. Just compile with -fno-strict-overflow, like I do.

For signed overflow in particular, sure. I was talking more about these kind of optimizations for UB in general.

Do you have any specific languages in mind?

Author of linked article mentions Rust

Fun thing there is rust feeds into the same optimisation pipeline as C or C++, so there's a definite risk of it inheriting some of their semantics via errors in the compiler implementation.

There have been several cases where Rust's use of "restrict" pointers exposed bugs in LLVM, and the Rust compiler had to disable some optimizations as a workaround. But I haven't heard of anything like that happening with signed overflow. (Probably any bugs with basic integer behavior would get noticed quickly?)

Another thing to watch out for here (especially if anyone's trying to transpile Rust to C) might be C's strict aliasing rules accidentally getting applied to Rust raw pointers.

The promise of Rust is that you never run into undefined behavior if you only use safe code. There are some caveats (using dependencies with badly written unsafe code, the noalias bugs others mentioned) but in the general case, if you're writing code without 'unsafe' blocks, you're not going to trigger UB.

You're certainly not going to run into LLVM optimizing your bound-check out of existence because it occurs after an overflowing operation.

Compiler bugs are always a potential issue for fuckup.

Hell, rust has had codegen issues because it extensively leveraged features which are almost non-existent in C, and thus were little exercised and poorly tested.

Rust is a good replacement for most use cases, but I think specifically in case where you're looking for a more predictable and less risky implicit behavior, the replacement should be more stable and predictable than Rust is at the moment.

What kind of language is C replacement without UB and better concurrency and more stable/predictable than Rust?

Question, but where do you get the idea that Rust is not stable or predictable? I understand you were asking for something _more_ stable/predictable than Rust, but Rust is already very stable and very predictable (in fact I can't think of any ways that Rust is "unpredictable").

Basically anything else.

C has a specification and weird stuff doesn't happen as long as you follow it. Doing so is very hard at times, but if a project cares about unpredictable optimizations, then it probably also cares about other kinds of unpredictable behavior. Which unfortunately eliminates a lot of languages that make no guarantees about their semantics.

But that code code is wrong in the first place with defined overflow.

If x == 0x804021 then x * 0x1FF yields 0x1DF, and then proceeed to access tab[0]. This is very probably NOT what the author wanted.

- Defined overflow doesn't help at all, in fact the compiler would preserve what is wrong in the first place

- overflow trap is a little bit better, at least you would get the occasion to think about it

Even if the compiler doesn't act on the UB the code is still wrong.

The root of all these problems is that C doesn't guarantee two's complement behavior when compiled for two's complement machines, as far as I understand. But why is that? Why are integer operations not defined as implementation-defined?

I get that the C standard cannot guarantee two's complement when C code can be compiled for other architectures. But looking at the list of supported architectures ( https://gcc.gnu.org/backends.html ), even exotic architectures like vax and microblaze seem to be two's complement.

Does gcc even support one's complement machines or those with CHAR_BIT != 8 ? If not, all those optimizations are utterly ridiculous. Basically an adversarial competition between compiler writers and users.

As I understand, the only ones' complement architecture in active use is UNIVAC (yes, believe or not UNIVAC is in active use and Unisys provides commercial support).

Non-8-bit char is a bit more common, probably the most common is TMS320 C5000, see https://lists.llvm.org/pipermail/llvm-dev/2009-September/026... for an example. As far as I know there is no GCC port, but it could very well have one, after all TMS320 C6000 port is upstream in GCC. (It is c6x in your table.)

Interesting, thanks. So the c5x (if it would be in the list) has 16-bit bytes and the c6x the usual 8-bit bytes.

Do you happen to know what compiler/language the UNIVAC folks use?

Wikipedia has pointers here: https://en.wikipedia.org/wiki/Unisys_OS_2200_programming_lan.... It seems links are now broken, but googling e.g. "Unisys publication 7831 0448" etc. should lead you to correct documents.

This is why undefined behaviour is insidious: if a tiny part of your code is undefined behaviour, the behaviour of your entire programme is.

Nitpick: it's not the behavior of the program that becomes undefined, but of the execution that it occurs in. For example, consider a program that reads argv[3] without making sure there's at least 4 arguments. If you call it with too few arguments, that entire execution is undefined (even stuff that happens before the out-of-bounds access), but if you call it with enough arguments, it is well-defined and the compiler has to emit code that will work.

Ah but the trap is that if the compiler can reason that UB always occurs (in reality it will reason that code is dead because the constraints it computes don’t allow for its execution) then it can remove the entire thing.

See Raymond Chen’s well known “undefined behaviour can result in time travel”.

Sure, but if UB always occurs, then every execution is undefined, so doesn't what I said still hold? (And the time travel is what I meant by "even stuff that happens before the out-of-bounds access".)

Yes. To spell it out: undefined behaviour doesn't just affect specific values nor just the behaviour of your program after triggering UB. It's the entire behaviour of your programme. So UB can 'travel backwards in time'.

Fun fact: not closing a string literal is UB. Ending a non-empty source file in anything other than a newline is also UB.

In addition to the points that other posts have made: it's also not especially new behavior. E.g. I see it at O2 gcc version 9.3.0 on i386. The optimization isn't performed by gcc 4.9.2 at O2 on i386. If I had to guess it probably shows up around GCC 5, but I don't seem to have any i386 vm's with gcc between 4.9.2 and 9.3 handy at the moment.

And 4.9 absolutely will do similar in other cases: fwrapv has existed since GCC 3.3 and fno-strict-overflow was introduced in GCC 4.2, which are flags to defeat these optimizations (and either of which avoids the crash on GCC 9.3)-- both introduced in response to increasingly effective optimization in prior GCC versions exposing errors like this. 4.9.2 just misses the optimization in this specific case.

Hmm, I had expected more from the article given the title.

This is about the least surprising UB. It didn't even travel backwards in time.

I think the lesson here is to stop using signed integer as an index into an array.

The lesson is to only use C when absolutely necessary, and regularly use all the sanitizers and other safety tools you can find. Not just when you suspect something fishy.

The lesson is you shouldn't use standard C, until the standard is fixed. Until then, compile with -fno-strict-overflow, like I do.

I am on split here. Signed overflow is UB and it is pretty well known, yet author is writing code that depends on such overflow.

But is there are a reason for `i >= 0` not raising a warning if it can't happen? Or is there a warning that was not enabled? I think `i >= 0 cant be false` would save a lot of headache

edit: one post mentions macros. Which makes sense in that case I think. You can easily write such impossible conditions with macros, so making it a warning would add a lot of warnings I guess

Although there are warnings for `if (true)/if (false)` "this condition is always true/false"

While I have been a member of the "unsigned counting variable" minority for a long time, this kinda drives the nails into the coffin for a lot of signed array index / offset use. It's just too big a risk to accidentally have the compiler go YOLO on you for some minor detail you missed.

Also, this UB optimization train has gone way too far and needs to back up a few stations.

I'm gonna say there needs to be a switch to make signed overflow not be UB. Maybe that already exists? starts checking docs

`-fno-strict-overflow`

-fwrapv

I compile all my C code with -fno-strict-overflow and -fno-strict-aliasing, and I recommend you to do so as well. C standard committee and GCC and Clang are being stupid, but that does not mean you should suffer their stupidity.

I am sort of uneasy with what the optimizing doing in here.

But I would not ever write something like this:

  int32_t i = x * 0x1ff / 0xffff;

Not because I am supersmart and will / can predict how optimizer will fuck it up. It is just that I am paranoid when coding and this type of code just simply hurts my perception for some reason.

This is the minimal example. For all we know, the original code might have made more sense in context?

It seems like a perfectly cromulent line of code to me, assuming you have overflow checks somewhere.

The trap in C is that you can't have overflow checks after the calculations testing if overflow happened by making assumptions about what the UB does. Once the UB has happened it's already too late. What you actually need is input range checks to ensure that following code can perform it's calculations correctly without hitting UB or you need to use helper intrinsics which perform checked math operations.

As already said it just ruffles my feathers the wrong way. I would not analyze why as I usually have better things to do with my programming time. The best answer I can come up with without thinking is something like this:

overflow is an error. I would prefer the code not to create errors unless absolutely needed.

D solved this particular problem by defining arithmetic as being 2-s complement, including wraparound behavior.

`-fno-strict-overflow` give you that behaviour in GCC and clang, too.

In gcc you can use the -ftrapv or -fwrapv compiler options to get defined signed integer overflow arithmetic.

These also work on Clang.

There’s no reason to expect the result of integer overflow to be negative, or any particular value at all. It is undefined.

So after checking that x > o there is no way for i (a product and ratio of positive numbers) to become negative.

The only reason this is surprising is that there is an expectation of wrapping on overflow. But this is simply not the behavior of the C virtual machine.

(Maybe today it makes sense to have even C wrap in a two’s-complement way? C is used in many places though, maybe there are still platforms in use that aren’t two’s-complement?)

Be careful: it's not just that the result of the multiplication won't be what you expect. The entire execution of the program becomes undefined. For example, if you had an unconditional `puts("hello");` between the multiplication and comparison lines, if the overflow happens, it would be allowed to print "goodbye" instead.

Yes, and in particular the compiler can assume that you stay within the defined behavior. That's what's enabling the optimization in removing the i < 0 check.

I think it was a mistake when CPU and language designers decided to make overflow in arithmetic operations not an error. This is wrong. Overflow can produce invalid data, which can break something elsewhere. Overflow has been a reason for security vulnerabilities.

For example, you don't want to have an overflow when calculating a total cost of ordered goods or size of an allocated memory block.

One could argue that it is a developer's responsibility to make sure overflow doesn't happen, but the history shows that developers usually fail to protect the program from such kind of errors. We have memory protection, why cannot we have overflow protection?

In my opinion, an overflow should cause an exception unless the program explicitly asks to ignore it. Of course this means that every addition now becomes a (very unlikely) conditional branch, but I guess this can be optimized by static prediction. Every memory access is already a conditional branch, and it doesn't cause problems.

But CPUs and programming languages seems to make writing overflow-safe code more difficult than unsafe. In assembly, you have to insert additional branch instructions, and in business-oriented languages like Java you have to write long sentences like Math.addExact(). If you write a complicated formula this way, it becomes unreadable.

Sadly, modern languages like Rust haven't fixed the mistake and also penalize writing safe code.

The only language I know with overflow protection is Swift: an overflow causes an exception there. Well done, Apple, you are lightyears ahead of open source software.

If your unsafe code is sound, you can't cause memory corruptions with wrapping integer overflows, no matter what you do with them. If your Rust code is engineered correctly, overflows are primarily logic bugs, not memory safety issues.

> Every memory access is already a conditional branch, and it doesn't cause problems.

What are you referring to here? The MMU/TLB?

> Sadly, modern languages like Rust haven't fixed the mistake and also penalize writing safe code

Writing safe code is smooth as butter. You just follow borrowing and lifetime rules and you get memory safety basically for free. Same with safe concurrent data structures.

Unsafe code is the one penalized. The amount of boilerplate and unstable features I need to use in Rust for ergonomic unsafe code is pretty staggering.

What's wrong with checked_add() and its friends in Rust?

They are longer to type than `+`.

It is easy to write unsafe code and difficult to write safe code. As a result, while most mathematical operations must be safe, developers use unsafe operations because they are easier to type and more readable.

Operators like `+` should be equivalent to checked_add().unwrap(), not to unchecked_add(). Why choose unsafe and rarely used operation as a default? It is a mistake. Swift uses sane defaults.

UPD: for example, let's say we need to allocate memory for N elements of size S and a header of size H. Here is the unsafe code:

    u32 s = N * S + H

And here is safe code:

    u32 s = checked_add(checked_mul(N, S).unwrap(), H).unwrap()

And here is safe code in Swift:

    let s: Uint32 = N * S + H

And unsafe code in Swift:

    let s = N &* S &+ H

In the "unsafe" example in Swift can you get to undefined behavior?

In your "unsafe" example in Rust, you still don't get undefined behavior.

https://huonw.github.io/blog/2016/04/myths-and-legends-about...

> In your "unsafe" example in Rust, you still don't get undefined behavior.

You get an unexpected behaviour instead. For example, imagine if you had a device for counting amount of bytes transferred over the network. If the counter overflows, you will get the wrong number, and issue an invalid bill to the customer. While that is a "defined" behaviour that is not what one expects.

I expect the counter either to work correctly according to normal math rules or indicate that it cannot perform the task due to hardware limitations.

One might blame the developer but history shows that developers fail to take account of overflow. For example, experienced developers writing Linux kernel made several mistakes, which lead to security vulnerabilities.

In most cases developers need standard math, not operations modulo 2^32 or 2^64. When you count bytes, you need an exact amount, not an amount modulo 2^32. Why do language designers provide modulo operations instead of normal math I don't understand.

Rust has chosen a weird path when the program works differently in debug and release mode. This is clearly wrong. I expect the same program to work the same way no matter whether optimizations are enabled or not.

> CPU and language designers

Which CPU? There are a lot of CPU architectures and their variants, some older than 40 years, and C is supported on most of them. Each CPU has its use case. Also, safety is not always a top requirement.

But I also think that true safety will come at silicon level, and not from a programming language.

> Also, safety is not always a top requirement.

Safety is the requirement for desktop, mobile and server processors but x86 and ARM penalize using safe math operations (you have to insert branch instructions).

> There are a lot of CPU architectures and their variants, some older than 40 years, and C is supported on most of them.

Between safety on modern CPUs and supporting 40-years CPU I would choose the first.

> Between safety on modern CPUs and supporting 40-years CPU I would choose the first.

I don't know what you mean with "modern", but x86 was first introduced in 1978, 1985 for 32-bit, and 2003 for 64-bit, for example.

There may be a water purifier somewhere running in a 1995 x86 box that still needs to be maintained.

> Safety is the requirement for desktop, mobile and server

There are several degrees of "safe", and I don't mean just "a hacker stole my data".

I am sure you hear about internet services crashing all the time. A failing service could be a safety issue too. And no, they are not written in C. There are entire companies living on providing redundancy and watchdog services. Often companies just go with "I don't care if the program crashes, we put another server up, just keep developing new features in whatever safe language we are using".

> There may be a water purifier somewhere running in a 1995 x86 box that still needs to be maintained.

Then they can use 1995 year compiler with unsafe arithmetic.

> Every memory access is already a conditional branch, and it doesn't cause problems.

Is that how it actually works or is it the kernel/hardware not letting you dereference 0?

I mean every memory access can cause an exception and handling an exception is a kind of branch.

C standards fault. GCC has intrinsics that let you do arithmetic safely with overflow checks

Just use them

https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins...

Those look rather inconvenient. Is there a sane way to keep things infix at least without having to crack open a spew of operator overloading?

Something like #pragma come_on_be_reasonable ?

(This is rhetorical. They point is it shouldn't be every coders personal responsibility to make the tool not be openly hostile. You may take pride in personal mastery of an unreasonable thing but that doesn't make it more acceptable)

Yes there is. Compile with -fno-strict-overflow.

Sounds easy enough. I've seen it before. I'll try to remember to use it. Thanks

> C standards fault.

I take issue with this sentence, since it makes it sound like there's something wrong with the C standard for it, which isn't the case.

Huh? There's all kinds of things wrong with the C standard. For example, they really went overboard with the UB even for cases that should have arguably been implementation defined or just throw an error.

Eg ending a non-empty source file with anything but a newline is undefined behaviour. So is not closing a string literal.

The C standard (finally) fixed this in C23, see <stdckdint.h>.

Sadly the macros they proposed are ugly, long to type and hard to read. People will not use them.

They seem reasonable to me? They're similar to the compiler builtins people already know and pretty short (7 characters…). What don't you like about them?

If these intrinsics are used can clang or msvc still compile it?

Clang should, MSVC has plans for it but no news since 2018: https://developercommunity.visualstudio.com/t/please-impleme..., so looks like it's a blackhole for now

There is an old german proverb that fits here, "Es kann nicht sein, was nicht sein darf", mocking lords and judges who purposefully mixed up things that are physically impossible to happen and things that ought not to happen by policy and made their life awfully easy that way.

People already weren't fond of that kind of logic a hundred years ago.

I think HTML5 sets a good example of how to do it instead: A large part of the standard is about defining behaviour for functionality the standard explicitly deprecates and disallows in compliant documents - all just so "legacy" HTML documents don't end up with "undefined behaviour".

This probably just an example, but why are they using a signed int for indexing???

They're correct to do this. Signed integers have more UB, therefore you /should/ use them in all situations when overflow isn't going to happen, because you're not going to need that extra defined behavior.

This lets you use UBSan most effectively and it's what the Google style guide says to do.

(Exception if you care about micro-performance: * / >> operations can be a little faster on unsigned types IIRC)

I would also prefer unsigned indexes but exactly because the compiler may assume that there will be no overflow, signed index access may be a bit faster and therefore preferable.

On most machines this days there is not really a performance difference between the math done on signed or unsigned integers. The only case would be if your wanting to the compiler to optimize on the fact that UB does exist. So like this in example "impossible things" get optimized out. The author here clearly does not want that.

Is there a reason why the language doesn't provide UB-on-overflow (and wrappring overflow) for both unsigned and signed types?

It always feels dirty deliberately using a signed type for something you know can never be neagtive just because that signed type has other properties you want.

The newest C standard will define signed integers as 2's complement. Getting rid of this instance of UB.

https://en.wikipedia.org/wiki/C2x

> I'm expecting this article to make the rust crew go in a crusade again, and I think I might be with them this time.

I think he makes a good point. By having overflow be undefined you leave yourself open to all sorts of issues.

Of course with well defined overflow you can still end up with a nonsense result that just happens to be in your valid range of values. If you don't plan to harden your code against integer overflow manually you need a language that actively checks it for you or uses a variable sized integer to be safe.

I've spent a fair bit of time running a fuzzer on Rust code that parses untrustworthy data. And the thing that saved my code time and time again was that Rust has runtime bounds checks. Even if I messed up index calculations, I'd get a controlled panic, not a vulnerability.

Ending up with a nonsense result ain't great, but it's still a step up from losing all guarantees about the behaviour of your program.

By the way, throwing an error on overflow is also a perfectly valid way to get defined behaviour and not end up with a nonsense value.

He did harden the code by having an explicit range check.

He is checking if the output is in a sane range. Not if the input values where. This may be correct for a trivial toy example but bite you in actual production code. Fun things like malloc(ab) vs calloc(a,b) where ab overflows in a well defined way and gives you valid pointer to an unexpectedly tiny buffer.

Only correct answer in this thread.

Ah someone else spotted it. The author's code is wrong whether signed overflow is well defined or not.

I think this article makes a far better case than the college hyperbole of "once you invoke undefined behavior it is allowed to format your drive and kill your cat."

As chance would have it, I am actually arguing with another team today about introducing undefined behavior into our code-base. They're arguing "well it works so its fine." I used this article to argue that it doesn't matter and we're inviting really insidious errors to come in later down the road.

I knew rust had to be mentioned. But this is a optimization bug. I break my code several times while optimizing thinking a minor change wont matter then end up undoing it all.

It's not a bug. This is perfectly acceptable behaviour according to the language standard. If anything, it's a standard bug.

This is not a bug. This is the model of C. You should blame C not GCC.

I blame GCC. The C standards committee is some out of touch with reality conglomerate working in a vacuum, taking 50 years of language history into account.

I expect my compiler vendor to be on my side, ie produce a compiler that helps me write good software and not get in my way. GCC is doing the opposite, it's deliberately looking to use the standard to fuck me over in the most subtle and unexpected ways. Signed integer overflow is undefined; that gives compiler authors the liberty to make it do anything they want, including well defined things that anyone would expect and find useful. But GCC decides to fuck you over so their devs can give you an arrogant reply and impose their superiority if you show up on their bug tracker.

Turns out people get upset when their code goes slower because the the compiler added traps on signed overflow.

Don't add traps, just let it overflow and don't fuck up any bounds checks the programmer added.

That's still slower than UB.

With UB compilers can assume that eg 'x + 1 > x' is always true for signed integers. That allows quite a few optimizations.

> just let it overflow

I agree with this. Making integer overflow UB is ridiculous.

> and don't fuck up any bounds checks

But I disagree with this. That's just normal UB-based optimization, one of the point of having UB at all is to do such optimization.

This is bad because few programs are built to deal with two positive numbers being multiplied returning a negative result.

GCC's developers work for companies that want more optimizations. UB is the way you get these optimizations.

But "gcc -fsanitize=undefined" is just as easy to run as "gcc -O3" is.

Well, that's undefined behaviour. It could have inserted system("rm -rf /*"); instead and be right about that.

I'm not so sure it's that wild. Signed overflow is UB so it makes perfect sense it's not possible to check overflow happened. You need to make sure overflow doesn't happen.

Similarly it's invalid to check if an access to an array is out of bounds after you have accessed it.

It's rude to spam bug reports like that. GCC has a mailing list, if a discussion is what you want.

Why did GCC change x < 0 to -1 < x? I guess it is faster - but why? x >=0 would have also been a valid transformation to archive the same flow

It's just an artifact of decompilation. It's the same operation in machine code (checking the MSB), there's no way to tell which one was used in the original source code.

An signed integer uses the leftmost bit for the sign: 00000000 = 0 10000000 = 128 = -128 https://onlinetoolz.net/unsigned-signed#base=2&value=1000000...

What the purpose of the / 0xffff?

Had the same question. This is the minimal implementation of the error, so there is no clue what was the original purpose.

I think as it is, it indexes a table where the first positions return tab[0], the next ffff/1ff will return tab[1] and so on.

If I see such a code at work I would reject it in less than 1ms

This is why I use unsigned integers for indices (and firmly believe most everyone should).

question about the condition removal he's lamenting

his code would have gotten an overflow at

int32_t i = x * 0x1ff / 0xffff;

where it not optimized away

so it seems gcc is maintaining the code behaviour, only reporting a wrong debug line for it due code being shifted around.

the part that was optimized away is `i >= 0` check. Which can't happen because x >= 0 already holds true

it also moves the division around

It should be noted that in this particular case it only happens when optimisations are enabled (-O2).

With -O2: https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename...

Without -O2: https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename...

[Edited. I originally wrote "It should be noted that this only happens when..." but inserted "in this particular case" after @eru pointed it out.]

In this particular case, yes. You can't rely -O0 to save your bacon in general.

Usually most UB only shows up when optimizations are turned on.

Make C Safe Again.

UBs in C language are the asleep plague just asking to get triggered into a głos bal disaster. Oops, too late.

There is another class of those there, described in some papers for CSmith. Milder but still quite bad.

Both got quietly not acted upon, when C++ got made.

Lets go back to Pascal/Modula/Oberon.

Reply on news.ycombinator.com