Hacker News Reader

Comment by anarazel | Hacker News Reader

First: I do dislike how hard it is to avoid some UB / how impractical some of the rules are.

But I also think a lot of discussions of this topic caricaturize compiler writers to a ridiculous degree. Almost describing them to write optimization passes looking for UB so they can over-optimize something, while cackling loudly in glee about all the programs they can break.

The set of people doing so overlaps with the set of people complaining that the compiler doesn't optimize their code sufficiently to a significant degree.

Lots of compiler optimizations need to know the ranges of values, hence logic to infer ranges. One of the sources for that is "can't happen" style logic - which nearly all of the time are things the code author would agree with if they thought long and hard. Not just about the code as written, but also good the code looks like after inlining (across TUs with LTO).

View on HN

I agree.

I don't have much sympathy for people who were doing things like writing multithreaded programs in the days before C documented its memory model and then becoming unhappy because new optimisations that legitimately help single-threaded code broke their programs.

In my experience C compiler maintainers have generally been open to the idea of offering guarantees beyond a narrow reading of the standard, but they want to be able to clearly state what it is that they're guaranteeing. "Keep old programs running" isn't enough.

I think the "Prevailing interpretation" that Yodaiken complains about is coming from the same place as suspicion of the "be lenient in what you accept" IETF principle: that sort of thing doesn't lead to robustness in the long run.

The way forward at this point is surely to define more things that are currently undefined (whether in the standard or by widely-implemented extensions).

> the same place as suspicion of the "be lenient in what you accept" IETF principle

No. It's fine (arguably desireable, but reasonable people might disagree) for implementations that encounter undefined behaviour to terminate execution immediately, especially if it's with a error message. The problem is when implementations silently willfully misinterpret what they accept, particularly in ways that cause (not "expose"; reading from a null pointer and discarding the value (for example) isn't) security vulnerablities.

> But I also think a lot of discussions of this topic caricaturize compiler writers to a ridiculous degree.

The inflamed backlash should tell you just how damaging it is to impose silent failure on meticulously written, previously fine programs.

> meticulously written, previously fine programs

With relatively few exceptions, if your program hits undefined behavior, then your program was already doing something pretty wrong to begin with. Signed overflow is a poignant example: in how many contexts is INT_MAX + 1 overflowing to INT_MIN actually sane semantics? Unless you're immediately attempting to check the result to see if it overflowed (which is extremely rare in the code I see), this overflow is almost certain to be unexpected, and a program which would have permitted this was not "previously fine" nor "meticulously written."

I feel compelled right now to point out that software development is a field where it is routine to tell users that it's their fault for expecting our products to work (that's what the big all-caps block of every software license and EULA says, translated into simple English).

> in how many contexts is INT_MAX + 1 overflowing to INT_MIN actually sane semantics? Unless you're immediately attempting to check the result to see if it overflowed (which is extremely rare in the code I see)

It's not that rare - I know that postgres got bit by particularly that issue, and several other other projects as well. Particularly painful because that obviously can cause security issues.

I recall a bug-report discussion that I sadly have never been able to find. It contains a pretty bad side-effect of this.

It had code like:

    int *p;
    // lots of code
    
    if (p != NULL)
        return 1;
    // use p

Then a later refactor wrongly added a single line before the if statement:

    int *p;
    // lots of code
    
    int a = *p;
    if (p != NULL)
        return 1;
    // use p

This meant the null check was optimized away, since de referencing a null pointer is undefined behavior, so the if-statement can be assumed to be always false. This then lead to actual errors (perhaps even an exploit, I do not recall) arising from the removed null-check.

I think in general the "sanity check" cases are the worst. It is hard to determine whether an expression causes undefined behavior if you cannot try and evaluate it. Perhaps a compiler intrinsic that checks (at runtime) whether an expression causes undefined behavior could be useful here. Though I can imagine such an intrinsic being essentially impossible to implement.

This was in the Linux kernel, which is compiled with special kernel flags which make dereferencing null pointers legal. In the context of that code, dereferencing a pointer and later checking it for null was absoutely meaningful. Optimizing the later null check was a compiler bug that was acknowledged and fixed. The compiler here didn't respect the semantics it had promised to kernel code. This was entirely uncontroversial; it was not a case of "unwanted optimization based on undefined behavior", it was a case of "compiler bug breaking well-defined code". Again, all this in a kernel context.

In user code GCC will still happily remove the null check because in user code this is an actual bug in the user's code.

https://lwn.net/Articles/342330/

https://lwn.net/Articles/342420/

Actually, there's a even worse version:

  struct foo { ...; bar_t bar[NBAR]; };
  struct foo* p = ...;
  bar_t* q = &p->bar[0]; // add rq, rp, #foo_bar_offs
  // other declarations
  
  if(!p) return NOPE; // optimized out
  // use p and q

Not even any dereferencing, just pointer arithmetic.

Yes, that one is especially nasty as &p->bar[0] is a constant expression completely solvable at compile time. It's equivalent to the offsetof() macro of stddef.h

Could you post a complete example please and tell us which compiler optimizes out this check? Clang and GCC don't: https://gcc.godbolt.org/z/dbTadEcra

> With relatively few exceptions, if your program hits undefined behavior, then your program was already doing something pretty wrong to begin with.

The author claims: “We have the absurd situation that C, specifically constructed to write the UNIX kernel, cannot be used to write operating systems. In fact, Linux and other operating systems are written in an unstable dialect of C that is produced by using a number of special flags that turn off compiler transformations based on undefined behavior (with no guarantees about future “optimizations”). The Postgres database also needs some of these flags as does the libsodium encryption library and even the machine learning tensor-flow package.”

So basically the programers of the most used C programs consider the C standard so broken that they force the compiler to deviate from it. (Or they are not able to do things right?)

Somewhat similar to sql situation. (difference being, i am not sure if anybody implements ansi-sql fully)

Standard is just common denominator agreed by actual competitors. (Standard takes into account the chipset where incrementing max_int produces a beep instead of min_int). If user wants to use more advantages of the product (dbms/compiler/hardware), one must sacrifice portability and use cnonstandard extensions.

> extremely rare in the code I see

Well, I learned of the change in compiler behavior some years back because I had written loop code with a sanity check which depended on signed integer overflow wrapping, along with a test case to prove that the sanity check worked, and that test case started failing:

   not ok 2 - catch overflow in token position calculation

   Failed test 'catch overflow in token position calculation'
   at t/152-inversion.t line 70.

To the extent I can be, I'm done with C. I leave it to people who think that silently optimizing away previously functional sanity checks is an acceptable engineering tradeoff, and who disparage those of us who have been bitten.

> people who think that silently optimizing away previously functional sanity checks is an acceptable engineering tradeoff

To be fair, as several people and TFA have pointed out, this isn't a problem with C, but with defective/malicous C compilers. Admittedly, that's not much help if you can't find a compiler that isn't defective/malicous, though, so I can only wish you the best of luck.

I don't get what you or the comment you're responding to are wishing for.

Do you want compilers to stop adding optimizations while staying within the bounds defined by the spec? That they somehow guess that a given piece of code that may trigger UB is too important for them to optimize it based on the assumption that the developer knew what she was doing and ensured that it wouldn't?

The case of a compiler update breaking the test sounds like desirable to me. It pinpoints a critical piece of code that was relying on a specific implementation's behavior that is not specified. This could have been triggered by a compiler or architecture change. If this is something that is a hassle to fix immediately, you can temporarily downgrade to the version of the compiler used earlier or disable some optimizations.

I fail to see why one would consider a compiler evolving while conforming to language specification defective or malicious (however, with that definition I fear that finding one that isn't may indeed be difficult).

I want default behavior which does not surprise me. I have come to understand that this probably means I want a different language, because the C spec essentially requires compiler authors to make optimizations which introduce surprising semantics in order to compete on performance with other languages.

It would be fine if I could opt into new semantics — something like Rust's "editions" would resolve my objections about these compiler optimizations. But that doesn't seem to be on offer in the C ecosystem.

Only moving away to other languages will do it.

C culture and to certain extent Objective-C and C++ ones are tainted by microptimizaitons while typing, where the compilers are the worst examples.

Unfortunely UNIX and C go together, so those that want to keep UNIX like platforms around bettter fix C somehow.

I think that's unfair. The taint is 100% on the C++ side of things.

Hopefully the newer compiled languages will kill C++.

Nope, the tait lies 100% on the copy-paste compatibilty with a C subset.

All you've done here is show that with C the concept of the l-user is still around.

That sucks. I can imagine next generation operating systems improving on Unix technically (standing on the shoulders of giants, yo), but establishing standards a la POSIX will be extremely difficult. It's hard to fight against the interest of platform vendors to lock their users in.

Microsoft fought for several years against C, trying to migrate everyone to C++ as the future of Windows systems programming.

Yet Azure Sphere, despite its security message, uses C only SDK.

Meanwhile Visual Studio now supports C11 and C17.

Market pressure, their customers weren't willing to buy into it, and the new Microsoft also wants all those POSIX FOSS packages written in C running on Windows.

> I don't get what you or the comment you're responding to are wishing for.

Quoting from another of my comments:

> > [What are you objecting to?]

> Inferring any propositional statement about the program (eg "this pointer is not null") from the fact that its negation would imply undefined behaviour.

That is what the problem is. Undefined behaviour is a licence to implement operations without regard for unusual corner cases, not to infer the absence of said corner cases from those operations and then apply that 'knowledge' elsewhere.

> I fail to see why one would consider a compiler evolving while conforming to language specification defective or malicious

It's https://en.wikipedia.org/wiki/Malicious_compliance in near-textbook form.

> Inferring any propositional statement about the program (eg "this pointer is not null") from the fact that its negation would imply undefined behaviour.

I'm not completely sure I understand that correctly, but do you mean that statements that are constant unless considering a possible (and "credible"?) implementation of UB shouldn't be fair-game for the compiler to optimize out? EDIT: I think I see a more tricky case that may be one of those you're referring to. Dereferencing a pointer further in the code shouldn't be a valid justification for optimizing out previous tests of it being null. I can relate with that but I suspect that it would prevent many classes of branch pruning.

I get what you suggest while describing malicious compliance but I can imagine that it could be a false impression resulting from trade-offs that favor optimization opportunities to "out-of-spec but canonical/natural" implementations.

> Dereferencing a pointer further in the code shouldn't be a valid justification for optimizing out previous tests of it being null.

Not further. Anywhere. If the implementation wishes to rewrite pointer dereferences from `use(*p)` to:

  if(!p) abort();
  use(*p);

it may do so (undefined behaviour!), but if it chooses not to do so, it may not later pretend that it did, and remove a explict `if(!p)` that the programmer wrote. Given:

  use(*p);
  if(!p) return NOPE;

this is fine:

  if(!p) abort();
  use(*p);
  //if(!p) return NOPE; // unreachable because of if, not because of use

but not:

  use(*p);
  //if(!p) return NOPE; // CVE-20XX-XXXXX

The implementation can optimize based on information (like "p is not null") that is actually true (even if that's because it made it true), but not based on information it assumed was true on the basis that it counterfactually could have made it true (but didn't).

> it would prevent many classes of branch pruning.

Yes, that's the general idea.

What is the meaning of use(*p) in case p is null? what should the compiler emit?

If p is statically null, it should emit compile time error. If not, it should emit machine instruction to deference p.

In all of the examples above it will emit a machine instruction to dereference p. What your grandparent is complaining about is that it will later remove an "if (!p)" test.

> What is the meaning of use(*p) in case p is null?

It dereferences a null pointer, invoking undefined behaviour, then calls the function `use` with the resulting value.

> what should the compiler emit?

Probably something to the effect of:

  ld r0 [sp+.p]  # if p is not already in a register
  ld r0 [r0]  # *p
  jsr use

but it would be fine to emit something like:

  ld r0 [sp+.p]  # if p is not already in a register
  jz r0 .panic
  ld r0 [r0]  # *p
  jsr use

because the jz can only be taken when undefined behaviour happens.

> I fail to see why one would consider a compiler evolving while conforming to language specification defective or malicious

Conforming to the spec is not a virtue. We want the compiler to be reasonable, regardless of whether the spec is. When the spec is malicious, conforming to the spec is malicious behavior.

For example, the Java spec says that the << operator (bit shift left) will accept any right operand, but performs a modulo-32 operation on the right operand before doing the shift. So `a << 4` is `a` shifted 4 places left, `a << 20` is `a` shifted 20 places left, and `a << 36` is `a` shifted 4 places left again.

This is absurd, and I'm comfortable calling it a bug in the spec. `a << 40` needs to have 0 in the lowest 40 bits. It does not need to have random values in bits 8-31.

This behavior is documented, but that doesn't make things better, it makes them worse.

But the philosophy that says "if it's documented, then it's OK" doesn't even allow for the concept of a bug in the spec.

Triggering signed overflow and testing whether it has occurred was how I protected the Delphi RTL memory allocation routines from signed overflow attacks, attacks which if not prevented, lead fairly directly to buffer overruns.

The code was written in Pascal and assembly, though, so it was safe from a C compiler.

I am quite thankfull that I learned systems programing via BASIC, Pascal and Assembly before coming to C.

As such my vision of low level coding isn't tainted by the ways of C.

Isn't checking for overflow depending on UB? i.e. in

    int a;
    // lots of code
    int b = a + 1;
    // check for overflow
    if (b <= a)
        abort();

the compiler is allowed to remove the check because in the absence of UB a + 1 > a, therefore the conditional is always false.

As brilliantly said further in this thread by someone else, maybe check that something is OK before doing it.

In that case, something like: [...] if (a > INT_MAX - 1) abort(); int b = a + 1; [...]

> As brilliantly said further in this thread by someone else, maybe check that something is OK before doing it.

Much easier said than done:

https://github.com/postgres/postgres/blob/master/src/include...

Good point that checking for overflow of a multiplication beforehand is hard. But I don't think checking after the fact would be much easier. So thus a C modification to remove the undefined behavior probably wouldn't be very useful still.

The thing is that it's necessary, and (x86, not sure about arm) asm tells you via the d register if not the overflow flag. So why doesn't the C standard arguably provide a misuse-resistant way of doing so? Instead every project having to reinvent it and get it wrong if they do the "obvious" (but wrong) thing.

What would the misuse-resistant way be? Would you want the multiplication operator (*) to return 2 values: the number and a bool telling you whether it overflowed?

Yeah, a standard library function that does this would be good. But many people would just use * instead of this function, and so the problem would partially remain.

mul_would_overflow(), mul_wrap(), mul_saturate(), mul_do_whatever_the_hardware_does_just_please_dont_break_my_code_by_assuming_ub. Can't help people who just use * and don't care about overflow without breaking compatibility, but at least the people who do care won't have to reinvent safe arithmetic.

Yeah, I agree with you. But it would be hard due to there being so many integer types. A binary operation needs to consider 3 types: the types of each argument and the type of the result.

In c++ it would be a little simpler due to templates, so the types of the arguments to the function can be derived. But the type of the result can still confuse programmers. Although maybe it's not so bad, because an overflow that happens during multiplication (or other math operations) is undefined-behavior, but an overflow that happens during assignment of the multiplication result to a variable that can't hold it is only implementation-defined behavior, not undefined behavior.

If the program was previously "fine" on version x.y.z of some compiler, then it is most likely still fine on it. That's the target that the program was written for.

There's some disagreement on whether you can call a program "fine" that breaks after switching to a newer version, or a different compiler.

I see a lot of programmers out there that unfortunately use the behavior of their code on whatever compiler they're using at the moment as a proxy for what the language actually guarantees.

Well, I can imagine a program having something where new C comments ( // ) makes it not divide, like:

a = 5 //* junk here */ 2 ;

I'm sure there are ioccc or underhanded C contest entries doing this to make code work differently on compilers based on if // is starting a comment line or not.

Sure it is an ugly way of writing stuff and you'd be hard pressed to find lots of real world traps like this, but when/if you did have code that "suddenly" miscompiles you might actually think your old code with an old compiler did work, and a new compiler for "the same" language breaks your program. I don't think everyone code base should need full rewrites ever time a new compiler comes out.

The current situation is not good for compiler writers either. But nobody has ever shown that either C programmers want to sacrifice safety for "optimizations", or that these UB optimizations actually improve performance of anything.

What do you mean by "these UB optimizations"? C is a low-level language; it's basically impossible for a compiler to reason about the code unless it makes certain assumptions. It needs to assume the code is not self-modifying to do pretty much any code-generation more intelligent than a macro assembler. It needs to assume the code isn't messing with the stack frames/return addresses in order to inline functions. It needs to assume the code isn't using an out-of-bounds pointer to access a neighboring local variable, so that it can move local variables into registers. "gcc -O0" is a good approximation for the performance you get if the compiler isn't allowed to optimize based on UB.

Yes, that means C without optimizing without UB is slower than Java. Optimizations need some form of reasoning about what's happening. For Java it's optimizing based on guarantees provided by the language (there's no raw pointers that could mess with the things listed above). But C doesn't provide any hard guarantees, so instead it needs to blindly assume that the code will behave sanely.

Also note that for many of the more manageable sources of UB, most compilers provide a choice (-fwrapv, -fno-strict-aliasing, ...). Yet few projects use these options, even when they use other gcc/clang-specific features. Doesn't that indicate that C programmers indeed want to sacrifice safety for optimizations?

Exactly.

For example there were programs 30-40 years ago that relied on exact stack layouts. These days everybody would agree they are completely broken.

The issue of course is that it is extremely hard to write programs that have no UB. It would be nice for compilers to have an option to automatically introduce assetions whenever they rely on some UB-derived axiom, basically as a sort of lightweight sanitizer.

In fact if we had sanitizers 30-40 years ago probably things would be better today.

> It would be nice for compilers to have an option to automatically introduce assetions whenever they rely on some UB-derived axiom

Modifying a value from a different thread without synchronization is UB. The compiler assumes this does not happen in order to e.g. move things into registers. Could you elaborate how (and how often) you would like to have this kind of UB-derived axiom ("this value remains the same from here to there") checked with assertions?

Obviously you wouldn't be able to catch many, or even most cases. Use-after-free is another case that would be very expensive to detect.

I was using it in 2000,

https://www.parasoft.com/products/parasoft-insure/

21 years later it is still an uphill battle to adopt such technology in C and C++ projects.

That's good example, because nobody would complain if stack layouts changed and those programs failed. But if the compiler chooses to "optimize away" checks on stack layout, that's a different thing altogether. Also note that if you use pthreads or Linux clone or you are writing an operating system you can need to rely on exact stack layouts even today.

Stack layouts are only really relevant at ABI boundaries. In these cases the layout is usually specified in extensions to C or in other ways, such as handwritten assembly.

Linux clone, pthreads, and os code commonly look at stack boundaries

Not sure what you are referring to with stack boundaries. Of course the ABI imposes some minimal requirements at ABI visible points, but these days you can't even rely on the existence of frame pointers to traverse the stack and you have to use the DWARF unwind machinery. And the content of the stack frame itself is completely unspecified of course.

So I create a thread with a custom stack which is an allocated buffer. At the top, I write a sequence of bytes in some order. Then I periodically read the top of the stack to see if the stack is getting close to overflow. Meanwhile, the thread code is also addressing the same store.

We had sanitizers since C exists, 1979 to be more exact.

"Although the first edition of K&R described most of the rules that brought C's type structure to its present form, many programs written in the older, more relaxed style persisted, and so did compilers that tolerated it. To encourage people to pay more attention to the official language rules, to detect legal but suspicious constructions, and to help find interface mismatches undetectable with simple mechanisms for separate compilation, Steve Johnson adapted his pcc compiler to produce lint [Johnson 79b], which scanned a set of files and remarked on dubious constructions. "

-- https://www.bell-labs.com/usr/dmr/www/chist.html

I was going to add "in open source compilers" to hedge my statement :).

That seems to be a static analysis tool though (which generally have not been great). Did it also inject runtime checks?

No, but still not even that gets the love it deserve.

And a famous commercial variant of it has been PC-lint from https://www.gimpel.com/.

Being available on open source compilers does little to change the culture, as per latest surveys only 11% of developers care to use any kind of tooling for improving their code quality in C and C++.

At CppCon a couple of years ago, only about 1% of the audience answered positively to Herb Sutters' question.

Those numbers are indeed a bit depressing.

Here is a recent survey, with a more positive number of about 37%, see question 10.

https://isocpp.org/files/papers/CppDevSurvey-2021-04-summary...

For your last point, the extent of UB driven changes to semantics is still not widely known in the programmer community. Programmers don't read the standard - they read K&R, and K&R is right now describing a different language. We've had 15 years of programmers repeatedly filing bug reports to be told that the expected, tested, relied on, behavior was ephemeral. Only very sophisticated projects figure out about UB.

Of course compilers have to make assumptions. The debate is (a) over what assumptions it is proper to make and (b) what are the permissible behaviors. The false dichotomy: either do without any optimizations at all or accept whatever UB gives you, is not a useful approach.

So what optimizations do you mean with "these UB optimizations" then? And would it change your mind to see a benchmark proving the usefulness of that particular UB optimization?

e.g. assuming UB can't happen: deleting overflow or null pointer checks, deleting comparisons between pointers that are assumed to point at different objects, ...

I'm surprised you don't think deleting a bunch of checks will improve performance.

> So what optimizations do you mean with "these UB optimizations" then?

Inferring any propositional statement about the program (eg "this pointer is not null") from the fact that its negation would imply undefined behaviour.

> most compilers provide a choice (-fwrapv, -fno-strict-aliasing, ...). Yet few projects use these options,

Even if you opt into those (and the projects I've been involved with have), the precedent is established: when new compiler optimizations are introduced with disruptive semantics, they are opt-out, not opt-in — a fail-dangerous failure mode.

> Doesn't that indicate that C programmers indeed want to sacrifice safety for optimizations?

Maybe so. Which is one reason I wouldn't call myself a "C programmer" any more. The demands to program responsibly in C are absurdly high.

> Even if you opt into those (and the projects I've been involved with have), the precedent is established: when new compiler optimizations are introduced with disruptive semantics, they are opt-out, not opt-in — a fail-dangerous failure mode.

Actually, the default for gcc is -O0. You are opting in with -O2 etc.

That is exactly how C should have kept being used, a portable macro assembler, while leaving everything else on the IT stack for more saner languages.

BCPL was anyway designed to Bootstrap CPL, nothing else.

To do UB "optimizations", the compiler first needs to figure out that there is an UB it can "optimize" anyway. At this point instead of "optimizing" it could, and in my humble opinion absolutely should, blow up the compilation by generating an UB error, so people can fix their stuff.

What about backwards compatibility in regards to a new compiler version deciding to issue errors on UB now? You don't have any guarantees about what happens with UB right now, so if you upgrade to a new version compiler that generates errors instead of "optimizations" everything would be still as before: no guarantees. And it's frankly a lot better to blow up the compilation with errors than to have the compiler accept the UB code and roll a dice on how the final binary will behave later. You can either fix the code to make it compile again, or use an older "known good" version of the compiler that you previously used as a stopgap measure.

I fail to see any reason whatsoever why compilers are still doing all kinds of stupid stuff with UB instead of doing the right thing and issuing errors when they encounter UB.

I also fail to see why the C language designers still insist on keeping so much of the legacy shit around.

> To do UB "optimizations", the compiler first needs to figure out that there is an UB it can "optimize" anyway.

That's not how compwillrs work. In fact in the general case it is impossible to figure out at compile time that "there is an UB".

The compiler instead assumes as an axiom that no UB can ever happen and uses the axioms to prove properties of the code.

These days if you want to catch UB, compile with -fsanitize=undefined-behaviour. The program wll then trap if UB is actually detected at runtime.

> These days if you want to catch UB, compile with -fsanitize=undefined-behaviour. The program wll then trap if UB is actually detected at runtime.

So, let me get this straight, someone wants to make sure pointer p is not null (in the wrong way), and codes something like the examples in posts above like if (!p) ... and if that doesn't trigger calls use(*p), but compiler decides p can never be null because that would be UB and hence removes the check.

The C coder dumps the code and gets upset because the check is removed and gets the hint to catch UB by adding -fsanitize .. that "catches UB" in the above scenario so that the program will "trap if UB is detected".

I think we just came full circle there.

Sure, the -f will catch ALL detected bugs and so on, but I still found it a bit funny.

It is a bit different.

Ubsan will abort an invalid program if it detect ub. It doesn't let you handle it. So you shouldn't remove the erroneous check, but fix it so it is no longer erroneous, and ubsan will help you identify these errors.

Also ubsan adds significant overhead so it is not really appropriate for production builds unfortunately (hence my wish for a less powerful ubsan-lite but with lower overhead).

I think you are misunderstanding the situation. Given code like:

    if (!p) {
        use(*p);
    }

(given no previous knowledge about p) no compiler will remove the "if (!p)" part.

What people are complaining about is the opposite case:

    use(*p);

    /* The compiler reasons that if p == NULL, the program would have crashed by now,
       so if we got here, p != NULL must hold. */

    if (!p) {  // the compiler can remove this branch
        report_error();
    }

> To do UB "optimizations", the compiler first needs to figure out that there is an UB it can "optimize" anyway.

The compiler assumes UB will never happen and it makes transformations that will be valid if there happens to be no UB. This doesn't require any explicit detection of UB, and in some cases UB or not is simply undecidable at compile time (as in no compiler could detect it without incorrect results).

Without these assumptions the resulting compiled code would be much slower, though some optimizations have different danger vs speed impact and there certainly can be a case that there are some optimizations that should be eschewed because they're a poor trade-off.

There are many cases where current compilers will warn you when you've done something that is UB. It's probably not the case that they warn for every such detectable case and if so it would be reasonable to ask them to warn about more of them.

I think your irritation is just based on a misunderstanding of the situation.

Compiler authors are C(++) programmers too, they also don't like footguns. They're not trying to screw anyone over. They don't waste their time adding optimizations that don't make real performance improvements just to trip up invalid code.

Yes, some UB are not decidable at compile time, but a lot could be easily speced to have a defined behavior at runtime, such as overflows.

The main reason to not spec these things is because people would be arguing "this makes compiled code on my esoteric 9-bit 1-complement chip slower" or "there was this chip in the 70s that did things differently" or "but a short int on Cray was 64-bit". Great, so now the spec has avoidable unnecessary undefined behavior all over the place, and the code other people wrote still does not run correctly on your 9-bit chip. Brought to you by the same people who decided "NULL is not necessarily (void*)0", and who define those integer types everybody uses (instead of stdint) with an "at least this big".

Yes, a lot of that is legacy stuff and was added to accommodate and model things that already existed (the wrong way to go about it, IMO, but hindsight is 20/20), but that's my argument: fix this stuff once and for all and for good in an upcoming spec iteration.

>Without these assumptions the resulting compiled code would be much slower

In some cases, this is true (for different levels of "much slower"), but the trade off here is still "running code that works, but a little slower" vs "running code that does not work and will launch a nuclear strike at Switzerland by accident, but really fast".

In a lot of cases, it will not be slower, or at least not much slower.

>I think your irritation is just based on a misunderstanding of the situation.

Frankly, not really. I started writing my first C (and C++) in the early 90s, and I think I do understand the situation pretty well by now. But I should have been more precise in my initial ranting comment, I give you that.

>They're not trying to screw anyone over.

I didn't say that they are.

Note that (void *)0 is always NULL, as mandated by the standard.

But, to address the content of your comment: defined behavior at runtime is not necessarily good behavior at runtime. Defining signed integer overflow to wrap, for example, is probably a bad idea, because this is rarely the intent of the code. Having all such operations trap might be a good idea, but now you're going to get the same "stop breaking my working programs" people angry at you.

Yes, thankfully at least with NULL they didn't fall into the legacy trap and messed up the standard with non-zero NULL that some machines before have been kind of using.

>Defining signed integer overflow to wrap, for example, is probably a bad idea

I wouldn't call it great behavior, but it's at least what most people expect will happen, and most people will be able to understand what's going on, and it's fast on most systems that matter. However, it's still undefined behavior. Just codifying overflows to be wrapping, would therefore be an improvement in my opinion, at least over what we have today.

I would say that most people expect it not to happen. If they really had to mandate a behavior, it should be to trap.

> Note that (void *)0 is always NULL, as mandated by the standard.

Also note that this is distinct from the memory representation of (void *)0 being all 0 bits, which is explicitly not mandated.

> But nobody has ever shown … that these UB optimizations actually improve performance of anything.

That’s just not true. Already examples in this thread: https://news.ycombinator.com/item?id=27223870

The caricatures are somewhat accurate though, optimizations that look at UB adversarially are never anywhere close to justified.

> The set of people doing so overlaps with the set of people complaining that the compiler doesn't optimize their code sufficiently to a significant degree.

There's no contradiction here, and the overlap is generally just "people who care". The optimizations that are not safe shouldn't exist, and the optimizations that are safe should be good.

> nearly all of the time are things the code author would agree with if they thought long and hard

I highly doubt this is the case for even one situation.

> optimizations that look at UB adversarially

The whole point is that there isn't such adversarial thing like "we're going to find the UB right there, won't even print a warning about it, and mess up your crappy code, haha!"

Optimizers aren't reasoning about code like people do (start to finish with high-level understanding of the whole function), but rather as series of mostly dumb, mostly isolated small passes, each pass changing one little thing about the code.

It just happens that one pass marks certain instructions as "can't happen" (like the spec says), then another pass simplifies expressions, and then another pass that removes code that doesn't do anything, usually left over from the previous steps. They sometimes combine in an "adversarial" way, but individually each pass is justified and necessary.

Compilers already have lots of different passes. Splitting optimizations into passes is a way to keep complexity closer to O(n) rather than O(n^2), but this architecture makes interactions between passes very delicate and difficult to coordinate, so it's difficult to instrument the data to avoid only cases of annoying UB without pessimizing cases that users want optimized.

Name one such optimization. We'll be happy to refute your points for that one.

Compiler dev here. Do you really think we just come up with code transformations and add them to the compiler just because?

All code transformations have a compile time cost and runtime perf impact. We don't add transformations unless the runtime perf impact greatly outweighs the compile time cost.

These optimizations are added because they measurably improve the performance of real code. This comes up in every review for new or updated optimization passes. This claim that they aren't justified is actually rather insulting to the effort put in to improve perf without taking days to compile.

However Fortran, Ada, Java, .NET Native, Swift,... are certainly less subject to such optimizations with surprises.

Reply on news.ycombinator.com