Hacker News Reader

It goes further than that. The debates around UB a more about whether the compiler can assume that there are no buffer overflows and perform optimizations based on that. For example, if you have a local variable `char buffer[16]`, and there is an access `buffer[i]`, should the compiler be allowed to derive that 0 <= i < 16, and perform optimizations on other uses of `i` based on that "knowledge"? If all bets are off for `buffer[i]` when i < 0 or i >= 16, why shouldn't it? But some argue that the compiler shouldn't, exactly because such automated formal reasoning can lead to arbitrarily wide reaching effects. The problem is that it's hard to see where a middle ground could be.

The result is a perpetually unstable situation fostering inevitable errors when the programmer is surprised by the compiler. There's no remedy except to move away from C to more tightly specified languages.

The problem is ecosystems like UNIX based platforms, that will never move to something else.

The only solution is to throw them away, e.g. Android style.

Android threw away a unix based platform? News to me.

It did, ask termux guys.

The Linux kernel is an implementation detail.

There is nothing UNIX related exposed to userspace.

Java and Kotlin applications, while NDK APIs are clearly defined,

https://developer.android.com/ndk/guides/stable_apis

Or Treble Project for that matter,

https://source.android.com/devices/architecture/hidl

Try to use private APIs (which is what the Linux underpinnings are) and the application might be blessed with being killed by the OS.

If the array index is unconditional, then sure that seems like a reasonable undefined behavior? If the access is conditional and the compiler can't infer whether the branch is taken, though, then it shouldn't make that assumption.

This reminds me of a bug in the msp430 variant of gcc. The compiler would replace "x <= literal" with "x < literal + 1" because it generated more efficient code. If the literal was UINT_MAX, though, the compiler would trudge ahead and roll over the literal to 0. It would then subsequently reason that the comparison was unconditionally false, and then completely optimize away the nominal path of my code. That was a frustrating day of debugging!

What I never see mentioned in this type of discussion is what this compiler trick buys us. Like how much faster does my program run because of deductions like these? And are there any patterns that particularly benefit from it? Could they hint the constraint manually?

There are plenty of examples in this thread alone.

In my opinion hinting won't help. Everyone who cares about the performance gain will enable the hint (which I expect will be almost everyone) so you have won nothing but added noise with the hint.

There is not a single compelling example. Just contrived examples with dubious speedups and obvious errors.

The trouble here is porting the maximalist disaster of one scenario to another scenario simply because the same phrase is used to describe them.

Because a buffer overflow may lead to arbitrary control flow escape via ROP or other hijacking of the instruction pointer, it does not follow that e.g. signed overflow is similarly dangerous.

Signed overflow, if not guarded against, may enable buffer overflow, but the deeper irony is that detecting when signed overflow has already occurred is close to impossible if the compiler is also aware that the check implies that it has occurred.

Thus, code designed for safety, guarding against buffer overflows by detecting signed overflow, ends up not protecting against buffer overflow, simply because signed overflow is described using the same name as buffer overflow.

Yes, now if the proposal would be not to redefine the semantics of UB (as the post to which I replied suggested) but rather about specifying that buffer overflows are UB1 and integer overflows are UB2, and UB2 means something different from UB1 (perhaps identical to implementation-specified behavior, perhaps something different), then that could be reasonable.

> given appropriate very specific input data

That's the operative words there. Compilers are not required to ensure that, in sufficiently perverse circumstances, undefined behaviour never results in demons flying out of you nose. They are, however, required to not actively put said demons there themselves, because that's part of what distinguishes a programming language implementation from a piece of malware masquerading as a programming language implementation.

Compilers never actively try to put demons in your program. However, they do occasionally end up making constructs that result from a reasonable (if not the most well-thought-out) chain of decisions that end up looking like demons to the untrained eye.

I didn't say "try"; if it quacks like demon, it's a demon, and the implementation is (whether the standards commitee likes it or not) required not to actively put it into programs that didn't already have it[0], deliberately or otherwise.

0: like this one:

  int* zero = 0;
  int bad = *zero; // maybe crash lol
  if(!zero) abort(); // definitely crash lol
  printf("Uln nasaloth geb hai!\n");

It just isn't that simple in practice. Modern compilers automatically apply a plethora of different rules in sequence when transforming code. This results in a long chain of transformations where each operation is perfectly reasonable when considered on its own. Unfortunately, for certain inputs, the sum total occasionally appears demonic.

The current state of the art is such that there is no way to reliably prevent this that doesn't also severely reduce the optimization abilities of modern compilers. Realistically, adding such language to the standard would (at best) result in worse optimizations by default and additional compiler flags to reenable the more aggressive "nonstandard" ones. (Such flags already exist for various arithmetic operations.)

> where each operation is perfectly reasonable when considered on its own.

No, it is not. For example, the transformation:

  int read_and_discard = *p;
  // vvvv
  int read_and_discard = *p;
  __unsafe_assume_always(p != 0);

is not reasonable, since p is not, in fact, always nonnull.

Your examples are far too simplistic. Real world code is going to be far more complex and will tend to resist trivial analysis.

For example, please explain how to prevent this without also (inadvertently) preventing the removal of unnecessary null checks when functions are inlined. What about an unnecessary null check that's hidden inside a macro? What about whole program LTO?

A macro could be used in a variety of situations. In some of them, you need the null check for safety. In others, the null check in entirely redundant. Short of AGI, the compiler can't actually comprehend the code it operates on. Yet surely you expect it to eliminate "obviously" redundant work? It accomplishes this by applying a large set of fairly simple rules.

(There are probably much better examples but I can't think of them off the top of my head.)

In practice, when I play with Godbolt I find that the "obvious" cases tend to result in helpful diagnostic messages.

> For example, please explain how to prevent this without also (inadvertently) preventing the removal of unnecessary null checks when functions are inlined. What about an unnecessary null check that's hidden inside a macro?

If a null check is unnecessary, that's because it is reachable only from the not-null side of some previous null check. The compiler can track that information just fine, using the same tools it uses to track false information derived pointer dereferences.

If:

  if(!p) abort();
  // we know p is non-null here, regardless of the dereference
  use(*p);
  MACRO(p);

expands to:

  if(!p) abort(); // first null check (this is relevant to optimizing out unnecessary null checks)
  use(*p); // pointer dereference (this isn't)
  if(!p) abort(); // second null check (unnecessary *because of the if*)
  utilize(*p); // more pointer dereference

the compiler can optimize out the second if (the "unnecessary null check" you refer to) based on the first if, regardless of whether the pointer dereference is even there.

This is still too simplistic. Think along the lines of:

    void foo(T *p) {
      if (!p) abort();
      bar(p);
    }

    void bar(T *p) {
      use(*p);
      MACRO(p);
    }

In other words, the first null-check and the latter check are in different functions that may not even be in the same compilation unit, or where the call sequence is hard to reason about due to function pointers etc.

Assuming bar isn't inlined (eg separate compilation unit, as you mentioned), the null check in MACRO is not unnecessary, because bar can be called from places other than foo, and those places might pass null pointer.

This is one of several situations where it might be useful for the compiler to excercise it's perogative to implement operations without regard for undefined behaviour to rewrite `use(*p);` as `if(!p) abort(); use(*p);`, which would make the null check in MACRO unnecessary, but unless it does so, the check is not unnecessary, just insufficient.

I disagree. The transformation should be valid; if p is null then the result is undefined and may crash, including the rest of the program after that occurs. However, I do think that it should not make such a transformation for a volatile read/write; in that case it should not make such an assumption (who knows if it is meaningful on the target computer or some sort of emulator or operating system or debugger or whatever).