That's a reasonable intuitive interpretation of how it should behave, but according to the spec it's undefined behaviour and compilers have a great degree of freedom in what happens as a result.
> Note that, apart from contrived examples with deleted null checks, the current rules do not actually help the compiler meaningfully optimize code. A memcpy implementation cannot rely on pointer validity to speculatively read because, even though memcpy(NULL, NULL, 0) is undefined, slices at the end of a buffer are fine. [And if the end of the buffer] were at the end of a page with nothing allocated afterwards, a speculative read from memcpy would break
‘Only’ on platforms that have memory protection hardware. Even there, the platform can always allocate an overflow page for a process, or have the page fault handler check whether the page fault happened due to a speculative read, and repair things (I think the latter is hugely, hugely, hugely impractical, but the standard cannot rule it out)
Even if it didn’t, I don’t think the standard should assume that “Platforms without memory protection hardware also have no problem reading NULL”
An OS could, for example, have a very simple memory protection feature where the bottom half of the memory address range is reserved for the OS, the top half for user processes, and any read from an address with the high bit clear by code in the top half of the address range traps and makes the OS kill the process doing the read.
Sometimes hardware is designed with insufficient input from software folks and the result is something asinine like that. That, or some people like watching the world burn.
That's implementation defined (more or less) ie teh compiler can do whatever makes mst sense for its implementation.
Undefined means (more or less) that the compiler can assume the behaviour never happens so can apply transforms without taking it into account.
> to allow compilers to use it as an optimization point
That's the main advantage of undefined behaviour ie if you can ignore the usage, you may be able to apply optimisations that you couldn't if you had to take it into account. In the article, for example, GCC eliminated what it considered dead code for a NULL check of a variable that couldn't be NULL according to the C spec.
That's also probably the most frustrating thing about optimisations based on undefined behaviour ie checks that prevent undefined behaviour are removed because the compiler thinks that the check can't ever succeed because, if it did, there must have been undefined behaviour. But the way the developer was ensuring defined behaviour was with the check!
As to your point about checks being erased, that generally happens when the checks happen too late (according to the compiler), or in a wrong way. For example, checking that `src` is not NULL _after_ memcpy(sec, dst, 0) is called. Or, checking for overflow by doing `if(x+y<0) ...` when x and y are nonnegative signed ints.
> memcpy(void* ptr1, void* ptr2, int n)
Copy n bytes from ptr1 to ptr2. UNDEFINED if ptr1 is NULL or ptr2 is NULL
‐------
It might also have come from a "explicit better than implicit" opinion, as in "it is better to have developers explicitly handle cases where the null pointer is involved
Of course, someone needs to check in the layers of abstraction. The user, programmer, compiler, cpu, architecture.. They chose for the programmer, who like to call themselves "engineers" these days.
>It's pretty much a tiny wrapper around assembler
Assebler has zero problem with adding "null + 4" or computing "null-null". C does, because it's not actually a tiny wrapper.
NULL doesn't exist in assembler, and in C, NULL is only a defined as a macro. It's not something built-in.
C doesn't have any problems adding 4 to NULL nor subtracting NULL from NULL.
#include <stddef.h>
int main() {
if (NULL + 4) {}
if (NULL - NULL) {}
return 0;
}"Having problems" is not a fair description of what's at stake here. The C standard simply says that it doesn't guarantee that such operations give the results that you expect.
Also please note that the article and this whole thread is about the address zero, not about the number zero. If NULL is #defined as 0 in your implementation and you use it in an expression only involving integers, of course no UB is triggered.
If you wanted to know what (for instance) memcpy actually did, you looked at the source code, or even more likely, the assembler or machine code output. That was "the standard".
There is an entire industry that does pretty much that... today. They might run in flash instead of RAM, but still, a few kilobytes.
Probably there are more embedded devices out there than PCs. PIC, AVR, MSP, ARM, custom archs. There might be one of those right now under your hand, in that thing you use to move the cursor.
Which industry runs C compilers on embeded devices? Because that is what the part you elipsised out was talking about.
jcxz skip
rep movsb
skip:When you're writing a compiler for an architecture where every byte counts you don't make it write extra code for little benefit.
Programmers were routinely counting bytes (both in code size and data) when writing Assembly code back then, and I mean that literally. Some of that carried into higher-level languages, and rightly so.
1. Initially, they just wanted to give compiler makers more freedom: both in the sense "do whatever is simplest" and "do something platform-specific which dev wants". 2. Compiler devs found that they can use UB for optimization: e.g. if we assume that a branch with UB is unreachable we can generate more efficient code. 3. Sadly, compiler devs started to exploit every opportunity for optimization, e.g. removing code with a potential segfault.
I.e. people who made a standard thought that compiler would remove no-op call to memcpy, but GCC removes the whole branch which makes the call as it considers the whole branch impossible. Standard makers thought that compiler devs would be more reasonable
This is a bit of a terrible take? Compiler devs never did anything "unreasonable", they didn't sit down and go "mwahahaha we can exploit the heck out of UB to break everything!!!!"
Rather, repeatedly applying a series of targeted optimizations, each one in isolation being "reasonable", results in an eventual "unreasonable" total transformation. But this is more an emergent property of modern compilers having hundreds of optimization passes.
At the time the standards were created, the idea of compilers applying so many optimization passes was just not conceivable. Compilers struggled to just do basic compilation. The assumption was a near 1:1 mapping between code & assembly, and that just didn't age well at all.
That optimization allows using 64-bit registers / offset loads for signed ints which it can't do if it has to overflow, since that overflow must happen at 32-bits. That's not an uncommon thing.
The strict aliasing rules are not violated by typical POSIX socket code as a cast to a different pointer type, i.e. `struct sockaddr` by itself is well-defined behavior. (and POSIX could of course just define something even if ISO C leaves it undefined, but I don't think this is needed here)
Basically all usage of sendmsg() and recvmsg() with a static char[N] buffer is UB, is one big example I've run into. Unless you memcpy every value into and out of the buffer, which literally no one does. Also, reading sa_family from the output of accept() (or putting it into a struct sockaddr_storage and reading ss_family) is UB, unless you memcpy it out, which literally no one does.
Many compiler devs are on record gleefully responding to bug reports with statements on the lines of "your code has undefined behaviour according to the standard, we can do what we like with it, if you don't like it write better code". Less so in recent years as they've realised this was a bad idea or at least a bad look, but in the '00s it was a normal part of the culture.
"You are not allowed to do this, and thus..." reasoning assumes that programmers are language lawyers, which is unreasonable.
bool foo(some_struct* bar) {
if (bar->blah()) {
return true;
}
if (bar == nullptr) {
return false;
}
return true;
}
Instead what happens is that you have something like a constant folding or value constraint pass that computes a set of possible values that a variable can hold at various program points by applying constraints of various options. Then you have a dead code elimination pass that identifies dead branches. This pass doesn't know why the "dest" variable can't hold the NULL value at the branch. It just knows that it can't, so it kills the branch.
Imagine the following code:
int x = abs(get_int());
if (x < 0) {
// do stuff
}
"The abs, labs, and llabs functions compute the absolute value of an integer j. If the result cannot be represented, the behavior is undefined. (242)"
242 The absolute value of the most negative number cannot be represented in two's complement.
Because of 2's complement limitations, abs(INT_MIN) can't actually be represented and it ends up returning INT_MIN.
Replace it with "int x = foo() ? 1 : 2;" if you want.
Surely that value constraint pass must be using reasoning based on UB in order to remove NULL from the set of possible values?
Being able to disable all such reasoning, then comparing the generated code with and without it enabled would be an excellent way to find UB-related bugs.
"These two pointers returned from subsequent calls to malloc cannot alias" is a value constraint that relies on UB. You are going to have a bad time if your compiler can't assume this to be true and comparing two compilations with and without this assumption won't be useful to you as a developer.
There are a handful of cases that people do seem to look at and say "this one smells funny to me", even if we cannot articulate some formal reason why it feels okay for the compiler to build logical conclusions from one assumption and not another. Eliminating null checks that are "dead" because they are dominated by some operation that is illegal if performed on null is the most widely expressed example. Eliminating signed integral bounds checks by assuming that arithmetic operations are non-overflowing is another. Some compilers support explicitly disabling some (but not all) optimizations derived from deductions from these assumptions.
But if you generalize this to all UB you probably won't end up with what you actually want.
NOT-reasonable: silently 'optimize' a 'gotcha' into behavior the programmer(s) didn't intend.
Acceptable UB: Do the exact same type of operation as for defined behavior, even if the result is defined by how the underlying hardware works.
NOT-acceptable UB: Perform some operation OTHER than the same as if it were the valid code path, EXCEPT: Failure to compile or a warning message stating which code has been transformed into what other operation as a result of UB.
If I tell you "open the door", that implies that the door is there. If the door is not there, how would you still open the door?
Concretely, what do you expect this to return:
#include <cstddef>
void sink(ptrdiff_t);
ptrdiff_t source();
int foo() {
int x = 1;
int y;
sink(&y-&x);
*(&y - source()) = 42;
return x;
}
Incidentally I had to launder the offset through sink/source, because GCC has a must-alias oracle to mitigate miscompiling some UB code, so in a way it already caters to you.
Offhand, *sink(&y-&x);* the compiler is not _required_ to lay out variables adjacently. So the computation of the pointers fed to sink does not have to be defined and might not be portable.
It would be permissible for the compiler to refuse to compile that ('line blah, op blah' does not conform the the standard's allowed range of behavior).
It would also be permissible to just allow that operation to happen. It's the difference of two pointer sized units being passed. That's the operation the programmer wrote, that's the operation that will happen. Do not verify bounds or alter behavior because the compiler could calculate that the value happens to be PTRMAX-sizeof(int)+1 (it placed X and Y in reverse of how a naive assumption might assume).
The = 42 line might write to any random address in memory. Again, just compile the code to perform the operation. If that happens to write 42 somewhere in the stack frame that leads to the program corrupting / a segfault that's fine. If the compiler says 'wait that's not a known memory location' or 'that's going to write onto the protected stack!' it can ALSO refuse to compile and say why that code is not valid.
I would expect valid results to be a return of: 42, 1 (possibly with a warning message about undefined operations and the affected lines), OR the program does not compile and there is an error message which says what's wrong.
> Again, just compile the code to perform the operation. If that happens to write 42 somewhere in the stack frame that leads to the program corrupting / a segfault that's fine. If the compiler says 'wait that's not a known memory location' or 'that's going to write onto the protected stack!
As far as the compiler is concerned, source() could return 0 and the line be perfectly defined, so there is no reason to produce an error. In fact as far as the compiler is concerned 0 is the only valid value that source could return, so that line can only be writing to y. As that variable is a local variable that going out of scope, the compiler omits the store. Or you also believe that dead store elimination is wrong?
> possibly with a warning message about undefined operations and the affected lines
There is no definitely undefined operation in my example; there can be UB depending on the behaviour of externally compiled functions, but that's true of almost any C++ statement.
What most people in the "compiler must warn about UB" camp fail to realize, is that 99.99% of the time the complier has no way of realizing some code is likely to cause UB: From the compiler point of view my example is perfectly standard compliant [1], UB comes only from the behaviour of source and sink that are not analysable by the compiler.
[1] technically to be fully conforming the code should cast the pointers to uintptr_t before doing the subtraction.
In fact, the compiler should not eliminate 'dead stores'. That should be a warning (and emit the code) OR an error (do not emit a program).
The compiler should inform the programmer so the PROGRAM can be made correct. Not so it's particular result can be faster.
In C89, there's just a general provision for all standard library functions:
> Each of the following statements applies unless explicitly stated otherwise in the detailed descriptions that follow. If an argument to a function has an invalid value (such as a value outside the domain of the function, or a pointer outside the address space of the program, or a null pointer), the behavior is undefined. [...]
And then there isn't anything on `memcpy` that would explicitly state otherwise. Later versions of the standard explicitly clarified that this requirement applies even to size 0, but at that point it was only a clarification of an existing requirement from the earlier standard.
People like to read a lot more intention into the standard than is reasonable. Lots of it is just historical accident, really.
> memcpy(dest, src, count)
> Copies count bytes from src to dest. [...] Note this is not a plain function, but a special form that applies the constraints dest != NULL and src != NULL to the surrounding scope. Equivalent to:
assume(dest != NULL)
assume(src != NULL)
actual_memcpy(dest, src, count)
I'd love to have a flag "-fno-surprizing-ub" or "-fhighlevel-assembler" combined with the above `assume` function or some other syntax to let me help the compiler, so that I can write C like in the 90s - close to metal but with less surprizes.
Plain functions can apply constraints to the surrounding code:
The problem, which you may realise with some more introspection is that "surprising" is actually a property of you, not of the compiler, so you're asking for mind-reading and that's not one of the options. You want not to experience surprise.
You can of course still get 1990s compilers and you're welcome to them. I cannot promise you won't still feel surprised despite your compiler nostalgia, but I can pretty much guarantee that the 1990s compiler results in slower and buggier software, so that's nice, remember only to charge 1990s rates for the work.
E.g., suppose we patch GCC to preserve any conditional containing the string 'NULL' in it. Would that have a measurable performance impact on Linux/Chromium/Firefox?
But there's nothing like that here.