The C++ Iceberg
fouronnes.github.io
fouronnes.github.io
We did a C preprocessor meta programming iceberg a while back: https://jadlevesque.github.io/PPMP-Iceberg/
The behavior is undefined
But also: turn warnings to 11, use UBSAN, ASAN, TSAN and static analysers. These are all easy steps to make C and C++ code a lot more robust.
Thinking that you know what `undefined` really is results in undefined behavior.
Every UB could verry well be wrapped in error checking and we could have 0 UB out there. We don't do that, but that's because of (our) choice, not some superficial entity that poses machine when running into UB.
Not in C and C++. How would you check that a pointer is safe to dereference for example?
In high level languages it gets a lot more complicated (look up "pointer provenance").
For instance from skimming the list:
- An object is referred to outside of its lifetime (6.2.4).
- The value of a pointer to an object whose lifetime has ended is used (6.2.4).
- An array subscript is out of range, even if an object is apparently accessible with the given subscript (as in the lvalue expression a[1][7] given the declaration int a[4][5]) (6.5.6).
(6.5.7) Additive operators
> If the pointer operand and the result do not point to elements of the same array object or one past the last element of the array object, the behavior is undefined.
Interestingly a[1][5] is also UB, but for the dereferencing a past-the-end pointer, not for the arithmetic.
Also, dereferencing a pointer with the wrong dynamic type.
> all ca 200 UBs in my head
Technically there is no enumerated set of all possible UB. Anything not explicitly defined in the standard is UB.
/extremely-pedantic
It's a bit like that "Migrate off obsolete DB server" card that's been sat in your team's (well most teams) backlog for a couple of years. Everybody agrees it should be done... But like not this sprint.
My assessment is that at their current rate WG21 is adding new UB to the language slightly faster than it's being documented for the appendix, so this won't actually finish.
> The unary * operator denotes indirection. If the operand points to a function, the result is a function designator; if it points to an object, the result is an lvalue designating the object. If the operand has type "pointer to type", the result has type "type". If an invalid value has been assigned to the pointer, the behavior of the unary * operator is undefined.
below in a note (emphasis mine):
> Among the invalid values for dereferencing a pointer by the unary * operator are a null pointer, an address inappropriately aligned for the type of object pointed to, and the address of an object after the end of its lifetime.
Now the note is not normative, but I assume there are normative wording for defining what the "invalid" pointer values are, scattered around in the standard.
C++:
https://eel.is/c++draft/expr.unary.op#1.sentence-4
C++ is very particular in what it means for a pointer to point to an object, and this is also UB there.
Compilers have been known to remove exactly that sort of error checking code, as there was undefined behavior present.
It's a mess.
What I meant is “we (as in) the c/c++ community (or rather the specification committee)” could tomorrow decide that, lets say, all null pointer dereferences should lead to program exit. Existence of UB is a choice.
There are many cases of UB that would be cheap to check, but there are many more that are incredibly expensive to check.
In older compilers, UB behaviour was a lot more predictable because the code transformations done by optimizing compilers were by far not as complex as today.
In the C standard, 'undefined behaviour' is basically a catch-all phrase for a lot of different things: from badly designed stdlib function that miss obvious security checks (like gets()) to the compiler generating non-sensical code.
The latter is what's much more dangerous. It's trivial to just not use the C stdlib functions, but code generation problems caused by UB are much harder to catch.
Compared to such 'obvious' memory corruption problems caused by invalid pointers or missing range checks, obscure code generation issues caused by UB are much harder to identify and the behaviour may also change randomly between compiler versions - this is the actually scary thing about UB, and it's also a relatively "new" thing.
The behavior is undefined.
One could also argue that most UB in C/C++ should actually be IB, but that's a different discussion.
Rust has UB if you don't obey the rules (Safe Rust doesn't because the rules are not your responsibility in Safe Rust, the unsafe superpowers allow you to do things the enforcement can't check but now all rules are your responsibility) but it's not somehow magically "more sane" than C++ it's ultimately lowering to the same LLVM IR that Clang uses. It's tolerable because you're only confronting the risk when you explicitly choose to do so. If you don't need to squeeze that last few drops of performance you can leave it. If you don't have the confidence, if your experts are too busy to review it, if you haven't had a proper night's sleep, you can write safe Rust today.
We live in parallel universes that must have intersected in this thread.
The only barrier is to actually know about and use those tools in the development workflow.
Rust is still superior of course (for memory safety at least), because it actually prevents most of those issues in the first place, but it's not like the rest of the world has been twiddling thumbs in the meantime.
Fuzz testing helps, but it's not commonly used in my experience.
This turns out not to be the case.
There actually is text in the current standard as to what acceptable behavior is when encountering undefined behavior.
This was turned into a "note" in later versions of the standard, but initially it was just as much a normative part of the standard as everything else.
The text is still there.
Compiler writers want to exploit UB, hence they say that this text effectively doesn't exist.
https://blog.metaobject.com/2018/07/a-one-word-change-to-c-s...
void f() {
char x[100];
gets(x);
}Most programmers (and pretty much all compiler writers) interpret it as meaning that the compiler can assume it can't happen (ignore the possibility) and translate accordingly.
Some, like mpweiher interpret it as meaning that the compiler must implement a trivial (but somehow unspecified) translation to the underlying hardware and let the behavior be what the hardware provides.
edit: We went through this many many times.
Right now, notionally serious people are making the case that C++ should enshrine the "bag-of-bits" model of pointers which is exactly this sort of stupidity. At first it looks like they want pointers are addresses which is kinda dumb but whatever, and you keep reading and they're like aha, also I should be able to dereference my bag-of-bits and have that work too... Full blown Provenance Via Integers, expect to spend the next forty years trying to specify how any of the optimisations you rely on can survive the resulting tangle, say hi to anybody who wanted Consume ordering and is still waiting.
That belief is endemic because that is the way things actually worked for many years! Compilers used to give us the obvious behavior, because they weren't smart enough to outfox us by doing anything else, because the CPUs we used were not fast enough to let them do the kinds of comprehensive optimization analyses which are now routine.
Each one of these optimizations have broken someone's code.
It is very popular in a small range of very influential companies.
As much as I'm not a fan of the status quo, the argument that changing "Possible" back to "Permissible" isn't particularly solid; the rest of the paragraph makes it clear it's not an exhaustive list of behaviours, merely examples from a "range".
The "range" clearly indicates that you can do something that's semantically in-between the options provided.
What current compilers do is nowhere near that range.
For example, C++23 got std::unreachable. From my dips into Rust, I expected something similar to std::abort, terminating the program in some sane and possibly helpful way.
However, in C++, std::unreachable just invokes undefined behaviour when called. It's not usable as a guard from programming errors, it's just an optimization hint. You still have to write the guard yourself.
I'm left wondering about the use-cases for this.
It's a double edged sword of course. If control flow actually hits the default branch, then all bets are off (because that means the code will access an out-of-range jump table slot and jump somewhere into the wilderness).
AFAIK compilers are free to perform something like Rust's panic when std::unreachable is actually hit, but that makes only sense in debug mode. Because in the above example, the compiler would need to add a range check to figure out if panic needs to be called and that would completely defeat the idea of removing that range check in the first place.
This also allows expressing intent: no default label means that I meant to handle all cases, and having a default means that I opted into a fallback, please don't warn. That's probably why -Wswitch-enum isn't enabled by default, too many false positives without a convenient way to suppress the warning.
if ((val < min) || (val >= max)) {
__builtin_unreachable();
}
switch (val) {
...
}
switch (e) {
case A:
foo(); break;
case B:
bar(); break;
}
std::unreachable();
switch (e) {
case A:
foo(); break;
case B:
bar(); break;
default:
std::unreachable();
}
edit:
duh, it's supposed to be returns here instead of breaks.
I guess they stole this idea from Terry Davis who implemented this in holyC. When you use square brackets it just doesn't do any range check. Terry, as always, been ahead of the times. Like so:
switch[abc] { [...] }
What happens is: Compilers sees unreachable and assumes that this codepath will never run, so it just yeet it out of existence. For example, maybe it will remove the preceding if-statement and rather then check its condition, just insert true or false. Afterall, why waste computing on checking condition if we know it will never resolve to the unreachable path.
In Rust, the same if-statement won't be optimized away, and if the code happens to go for the unreachable path, it will call panic.
Now, the example is a simplification, but it's just to demonstrate that UB comes from compilers (verry aggressively) assuming UB will never happen, and not because someone decided to "add it".
Rust does also have the safe macro core::unreachable which will panic if reached, but while the compiler can assume this probably won't happen and optimise accordingly, it usually can't know that it won't happen (and if it can your decision to add the macro may be unwise)
One reason C++ gets more and more Undefined Behaviour is consistency. The committee will, unless prompted hard not to, draw analogies to existing UB in the language and use that to justify making your new thing Undefined in the tricky cases. The results are pretty embarrassing.
What I don't get is, why are we still adding more undefined behaviour to the standard?
You can define undefined behavior if you wish. Make your own unreachable that prints an error and aborts gracefully when reached in debug builds, and calls std::unreachable in release builds.
Why do you believe this is some kind of problem? Can you explain in concrete terms what issue you have with undefined behavior?
> However, in C++, std::unreachable just invokes undefined behaviour when called.
It does, by design and as a testament to great design choices.
> It's not usable as a guard from programming errors, it's just an optimization hint. You still have to write the guard yourself.
I don't understand what point you tried to make. Nothing in your comment has any relation with undefined behavior. Instead, you're complaining that in your personal opinion different languages have similar-sounding features that work differently.
Oddly, you should really read up on std::unreachable, because one of the reasons explicitly stated in it's references is to "trap them to prevent further execution".
To explain what this means in no ambiguous terms, the standard ensures that a) std::unreachable is valid C++ and made available to the world to have the semantics to specify that a code path is unreachable, and b) allow everyone to handle that as they see fit by flipping flags in the build system.
Think about it for a second. You want to call std::abort when std::unreachable is hit. Great, go with that. I don't, and instead I want the compiler to optimize away that code path and anything it would touch, but on debug builds I want it to output a warning and trigger a breakpoint. Have you noticed that all these cases introduce behavior that's entirely different, arbitrary, and implementation-defined? How do you, as a standards committee, get to cover all possible use cases?
By specifying it as undefined behavior.
No. If you wanted implementation-defined behavior, you'd specify it as implementation defined.
Consider that abs() of signed integers has UB! Because the signed integers aren't balanced it's tricky to decide what should happen if we abs() the most negative value of a particular size of signed integer. In a language like Rust you get to specify what you meant here, if you don't then you get a panic when it happens. But C++ instead gives this... Undefined Behaviour!
Or take division by zero. It's easy enough to say that you think the signed integers should have wrapping arithmetic. Doesn't make much sense, but hey, it's defined at least. However, even if you define overflow as wrapping, division by zero is a different problem. In Rust they can explicitly say this can't happen using NonZero types (e.g. NonZeroI16 is a 16-bit signed integer, but it can't be zero) and get the high performance machine code, but in C++ to get that same code they just say division by zero is Undefined Behaviour.
If you want to do actual arithmetic, you have to get the internal i16, perform the arithmetic, then rewrap it.
There are of course some functions where it would have been nice to allow passing in an invalid value and the function just returning its normal error or "no" return value. E.g. isalnum might be nicer if it didn't crash for input > 255. (But perhaps it's not possible in all implementations without adding some other constraints that didn't exist before.)
I think people tend to parrot undefined behavior as if it's some kind of gotcha when in practice this only means two things:
* Don't write code that you didn't bothered to understand what it does,
* You bothered to check what your implementation does and decided to write implementation-specific code for your own personal reasons.
By definition, each and every single programming language in existence which is not specified in an official reference document is comprised exclusively of undefined behavior. Why? Because none of it's expected behavior is defined. That's what these nonsensical discussions boil down to.
So why is this somehow expected to be such an insurmountable gotcha with C++ although the rest of the world uses languages which are entirely undefined without any concern?
Boggles the mind.
No, not really. In fact an excellent counter example can be found on this very iceberg: https://blog.regehr.org/archives/161
C++ at the some optimisation levels says the example program disproved Fermat's Last Theorem, which should come as a surprise as their are no known counter examples. The program is also valid C. When compiled with good C compilers at all optimisation levels (both gnu and clang count as good), the program never exits because it doesn't find a counter example. But the gnu and clang c++ compilers do behave as described. The difference is caused by what the C and C++ language standards consider to be UB.
Why? C++ (but not C) defines an infinite loop as undefined behaviour, and at high enough optimisation levels the compiler spots the infinite loop (which is the correct behaviour if the theorem is true) and goes with the other option. (As an aside, it's impressive it does detect this particular infinite loop.)
Why is that a problem? Well lots of programs are deliberate infinite loops. In practice many don't qualify as UB because of various conditions listed in the article, but you would have to be a language lawyer to know them. The author gives real life examples of how he was caught by the surprise removal of his loop.
Deliberate infinite loops are actually fairly common and are perfect correct code. Proving they are infinite is famously undecidable, so you can usually get away with it safe in the knowledge the compiler won't spot it. But compilers keep chipping away at the edge cases so one day you can find your program changes its behaviour drastically just because it was compiled with a new version of the compiler, or even just different optimisation options.
I'm not sure you read the blog post you cited, because it states exactly this:
> In C, overflowing a signed integer has undefined behavior, and a program that does this has no meaning at all. It is ill-formed.
I think this is quite clear. This is exactly what I stated in my first example. I'm not sure why you missed that. It's the whole point of undefined behavior, and what people to confuse what it means.
Then there's the nuance that those who mindlessly parrot undefined behavior as some kind of gotcha clearly fail to understand, which is the fact that implementations (meaning, compiler vendors) are given free reign to implement whatever implementation-specific behavior they see fit. Why? Because the standard left tha behavior undefined, which also means the standard does not prevent implementations from defining their own behavior?
Do you understand the whole purpose of this? The blog post you cited clearly shows examples of C programs what use data types that are system- and implementation-specific, and thus overflow is left undefined because, just like the integer types, its behavior can and does differ among systems. It would be absurd to specify overflows should wrap around/saturate values/stay MAX/throw exceptions/terminate program because systems implemebt this differently. The role of a programming language is to write programs that machines run, and thus the programs need to express wha each machine does, nit what some other machine does.
I recommend you re-read what I wrote in my post to see the whole point of undefined behavior and its critical importance. It is high time that this misconception is put to rest.
Uhmmm, did you read what I said? I ask because it looks like a red haze descend after descended after you read the first few words. I've been writing C for 40 years. I know why C made integer overflows UB. It seemed perfectly sensible to me. That's why I didn't mention it. I'm not sure why you brought it up. In the article it says this:
Third, there are no integer overflow games going on here, as long as the code is compiled for a platform where an int is at least 32 bits. This is easy to see by inspecting the program. The termination problems are totally unrelated to integer overflow.
In other words, from the point of view of the language semantics, a loop of this form is no better than an out-of-bounds array access or use-after-free of a heap cell. This somewhat amazing fact ...
I think the way C and C++ handles UB belongs to a bygone era. In their defence when I was writing C 40 years ago there was no choice. Checking if every integer operation overflowed imposed an unacceptable performance penality. While today in languages like Rust compilers statically check for UB at compile time, back then C compilers were barely more than overblown assemblers. UB simply meant "you get what the hardware gives you". Notice that means on a given arch there was no real undefined behaviour. I knew perfectly well what would happen on x86 if an integer overflowed, I know whether a char was signed and I wrote my programs accordingly, sometimes deliberately exploiting what I knew would happen. UB only bit you hard when you ported your program.
What happened is compiler writers seeking to squeeze every last bit of performance pushed "undefined" to mean "since it's undefined, I can do whatever I damned well please", and with C++ and infinite loops they've managed to push that beyond absurdity. Meanwhile other languages have gone in opposite direction. Rather than giving programmers more rope to hang themselves with they've got runtime and compile time checks that either outright band undefined behaviour, or warn you about it.
The most glaring error in the iceberg IMO is the claim that shared_ptr is an antipattern.
Shared_ptr is brilliant for immutable data in parallel programs. It’s not an antipattern. It’s useless for sure as a building block for data structures and other linked data with potential for circular references.
But really, if a resource is a) immutable and b) needed in a shared context then a thin wrapper around a map of shared pointers or such saves the pain of needing to implement a much more complex resource management scheme.
I’m a huge gain of stlab::copy_on_write<T>, which is fundamentally very similar but which is value-semantic, doesn’t let you make loops, and gives You local reasoning.
I personally found out the domain was named after the creator when I saw a CppCon talk about the explorer from him!
In D, CTFE (Compile Time Function Evaluation) is guaranteed to run at compile time. CTFE happens whenever there's a constant-expression in the grammar, such as the initialization of an enum:
enum A = func(3);
Also, constexpr on a variable is not a hint - it must be evaluated at compile time, where-as a constexpr function will be evaluated at either compile time or runtime depending on the context and arguments passed to it. For a function that you either want to work at compile time or not at all (compile error), consteval is the hardened version.
Yes, sadly this means "constexpr" variables and "consteval" functions must be calculated at compile time, while "constexpr" functions can be used both at compile time and runtime. An annoying distinction.
Simply no need for tagging them one way or another.
You are expected to watch most of the video, before learning that the purpose of the video is to argue that in almost all cases one should use "static constexpr" and not just "constexpr".
The argumentation can be interesting, but it would have been nicer to just state the conclusion from the beginning, allowing a decision of whether it is desirable to spend time to also know the reasons for this.
People are just confusing it with the inlining optimization that a compiler might do.
Marking the symbol as weak is what inline indeed does from a technical point of view.
I’m actually also not familiar with weak, but a quick glance suggests that it might not mean the same thing as inline. Can weak symbols be defined multiple times? EDIT: yes, according to an SO post compilers often use weak to implement inline.
Beginners fail to understand for example that providing a class member function definition within the class definition makes it inline, but that's already what you're doing textually, and that's what the keyword means.
It's only needed for functions defined in header files where multiple definitions may exist after compiling object files.
Member and template functions are special cased to be implicitly inline.
Non-inline definitions need to be in their own translation unit.
struct X { inline void f(); };