static_assert is all you need (no leaks, no UB) | Hacker News Reader

static_assert is all you need (no leaks, no UB)

twitter.com

98 pointskris-jusiak71 comments

Per standard wording if an evaluation contains an undefined operation then it is not a constant expression. It means that undefined behavior should result in compilation failure if it happens in a context where a constant expression is required, like in static_assert.

However the standard only requires this for language-level undefined behavior. For undefined behavior happening in the standard library it's unspecified whether the expression is a constant expression or not. So no, constexpr tests don't cover all possible UB.

Also even if in theory language-level undefined behavior should be caught in constant expressions, in practice compilers miss a number of undefined behaviors. They are generally good at catching out-of-range indexing, using objects outside of their lifetime, using uninitialized values, signed integer overflow and modifying const objects. However there are a number of subtle undefined behaviors that they don't catch, like unsequenced operations on the same object, invalid values for unscoped enums.

There might be some overlap with runtime tests with -fsanitize=undefined,address. For catching uninitialized values at runtime though you probably need msan, which is a pain to set up, but constexpr tests cover that. On the other hand the function you test might not be available at compile-time.

Anyway, constexpr tests are a valuable tool. It's not a silver bullet.

With C++20 almost anything can be used in constexpr context (vector, unique_ptr, virtual, function, etc.) and as long as it's in the scope it can be tested at compile time which guarantees memory safatey, no UB, etc. Additionally, since constexpr can be executed at run-time and code has been tested at compile-time already therefore 'static_assert' is (almost) all you need - https://godbolt.org/z/P4cqboGx6.

I really don't think it works that way. You can for sure write code that will leak and otherwise be UB but run compile-time just fine.

https://en.cppreference.com/w/cpp/language/constant_expressi...

Point 8:

> A core constant expression is any expression whose evaluation would not evaluate any one of the following:

> 8. an expression whose evaluation leads to any form of core language undefined behavior (including signed integer overflow, division by zero, pointer arithmetic outside array bounds, etc).

This does not really "guarantee" anything. You can still have as much UB (incl leaks) as you want, as long as they are not evaluated at compile-time. i.e. it's at best equivalent to running valgrind.

What would be examples of runtime UBs that wouldn’t be reported when used within a constexpr?

The point is that the code is only checked for UB with the arguments it is given at compile time. There is no guarantee that it can't invoke UB with other arguments it might receive at runtime.

For example, here is a modification of the original program that does invoke UB, but compiles just fine:

https://godbolt.org/z/MaqhcEYqn

Oh yes, I see your point. Thanks for clarifying

Indeed, it fails if you invoke UB. For example, an int overflow in a constexpr causes a compile error:

  <source>:5:17: error: static assertion expression is not an integral constant expression
    static_assert(1024*1024*1024*3 != 0, "UB");

https://godbolt.org/z/Wc591En7E

That only works if you know the values at compile-time though:

  constexpr int foo(int x) {
    return 1024*1024*1024*x;
  }
  
  int main()
  {
    int y;
    std::cin >> y;
    static_assert(foo(1));  //all good
    foo(y);                 //oops, UB if user enters 7
  }

https://godbolt.org/z/K8h4Kj99s

Edit: small correction so that the numbers are big enough to cause problems...

Yeah, the tests will only fail if the tests trigger UB. It's like all testing, it only detects issues if you trigger the issues in the tests. Using static_assert as your test system obviously doesn't obviate the need for writing good tests.

Many people in this thread think that if a constexpr function can be called at compile time successfully, it will also be guaranteed not to have UB at runtime, in general.

I was pointing out that this is only true for the cases you actually test, not the general case.

Even then, it's not fully true, as a function may have different behavior at runtime as opposed to compile time (e.g. because of multi-threading), and so it may display UB even when called with the same arguments that didn't display UB at compile-time.

Overall, this static_assert trick is just a nice way to make sure your tests don't accidentally pass while still invoking UB, to protect from false negatives.

Aha, I think I see. Upon reflection, I do remember that some people seem to think that UB is a property of the code, not the execution; that a piece of code either "contains UB" or does not. I suppose it makes sense then that some people may think that code which works under constexpr can't "contain UB" and get the wrong idea.

Try forcing the call to happen at compile-time.

constexpr requires that the function be callable at compile-time, not that it is so.

You can't call an expression whose value is IO-dependent in a compile-time context, obviously.

The tweet seems to imply that if you can call your functions at compile time, they will not present UB at runtime either. I am trying to point out that that is not the case at all.

That seems unrelated to the sub-thread in question.

The example program is calling foo(y), where the value of y is read from standard input. You suggested to evaluate that expression at compile time, which is not possible. How is this unrelated to the thread?

By forcing the call to happen at compile-time, he would get an error.

The input being potentially known at compile-time is not sufficient to make a constexpr function be called at compile-time.

Yes, but that error would simply complain that a variable that doesn't have a compile time known value is been used in a compile time context.

Conversely, the origin of the thread implies that if a function has been successfully called with some argument from a static_assert, it will not have UB at runtime even if called with some other arguments. This subthread was showing that this is not the case, and that you can't test all uses of a function using static_assert to guarantee that it will not exhibit UB.

In case it's not clear, this piece of code will also fail to compile, even though it exhibits no UB:

  constexpr int foo(int y) {
    return y;
  } 
  int y;
  std::cin>>y;
  static_assert(foo(y));

https://godbolt.org/z/xvhP8nq7z

I hope there’s also some text in the standard prohibiting implementations from allowing any other expressions as a constant expression (which they otherwise could as a language extension), and thus requires compilation failure for such expressions?

Pragmatically, you can't stop extensions; if the fine print for `--std=cool++23` says that this mode is not actually C++23-compliant, nearly nobody will ever notice or care. Pragmatically, if a popular compiler makes `--std=cool++23` the default, and requires `--std=C++23 --iso-eic-jtc1-sc22-wg21 --pompous` to get standard-conforming behaviour, nearly nobody will do that; instead they will complain that other compilers lack the extensions.

Extensions can be standard-compliant, in the sense that they don’t violate any prescription by the standard, and thus a program cannot assume their absence. My question was whether the standard actually takes care to render the acceptance of constexprs-with-UB non-standard-compliant. That is, in addition to “must accept X”, does it also say “must only accept X”?

constexpr has to checked for leaks and UB so as long as there is coverage at compile-time (static_assert + constexpr) I would assume there shouldn't be neither leaks nor UB. But the context is limitted where that can be applied and actually compiles. For example, there is no way to do it with global variables but with limited scope that's possible.

I _think_ you cannot conclude from a constexpr not having UB at compile time that it won’t have UB at runtime.

I guess so, can't think of an example now but I'm pretty sure there are subtle corner cases (as always) and it depends on the testing, coverage and potential limitations of checking things at compile-time, though, IMHO, the technique is promissing and can help with a lot of use cases but defo not everything.

(nevermind)

Successful compilation of UB is explicitly disallowed by the standard in a constexpr context

Thanks, I wasn’t aware. Although that seems to be restricted to core language UB and not include standard-library UB: https://stackoverflow.com/a/72494688/623763

I think it's worth pointing out that your statement is true by definition, perhaps not true by implementation in these cases. It's not like UB produces _random_ behavior, it's just not specified what compilers _should_ do in those cases.

Of course, cannot be relied upon.

The standard explicitly disallows compiling of UB for constexpr.

Otherwise, what you write is provably correct - UB is not statically decidable in all cases (and depending on the type of UB, not even in a lot of cases).

Doesn't it depend on the actual values?

If your compile time evaluations don't trigger signed integer overflow (or any other UB) does it follow that at runtime you couldn't pass a parameter that would trigger signed overflow?

I mean it's still useful because at least you know your test code is not artificially passing because of some UB makes it look like passing

> it's still useful because at least you know your test code is not artificially passing because of some UB makes it look like passing

Right, that's the extent of what this does. When I saw it on r/cpp I thought OK, somebody realised now they can make their C++ tests work as a reasonable person would expect, or perhaps realised that without this C++ tests are almost worthless because they can invoke Undefined Behaviour silently.

But increasingly I suspect the OP mistook this for a breakthrough in correctness which it isn't, otherwise why post it to HN?

On the other hand, the prohibition on UB for constexpr doesn't reach up to where IFNDR lives, so I'd guess most non-trivial C++ software is technically nonsense with no defined meaning as a result of IFNDR regardless of how many or few unit tests were written or whether they use constexpr to prohibit Undefined Behaviour. A cheerful thought.

[Ill-Formed, No Diagnostic Required: A recurring statement in the C++ ISO document which basically says if you did this then too bad, that's not a well-formed C++ program, however your compiler may not notice that this isn't a C++ program, so, your program might compile, and even execute, but what if anything happens when you run it isn't specified in this ISO standard, good luck.]

No, it doesn't depend on the values. Yes, it does follow. The compiler is not allowed to compile constexpr code that could produce UB at runtime. period. :)

That is, conditional constepxr code that depends on values and could produce UB is not valid constexpr code, and a compiler is not supposed to compile it.

This is very explicit in the standard.

Think of it as a statically decidable set of code.

Now, it wouldn't shock me if compilers don't achieve this right now, but the standard is clear that constexpr code may not contain operations that could produce undefined behavior at runtime.

That is a complete misunderstanding.

A constexpr function can very well take an input, and it can invoke UB based on the runtime value of that input.

What the standard prohibits is compile-time evaluation of an expression which invokes UB. So, if you actually call your function at compile-time with a constexpr value that ends up invoking UB in the function (say, an integer overflow), THEN the standard mandates that the compiler throw an error rather than compiling some random value in.

For example:

  constexpr void foo(int x) {
    std::cout << 1024 * 1024 * 1024 * x;
  }

  int main() {
    static_assert(foo(100)); // will fail because computing 1024 * 1024 * 100 is signed integer overflow, which is UB
    foo(100); // invokes UB at runtime; in practice, will perhaps print some overflowed value
  }

Edit: I should also add that you can very well invoke UB in a constexpr expression if it is standard library UB and not core language UB (e.g. if you try to pop() from an empty std::vector).

"A constexpr function can very well take an input, and it can invoke UB based on the runtime value of that input. "

No, it cannot. It is not constant if it does that. As such, it cannot be used in in any context that requires a constant expression. It explicitly says the operations in a constexpr may not produce undefined behavior.

That isn't "may not produce undefined behavior except if you pass the wrong values at runtime or don't evaluate it". It says: "An expression e is a core constant expression unless the evaluation of e, following the rules of the abstract machine (6.8.1), would evaluate one of the following expressions:

…

an operation that would have undefined behavior "

In your case, the evaluation would evaluate (at runtime) an operation that would have undefined behavior.

It is true that where you do not require a constant expression, it does not require it be constexpr at all, but the question is whether an expression that produces undefined behavior is constexpr is "no".

The standard even clarified this to say that foo is simply considered non-constant in your example (IE it's not constexpr ).

IE see defect report 695

"The consensus of the CWG was that an expression like 1/0 should simply be considered non-constant; any diagnostic would result from the use of the expression in a context requiring a constant expression. "

In your example, foo is non-constant as used. It is not constexpr.

I suspect you are using "constexpr code/function" in a strange way. I believe most people use it to refer to code that is constexpr-qualified, such as "constexpr void foo() {...}". You seem to be using to refer to code that is a core constant expression, and that is where (most of) the misunderstanding comes from.

Still, it should be noted that even code that looks identical to a core constant expression can display certain kinds of UB if invoked at runtime, per the standard. For example, if the code is using certain STL classes or threading constructs in a way that is UB, it can still be considered a core constant expression. Additionally, the code may be well defined when used in a single-threaded context, but become UB when used in multi-threaded contexts, say if two threads are accessing an object member concurrently without the right barriers.

Here is an example on godbolt. Notice that bar() and baz() are defined the same way in the static_assert block and the runtime block, yet executing them constitutes UB in one case but not the other.

https://godbolt.org/z/3q5sTGe4j

When I say "a constexpr function", I mean "a function that is legally constexpr-qualified". An expression calling such a function with a constant argument is a core constant expression, as long as the evaluation of the function given that argument doesn't produce UB or leak memory.

But, this tells us nothing about whether calling the same function with some other argument would produce UB or not. Unless we try to form a core constant expression containing a call to that function for every possible argument value, we can't be sure if the function is free of UB for every possible input value.

My point, more simply, is: the fact that foo(1) is verified by the compiler to be a core constant expression is not a guarantee that foo(2) would be a core constant expression as well. Your original comment seems to me to imply that it does.

The consequence of this is that, as said elsewhere, this trick only helps to ensure that a unit test is not accidentally passing because of UB (and that memory is not leaked for that input value). It doesn't help provide any other guarantees.

You're both right, but you're talking about different things.

If you use the same values at run time as you use in the tests, the tests predict the behavior as desired.

If you use different values at run time as you use in the tests, the tests do not predict the behavior as desired.

> You're both right, but you're talking about different things.

The GP post is explicitly claiming that the values don't matter, and that per the standard a constexpr function should be guaranteed by the compiler to be incapable of producing UB (or else compilation should fail and it should not be allowed to be declared constexpr). This is simply false. Here is the GP statement:

>> That is, conditional constepxr code that depends on values and could produce UB is not valid constexpr code, and a compiler is not supposed to compile it. [emphasis mine]

> If you use the same values at run time as you use in the tests, the tests predict the behavior as desired.

Even this is not fully guaranteed by the standard, as core constant expressions must only be guaranteed not to invoke core language UB. They may still be considered core constant expressions and evaluated at compile time even if they exhibit standard library UB (e.g. if they are not respecting some preconditions of an std::vector method).

"The GP post is explicitly claiming that the values don't matter, and that per the standard a constexpr function should be guaranteed by the compiler to be incapable of producing UB (or else compilation should fail and it should not be allowed to be declared constexpr).

They are so guaranteed - as i cited to you, your example of foo, as used in foo(100), is considered not a constant. There is no such thing as a constexpr expression that can produce UB.

If it can produce UB, it is not constexpr.

That does not mean it will not be validly usable in non-constant contexts! But it is not constexpr.

But foo(1) is a constant expression, and the compiler accepts it.

Not sure what you mean by a "constexpr expression" exactly. My point is that a function can legally be constexpr-qualified even if it can produce UB when called with certain arguments. I think it's very common to call such a function "a constexpr function". Of course, we agree that not every expression where a constexpr function is called is a constant expression (such as foo(100) in my example). However, expressions where that function is called with arguments where it does not produce UB are legal constant expressions (such as foo(1) in my example).

Perhaps I misunderstood your original comment and we are in fact in violent agreement. If so I apologize.

Unless you used std::is_constant_evaluated() and the code running at runtime was not the one tested at compile time

At the very minimum, you can simply do is_constant_evaluated to change behavior depending on runtime vs compile-time...

>constexpr has to checked for leaks and UB so as long as there is coverage at compile-time

Source ?I am not sure constexpr give any garanty regarding UB and/or leaks

Constexpr at compile times gives that guarantee, not constexpr in general. Hence static assert to force comptime evaluation

> Constexpr at compile times gives that guarantee

Can you point to a source for that ? I am not trying to be pendantic, but genuilly curious

From an overall performance point of view, I wonder how the timing works out for the compiler to run your unit tests like this vs to produce and invoke a binary.

I bet it's mostly a wash, and the ergonomics of conventional gtest macros look way better to my eye.

The idea to let the compiler run the tests only works if you constexpr everything, which means putting all code in headers. This effectively means giving up on separate compilation. Worse, if you use some mixture (most code in headers but you still have >1 compilation unit), your compile times completely explode as essentially all code is compiled repeatedly for each unit.

There are other ways to do this. I made a proof of concept of using linker sections to allow you to sprinkle tests within the implementation inline once... https://github.com/cozzyd/examc (this is obviously not production-ready, just serves as a proof of concept).

Basically the idea is that the test code gets written to a different linker section that your test runner can iterate through, when tests are enabled. This is easy on gcc because it generates automatic constants for the beginning and end of different linker sections. There may be away to do this with clang as well, but I never use clang.

Pragmatically you can write your code in such a way that you can get immediate feedback in your IDE as you write the code if a static assert fails as you are implementing your function.

You could of course set up your runtime tests in a similar way, having the ide run them back to back as you are writing code, but it is more complicated, especially if the code is in an intermediate state that it is not fully compilable.

So in the end it is not a huge breakthrough, but having compile time tests is still quite a nice feature.

Hmm, there are obvisouly trade offs (it depends on the compiler how many tests, how are they written, etc.) but for apples to apples comparision the gtest binary would have to be either compiled with sanitizers (that would be probably slower to compile than static_assert tests without sanitizers) or run with valgrind or similar (execution would be much slower, static_asserts tets don't have to be executed, compiles=green).

Depends on how you define a class. E.g. you are not using std::list in your example.

Have zero cost abstractions gone too far?

(Offtopic but) Kris, I have used your micro unit-test library in the past and it was a pleasure to look at your code. You're the kind of crazy guy (in a good way) that gives me the motivation to learn new stuff. Thanks.

This makes sense for straightforward tests, but static_assert is not all you need in general, because some things has be executed in runtime, after series of steps or after some timeframe. Good luck reproducing or testing these in compile-time.

Could you not put a series of steps or something that mocks time into constexpr? A comment mentioned above that in C++20, almost all features are available at compile time now.

With few limitations, yes. In C++20, you could for example test for constant evaluation[1], using that as a mechanism to fall back to a real clock during runtime.

[1]: https://en.cppreference.com/w/cpp/types/is_constant_evaluate...

What's going on in this code for someone completely alien to C++?

The C++ code is a simple list type, and also a bunch of tests for that type to confirm that it works as intended. The crucial trick here is that because static asserts are used, the test values are computed during compilation, such computations are forbidden (by the standard) from having any leaks or Undefined Behaviour. Anything allocated must be freed by the time the tests complete, and no language Undefined Behaviour is permitted.

The latter is pretty normal for other languages but is a big deal in C++ where UB is a constant plague. However many languages have either forbid or have strict limits on compile time heap allocation - after all that heap isn't going to still exist at runtime. Requiring that you free everything allocated fixes that hole and means you get free leak detection.

The compiler is doing a bunch of complicated stuff at compile time. In fact it's both a C++ compiler and a C++ interpreter.

static_assert is a compile-time check.

[] { ... }(); is an immediately executed lambda function (IIFE in javascript parlance).

list<int> list{} is a linked list of integers (double-linked, forward and backward). push_back() allocates more memory. pop() / clean() deallocates memory.

It is creating a bunch of objects, modifying them, then asserting their value all at compile time. For example the first example creates a list and asserts its size is 0. List normally allocated on heap, so I am guessing they have made changes in thay area in c++20 by making it Constexpr, which is a fancy way to say an expression can be known at compile time.

This is awesome!

I wonder what the compile time cost would be to just have this kind of unit testing in all the time.

Thanks! Most likely not yet applicable at Google's scale but smaller project can defo leverage the approach. Personally, I'm writing most of my tests this way and with TDD the red phase is always a compilation fail which is quicker than buiding and running in my experience. But that's for a medium size project. But as always it depends there are trade offs.

I’ve been using this technique as well, but I found that debugging static_asserts is quite hard. I often fall back to calling the failing test at runtime and stepping through. Any suggestions for a different workflow?

IMHO the best approach is to avoid the problem by applying TDD. Then there is very little need to debug anything. But otherwise, there is https://github.com/mikael-s-persson/templight for compile-time debugging which is pretty cool and having something like `expect(auto... args) static_asert(args...); assert(args...);` may help with being able to debug at run-time and get the coverage (though, the code has has to compile aka pass first).

You need C++ 23 to run this code. I tried it out with C++ 20 and got compile errors.

Hence "-std=c++2b" on compiler explorer example.

New user of C++ discovers C++ has a mechanism for code to be evaluated at compile-time, and decides to share his findings on twitter.

Why is it a story? Slow day on the Internet?

Someone found ot interesting enough to write about this and someone else found it interesting enough to upvote.

In the end it just an excuse to have a discussion on an interesting topic.

And yes, it is a slow day.

Reply on news.ycombinator.com