Constexpr is a Platform (2020) | Hacker News Reader

Constexpr is a Platform (2020)

foonathan.net

101 pointsSindisil53 comments

> we could imagine a future version of C++ where this [constexpr annotation] isn’t required: if we’re calling a function at compile-time, the compiler tries to execute it at compile-time. If it works, good, otherwise, it issues a diagnostic.

This is how D's CTFE (Compile Time Function Execution) works. You can execute at compile time any function, and it will work as long as the source code is available to the compiler, and it doesn't access globals or do pointer monkey business.

You can indeed build a program that runs completely at compiler time. Here's a compile time ray tracer in D circa 2006:

http://h3.gd/posts/ctrace/

Jonathan Blow’s programming language allows almost anything to execute during compilation as well. It’s neat. Here’s an old video where he demonstrated this: https://youtu.be/UTqZNujQOlA

That's true. His implementation also allows compile time function execution to make system calls.

D disallows that in order to prevent malicious source code from being distributed and tricking people into compiling it. There is a mechanism to sent output to stdout, input can be read from a file residing on the compiler's system, and that's it.

It's also why CTFE doesn't allow pointer monkey business, like coercing an integer into a pointer and dereferencing it.

If you're compiling malicious code, haven't you already lost? I don't see how preventing pointer manipulation at compile time (or anything else, really) improves security unless people never run the executable they compiled.

People expect to check source code before running it. They do not expect to have to check it before compiling it.

Malware works by taking advantage of peoples' expectations.

People expect to check source code before running it. They do not expect to have to check it before compiling it.

Why are they compiling untrusted code? To see if it's syntactically correct? If the code you're compiling is using a build system, you're already compromised.

Compilation of untrusted code should always be sandboxed, regardless of compile-time programming capabilities.

Given the way modern IDEs work, unrestricted compile time evaluation would make it dangerous to even open potentially malicious code.

It means that you need to do

    $ sandbox "foobarc malicious.foobar -o executable && ./executable"

instead of

    $ foobarc malicious.foobar -o executable
    $ sandbox ./executable

Which isn't totally intuitive.

Yes, I don't really want to have to recommend to people that they have to run the compiler in a sandbox.

And what if you are compiling more than a single file? What if the project or library you are compiling uses make/cmake/gn/etc?

The point is that restrictions on compile-time programming only protect you in trivial cases. And I can't think of many reasons to fully compile code without the intent of running the executable.

Testing. If you wanted to test that your compile-time code is correct, I'd imagine the typical--if not only--way to do that would be to compile it. Certainly that's true for constexpr in c++.

you could compile it to verify, but you usally compile it using a build system to set include paths right etc. that build system could include a `rm -rf $HOME 2>/dev/null` or similar already.

Testing code requires executing it. This is true for compile-time code or run-time code.

Quite a lot of the tests in the D compiler suite are compile time only. It makes for running the test suite much faster.

To test ImportC, which inherits D's ability to execute functions at compile time, I've been using a lot of _Static_assert() constructs.

https://github.com/dlang/dmd/blob/a5b16bd8e72a447d509cca862e...

If you want a compiler that is designed to enable ransomware injection into your system merely by compiling a file, that's fine with me. But it won't be D.

I don't expect D to change the way it does things. But I think your security concerns lack merit.

It doesn't make sense to produce an executable, not execute it, then go audit the code. Just audit the code in the first place. And like I mentioned earlier, virtually no project is purely static code. They include shell scripts, make files, and tons of other things. Working with untrusted code is dangerous, period. Making compile-time programming a little safer is just passing the buck.

> It doesn't make sense to produce an executable, not execute it, then go audit the code.

If you're auditing the code by examining it with an IDE, it could certainly get compiled.

Many IDEs automatically start the produced executable too.

Again, I don't understand why you'd want to wait until you have an unwanted/untrusted executable to begin auditing code.

Do you audit every repository you clone and build?

In Perl we only discourage IO at compile-time, because this only runs at the build system, not the client. And IO is mostly side effects only, and not restorable. Everything else is free. In perl this is simpler done, everything inside BEGIN{} blocks.

I don't see the value TBH. constexpr is excellent for code readability. Even if the compiler could figure it out I'd vote to keep it. That said, if compiler could figure it out than you can probably set up your editor to annotate it.

As the article points out, an entire function doesn't need to be CTFE compatible, only the particular path taken through the function by the interpreter.

This makes constexpr more of an impediment than an enabler.

Besides, people soon discover the utility of using functions at both compile and run time, and nobody has ever asked for D to require an annotation for it in 15 years.

I don't see how constexpr is helpful for readability. As it stands, constexpr adds a significant amount of syntactic noise. D's approach is the correct one, IMO.

Check Circle for what would be possible in C++, if the compiler would do it for you.

https://www.circle-lang.org/

Virgil initializers are run at compile time and can literally run any code in the program to build up any data structures you want. The result of initialization is the transitive closure of objects reachable from the globals. The compiler goes further and optimizes your code against the initialized heap, devirtualizing, inlining, and constant-folding, throwing away dead code, dead fields, and dead objects. That's great for microcontrollers because you can just allocate your entire heap at compile time and then bake it into your image.

I wrote a paper about it in 2006: https://escholarship.org/uc/item/13r0q4fc

I still work (primarily, in fact) on Virgil. It's great not having any distinction between compile-time initialization and runtime.

And Circle, which is a C++ compiler with an added annotation,@meta.

https://www.circle-lang.org

There are some issues with it, because many ISO members want compile time capabilities constrained by constexpr, instead of the complete language, like Circle compiler allows.

I originally thought this essay would be about the fact that something like constexpr should appear to be the same platform as your target.

This is an interesting property. For example, if I insist on writing down a 64-bit integer in memory and then staring at the individual bytes, the platform determines the layout of that structure. You don't want to know what the layout is for your compiler but for your target.

Floating point maths is often tricky in this world. You expect the constexpr platform to have the same behaviour as your target, but emulating that (assuming you aren't in the trivial case of compiling on the target) can mean some pretty fearsome work that can't be justified. So, maybe the ARM CPU at runtime calculates a very slightly different value for the "constant" internal_damping_factor from the value used in constexpr functions on this x86-64 cross compiler...

However it turns out I once again underestimated how far out C++ wants to go here. Shine on you crazy diamond.

Floating point can be easily handled as long as the compiler says this is the way cte floats are handled irrespective of platform. It only gets harder if you must match the target platform. And even that could be pretty easily done with qemu for most platforms I'd wager.

The analogy is fun, but I'm not sure I agree with the author's conclusion:

> a constexpr marker should be unnecessary, just like a hypothetical linux marker

In fact, I'd love to have a "linux" annotation for functions which can only be called on Linux! Using the type system to manage the context in which a function can be called is a good idea, and allows safer code.

The alternative is global type inference (which is slow) or runtime failures (which defeats the point).

I thought the author's conclusion was that constexpr annotations were necessary?

No, they quite clearly say the opposite, I edited my comment to quote the post.

Interesting, I see that in the conclusion. Seems to me that the conclusion and the body of the article disagree.

I agree. He made a point that, if I understood I agree with, that constexpr is not necessarily as strict as it should be. Specifically that you only need ONE path to work at compile time to be able to be constexpr. I might be missing a reason this is a good idea but at least to me it seems bizarre.

It would 100% be nice if code could be tagged by context in more ways than constexpr so the compiler can better tell you things like "put this behind an ifdef for Linux" etc (maybe with a compiler option to say this will only ever target one context and you don't need to check for that).

In Rust you can tag a function with arbitrary configuration parameters, it's common to write

#[cfg(unix)]

but you could equally write

#[cfg(linux)]

It's just that you're much less likely to need to say this. For example OSStr (borrowed reference to the Operating System specific string type) has #[cfg(unix)] reflecting APIs that make sense for a Unix-style string while #[cfg(windows)] wraps APIs that make sense if your operating system once thought characters are a u16. But the ordinary str and String don't have such APIs, you have to explicitly tell the language "I need this gross non-portable stuff" before you have to start using macros to get to the specific platform dependant elements.

I suspect you could do a similar trick in C++ but it may not be standardised anywhere.

Moreover, Rust is working on portability lints:

https://github.com/rust-lang/rfcs/blob/master/text/1868-port...

So that even when you only compile and test your code on Linux, you will get warnings about calling Linux-only functions from maybe-Windows functions.

For some definition of “working on”. The RFC was accepted several years ago, but nobody has stepped forward to actually implement it.

You can do that in C# : https://docs.microsoft.com/en-us/dotnet/api/system.runtime.v...

(Strictly it's an attribute rather than a type, but it's enforced by the compiler given a target runtime)

With constexpr and not consteval you can really do crazy stuff at compile time. It's super neat and it has eradicated macros everywhere in my code, except in those places where I need conditional compilation.

I really wonder why I'm finding very few uses of constexpr in my code. I feel like I'm missing out on something important but just can't find places where it works for me.

Here's some examples:

1. Declare all your constants as constexpr. They can then be used everywhere in constexpr functions, and every time you invoke a constexpr function with constexpr arguments or literals there's a very high chance it will be executed at runtime. You can make expressions clearer and express stuff with functions, without paying the cost of the invocation. For instance, Intellisense and Clangd often show the value of very complex expressions on hover.

2. `if constexpr` is very effective to execute code conditionally in functions or lambdas with `auto` parameters, you just need to combine it with `std::same_as` or `std::is_same_v`.

This is immensely useful with std::variant and std::visit, for instance.

Yes. Just last week I replaced a std::enable_if that need 3 functions with one function that used 'if constexpr'. I'm sure I'll find more places soon.

This article is excellent ammunition for people who don't think C++ is good for our industry. We need fewer things to reason about while programming, not more "platforms" to reason about concurrently.

Platforms are good

It's on your team culture to enforce good standards of what not to use.

You can just as easily write a clusterfuck in Python (it's a huge language!) as in C++. Python just has a good overall culture of "what is pythonic" that helps this matter.

Python is kind of ironic, it is C++ like in power, yet due to its BASIC like usage, gets a free pass from people not aware of its might.

Those same persons certainly won't be using Java 17, C# 10, F# 5, Python 3.9,....

This is the fundamental observation that the Zig language is built on, and it proves to be an extremely powerful way of understanding your system when taken to its logical extreme.

While the (2-3 paragaphs-in observation) is common between the post and zig, I wouldn't go so far as to call comptime a "platform" in zig, it is a very different class of thing than say arch/os/abi.

This tool embeds a C compiler as a constexpr, so in theory you can do all the work at compile time and just output a static string at run time. But currently I am having trouble getting it to work, so it's perhaps not so glorious other than the amount of time it takes to compile.

ELVM [0] also has a C++14 constexpr target, also using 8cc as the frontend !

[0] https://github.com/shinh/elvm

Reply on news.ycombinator.com