Just how constexpr is C++20’s std:string?
quuxplusone.github.io
quuxplusone.github.io
This is not a bug in MSVC, rather it is due to MSVC implementing std::string SSO differently than gcc or clang. Instead of initializing `data` as pointing to the internal buffer for small strings, MSVC uses the string's capacity to determine whether to access `data` or the internal storage [2].
Hence this code compiles, as it's not initializing the string using a stack address. [3]
[1]: https://godbolt.org/z/1ErrKjdbq
[2]: https://devblogs.microsoft.com/oldnewthing/20230803-00/?p=10...
I spent a while porting a codebase that was written by someone far more clever then smart. They had discovered that you could clear a string under MSVC by `memset()`ing over it.
Obviously, this exploded in linux. That was a fun bug to track down.
It sets the capacity to zero. MSVC then interprets it as a short string with length 0, which doesn't have an external allocation.
> Wouldn't it leak memory?
Yes, but I assume the "far more clever then smart" programmer didn't notice.
On MSVC, an all-zero string object is the correct representation for a std::string containing "" (and the memory leak is easy to miss if you don't do it too much).
On both Clang and gcc, an all-zero string object is not using the short-string optimization -- it is a string with size zero, capacity zero, and an external buffer of nullptr. (On clang this happens because capacity is even, and on gcc because &str->buf != &str). Any code that attempts to access its null terminator will dereference the null pointer and crash immediately.
Why was that? Was it too problematic to just call std::string::clear ?
It's a somewhat common pattern in C, but requires care.
C++ isn't C, though.
If I recall correctly, from C++20 onward all member functions of std::string are constexpr.
You are pretty much limited to macros and macro-like languages (llvm-tablegen) if you want to do this today.
Users, library writers, and compiler writers all benefit from this in some way.
Also, constexpr symbols can be demoted to regular const as needed by the compiler if necessary, such as when getting a pointer to one. constexpr doesn't mean "compile-time only", it means "compile-time compatible"
That assumes your API accepts string views. How do you pass a string_view (safely) to fopen, or CreateFile (on windows)?
Then again, I am primarily a Python and C programmer, so there is likely some underlying reasoning I'm missing.
in C++ cannot change after construction, but is not compile time constant. This means that it has to run some code and allocate some memory at runtime.
In contrast constexpr is fully compile time and ends up in the read only portion of the binary. No code code execution or allocations necessary.
That said, here's an example:
// imagine this with string_view
FILE* openFileInPath(const std::string& path) {
if (path.starts_with("some_dir"))
{
return fopen(path.c_str());
}
return nullptr;
}
I should be able to pass a constexpr string to that function the same way I can pass a runtime created string.
I think of constexpr functions as something which occur at compile time, and I thought this linked-to piece was about what can take place inside those functions.
As I understand it, you want to create a constexpr string so it can be passed to a char*. The issue is this requires heap allocation (for large enough strings, based on the compiler implementation).
The C programmer in me wonders why you don't "char[]" allocate the static memory buffer, then pass around a string view, so the constexpr doesn't need heap at all.
There's a few reasons. Avoiding the heap allocation is one, yep. There just shouldn't be a difference between a compile time string and a runtime string. As a programmer I don't want to have to deal with "which flavor of string am I dealing with".
The issue with passing to a c API (I used fopen, but think of any c library - zlib/gzip, openssl, sqlite) is that the string needs to be null terminated, and string view doesn't guarantee null termination.
int foo(std::string_view view) {
return strlen(view.data());
}
std::string str("hello world");
foo(str):
std::string_view view = str;
foo(view);
view.remove_suffix(6); // now view is just "hello" with no null termination
foo(view); // this is wrong.
In the 1990s there was only one string type - NUL-terminated characters. ;)
I keep reading about all the neat algorithms (and multi-threading) support in modern C++, but the complexity of the decades of history and the difference in mindset between {C, Python} and C++ have made me shy away.
I think I'll keep mum from a distance in future discussions until I decided to (re) take up the language seriously.
Indeed. If I could wave a magic wand, I would add a fat pointer to the language that we use to represent strings and arrays, and that we can automatically convert to/from const char* at compile time. This would replace string_view and span. All of the functionality of std::string would be available to it, and a call to strlen, or strcpy on it would work via the length, rather than the null terminator.
But alas, no magic wand.
For const std::string&, you're further incurring a copy in order to create a temporary std::string.
So you might need to deal with a string that looks like "foo\0bar\0\0".
If there was reflection, it would be useful to talk about class names, function names, etc, at compile time.
template<class E> requires std::is_enum_v<E> constexpr std::string_view joinedEnum() {
constexpr auto generate = [] -> std::string {
return []<template<class...> class L, class... D>(L<D...>) {
std::string str;
(((str += (str.empty() ? "" : ", ")) += D::name), ...);
return str;
}(boost::describe::describe_enumerators<E>());
};
static constexpr auto arr = [&] {
constexpr std::size_t N = generate().size();
std::array<char, N> arr;
std::char_traits<char>::copy(arr.data(), generate().data(), N);
return arr;
}();
return std::string_view(arr);
}
Putting together strings through string interpolation involving somewhat expensive operations is a common usecase. Given the choice, it's preferable to not have to compute them each and every single time a function is invoked.
(const char)*
const (char*)?
char* const
const char* const f() const
const char * char * const
That rule of thumb doesn't work though for
char const * char * const *
Oof no I HATE this.
I also hate it when people write
char *a;
In the first, the variable `a` is const, but `*a` is not const. That is, you cannot change the value of `a` but you can change what it points to.
In the second `a` is not const, but `*a` is. That is, you can change `a` but you cannot change the value it points to.
I feel that your confusion is tied up in not understanding how C and C++ type declarations work, as is your insistence on `char* a`.
Dennis Ritchie (the C language designer) intended it to be written as `char *a` and talked about this at length over the years as an important design decision: variable declaration follows _usage_, so it’s `char *a` because the type of `*a` is `char`.
For better or worse, Stroustrup took this design decision into C++.
Sure both will parse in this simple case, but keeping this in mind is also the way that more complex declarations make sense.
But it's not.
char *a, b;
Read the pointer declarations right-to-left.
* const X* p means “p points to an X that is const”: the X object can’t be changed via p.
* X* const p means “p is a const pointer to an X that is non-const”: you can’t change the pointer p itself, but you can change the X object via p.
* const X* const p means “p is a const pointer to an X that is const”: you can’t change the pointer p itself, nor can you change the X object via p.
And, oh yea, did I mention to read your pointer declarations right-to-left?
For example, we use a wide_integer type for 128 and 256-bit integers: https://github.com/ClickHouse/ClickHouse/blob/master/base/ba...
It was developed by a C++ expert to fit into the C++ standard, so it has constexpr everywhere. But for this reason, we cannot use memcpy inside its methods and have to wait for a new standard with constexpr memcpy. Note: memcpy is to satisfy strict aliasing - we can use std::bit_cast instead, but there was some trouble.
PS. When reading about `consteval`, `constinit` you might think, "they (C++ committee) are totally crazy, how they think we can follow"
Wait what? This is what i was doing right now. It defines a local compile time constant and hence is an important mean to express an abstract concept. You could use a macro, but we are told to get rid of them. And a local const variable is an entirely different thing.
Surely any competent optimiser will make them the same though.
Hmm… this is just unnecessarily dogmatic. I use constexpr all the time, for example, for local numerical constants that don’t need to be a global.
I don’t think static would add anything under these conditions.
If the latter, then it is pretty annoying that non-standard behavior is happening now related to standard libraries implementations and constexpr.
It is more limited, no question. But you don't have to hope that the compiler will do things, it is guaranteed to do them.
There is pretty much nothing useful, this post included, that can be done with const fn in rust.
Feels like a straw man? I never promoted Rust, certainly I didn't bring it up unprompted. I corrected someone who brought it up.
> unless you think any mention of rust must be positive.
Again, feels like a straw man. I said nothing of the sort.
> There is pretty much nothing useful, this post included, that can be done with const fn in rust.
I feel like you're simultaneously upset that I corrected your post... but also you're challenging my point, in an attempt to pursue discussion? It's confusing because you're wrong but there's also an interesting discussion to be had - although I think this post covers the general issue of "what is a pointer at compile time" quite well.
I’m not upset about anything. You said I’m wrong, so show me how you use const fn to make a heap allocated string at compile time as done in this post.
> You said I’m wrong, so show me how you use const fn to make a heap allocated string at compile time as done in this post.
You:
> where you are forced to stick everything in a macro or pray the compiler is smart for compile time programming.
`const fn` exists. You do not need to "pray the compiler is smart" - you can run code at compile time. No need for macros either.
As I also said, it is not as powerful as C++. You can not do heap allocations with const fn.
The sentiment you're looking for is "talking past each other"
(i'm assuming mods can detach my comment from the thread?)
We are discussing a compile time string. So how can you use const fn to make a compile time string in rust with no macros?
Right, this is a straw man. I've never said anything at all about people criticizing C++ and promoting Rust. You're creating an argument out of thin air and attacking it.
> We are discussing a compile time string.
Incorrect. Here is what you said:
> Much more than rust, that’s for sure, where you are forced to stick everything in a macro or pray the compiler is smart for compile time programming.
In no way did you scope it to creating heap allocating strings. Even the post, which ultimately discusses that, discusses other things.
For example, you can write a substr function (firstName) using a const fn in Rust. No macros, no "praying" that the compiler will optimize things.
It would be very odd to limit the discussion to a `string` since this post focuses on how difficult and complex that is in C++, so using it as a "Rust can't even do this" would be sort of ironic.
Anyways, I've engaged in good faith in this discussion for far too long. Your posts are borderline incoherent, frankly, and I think any reader of this thread will have long gotten the message that your initial post was both irrelevant and incorrect.
The title of the post asks how constexpr c++ strings are. I answered with “more than rust’s”. The existence of const fn has yet to prove me incorrect.
It often feels like math professors discussing math for the fun of it, not for solving practical problems.
For example, even though both Chromium and Unreal Engine are using C++, good luck finding basically anything high level in common. All of the naming schemes are different, the formatting is different, the high level libraries are unrecognizable. It almost feels like these projects are written in different languages.
Beneath all of that, they do make use of the same low level mechanics. A pointer is still a pointer, no matter what libraries you build on it. A stack allocation is still a stack allocation, passing something by a const reference is still the same, passing universal references to container templates is still the same. Includes work the same everywhere. And so on.
Basically, unlike most other languages, you want to have a solid understanding of these underlying mechanics so you can then quickly understand whatever high level libraries the project decided to build on top and hit the ground running using those. There's gonna be some kind of resizable array thing in every project.
But yes, you're basically starting over re-familiarizing yourself with the language on every large project or framework you explore. I've been using C++ for many years and never once used a std::string, since each framework has its own clearly superior string...
Java can be really fast and handle big codebases too, and it’s also been huge for long enough it can buy a beer, and it’s also getting a little hairy.
Uh, other languages in the niche seem to be starting their runs at comparable levels of complexity, if they’re as successful as it looks like, writing them will be quantum chromodynamics in another 30.
IMO Java is so simple you can teach it to any programmer in a day or two. I'm not sure which part are "hairy", maybe frameworks?
Memory order is hard but that's only very rarely used in idiomatic Java, and mostly through the atomic wrappers. Synchronized is simpler than equivalent barriers in almost all other languages.
There aren't that many keywords, no operator overloading, barely any metaprogramming (just annotations), almost no compile flags, etc.
I guess tuning the runtime can be hairy, and some of the standard lib stuff is weirdly over complicated (nio vs io), but as far as languages go, Java has been incredibly conservative. In fact, a lot of Java app dev would probably be easier in practice if it had more features.
Old presentation https://youtu.be/a0FliKwcwXE?t=31
that when I did watch for the first time did remind me A LOT the memes about different levels of haskell engineers writing factorial function, with people somewhere in the middle implementing their own numerics based on basic number theory things. This presentation is literally exact scenario - you take one thing and start building poor man's programming language using it. And it's horrific.
Problem with C++, that it is extremely slowly addressing is that it doesn't have many facilities to improve that nested informal templated language built on top of templating constructs. C++ is getting them, but it at glacier pace. And when I see that presentation after having exposure to other languages, I just feel that this is so much waste here, it could've been much better if language had better thought put into metaprogramming part, instead of letting random crazy geniuses to build hacks on top of hacks to get something useful.
Unfortunately for C++, it gives significant problem for starting. Because to learn something, it is very useful to look into how standard things, like stdlib is implemented. But what you see, often, is not C++, it is that another _thing_ that is using c++ constructs to have its own thing. That has its own patterns and quirks (and there are many of them, so many).
C++ went into local maxima by letting to do many things by various template tricks, but in the end I think that this is dead end. C++ need real metaprogramming, type level programming, you name it - that can be done in (constrained) C++, not in some fungus that has grown on top of template language.
If you fast-forward to C++20, metaprogramming is mostly, though not always, concise and easy to follow. The language was redesigned to make template hacks that people found highly useful into first-class features of the language. For reasons of backward compatibility, you can still express these things the old ways. But it isn’t required anymore, there are concise and direct ways of expressing most metaprogramming things you might want to do.
The extreme expressiveness of modern C++ in application domains with unusual requirements remains its greatest strength. For pure systems languages, nothing else can express as much in a type-safe way in so few lines of code, almost entirely due to its metaprogramming facilities.
I'd imagine ms vc++ has tons of stuff to support this?
Again, I haven't used a commercial compiler in a while. But that seems like really standard stuff you get with them, yeah?
Meant to be ‘::’, of course. (This abomination can only compete with the ML’s double semicolon.)