I know it's probably hyperbole, but why
> yes, I know, friends don’t let friends use STL, but that is a topic for another discussion
?
Are there known shortcomings with the standard library in C++?
> yes, I know, friends don’t let friends use STL, but that is a topic for another discussion
?
Are there known shortcomings with the standard library in C++?
A "drop-in replacement" would commit to all the weird things about the standard library type which result in poor performance, some of them might be important to you, and you'll probably insist oh, it's not a "drop-in" replacement if I can't have those, and across the entire C++ population that means there is no high performance drop-in replacement, the type guarantees any number of weird properties and you could depend on all those, so, too bad.
The most likely reliance you'd have is reference stability, maybe you give out references to items in the table and they're long lived so you mustn't invalidate them when/ if there's a resize, let alone on unrelated changes to a table which keeps the same size. Abseil, Boost and Folly all provide types which lose some performance to get this back if it's all you need to be "drop-in".
The most unlikely reliances are two fold, firstly maybe you insist on messing with the bucket API, which makes no sense in open addressed tables (ie all the ones with great performance). In this case you're wedded to std::unordered_map because the details of how the type is structured are welded into your code. Sucks to be you. Secondly though, you might have decided you care about the order, the std::unordered_map says it doesn't promise an order, but on a single platform none of the implementations actually kick you if you assume the order is preserved, so you can merrily write a unit test which checks that after adding fake users Alice, Bob and Carol to your user hash table, when asked for the users back in no particular order you get exactly: Carol, Alice, Bob. Is that a good unit test? No. But the replacements will make it go red, they're not "drop-in" replacements.
Indeed, Google's initial migration from std::unordered_map to swisstables moved everyone to absl::node_hash_map since it was safer, even though flat_hash_map generally offers better performance.
> after adding fake users Alice, Bob and Carol to your user hash table, when asked for the users back in no particular order you get exactly: Carol, Alice, Bob.
Ow :(
This reminded me of Matt Kulukundis at cppcon talking about migrating away from std::unordered_map, wherein he brought up this point exactly: https://youtu.be/JZE3_0qvrMg?t=740
Matt then goes on to describe how Abseil adds a nondeterministic iteration offset to each hash table to prevent anyone from relying on this moving forward.
Also: Unit test people, it's great that it's easy to check that "foo == 5" is true, and I can see why you thought it should be equally easy to check "foo == Carol, Alice, Bob" but could you perhaps nudge the developer to check they didn't mean "foo contains all of Carol, Alice, Bob" instead when they do that? Maybe have them pick "in order" versus "any order" with no default ?
There are a few more like this, case sensitivity is one, my newest colleague is going to write a test for "dnsName = example.com" even though actually dnsName might sometimes be "EXAMPLE.COM" or "Example.Com" or even "eXAMplE.cOM" and it'd be nice if they were nudged to spend that extra moment to decide whether that's OK or not before a test goes red spuriously.
Growable arrays are a very popular idea. Your preferred language almost certainly has them, but they might be built in to the language as an optimisation, if "normal" arrays in your language can grow and that's fast enough, chances are you're got the growable array type just as the only array, which is fine (but won't fly on tiny embedded systems).
This type has amortized O(1) aka constant time growth, which is one of those tricks which is amazing if you've never seen it before, then it just seeps into your understanding of the world and so it's easy to imagine why C++ doesn't do any better.
As a necessary optimisation beyond the amortized growth we must be able to tell a growable array hints about its future size. The only hint provided in the C++ STL is reserve, which tells the array its maximum final size. Rust calls this hint Vec::reserve_exact, if our array isn't this big, we should grow immediately, makes sense.
Here's where the problem kicks in: Suppose I don't know the maximum final size, but I do know I need to put say, at least 50 more things in the container? Algorithmically the answer is actually yes, the amortized growth can be preserved if we either grow sooner or skip growth steps and otherwise the hint can be ignored, this is Rust's Vec::reserve
If we try to use the C++ reserve here (aka Vec::reserve_exact) even though this isn't the final size, we destroy our amortized growth performance, our container may have lousy performance. If we just don't bother because we can't express our hint, our container has worse performance than necessary because we grow more often or less optimally.
But I've burned myself on this a few times before.
But I wonder what use case there is where you can't just push_back() one by one? The only potential issue here is that there might be a reallocation happening while append this range of N elements, but can't imagine what might break.
This is a Quality of Implementation issue in the library standard itself.
Never had a problem myself with that though, my main use case for std::vector is with reserve() right at the beginning and never provoking a reallocation (so I often end up coding up my own container to assert reallocation can't happen accidentally).
Sometimes (infrequently) I use the script-style push_back()-only pattern. But I feel very uneasy about uncontrolled reallocations in larger systems, and anyway they constrain the implementation space because of pointer and iterator invaliation.
Then there's the other segment of C++ programmers talking about how modern C++ is so much better now to the point that Rust and other languages are unnecessary.
The cognitive dissonance is intoxicating.
It has been a while since I was in gamedev so perhaps not true anymore but sometimes these things change slowly with people/culture.
The C++ STL is probably one of the most commonly avoided standard libraries of any language. I can’t think of another that is so often replaced by bespoke code
However, in C++ because the STL is a big part of their Standard Library, and because Microsoft insists on naming their entire implementation of the Standard Library STL, it's usual to say STL to mean the entire stdlib, even though many parts of it aren't about generic programming at all.
So the STL is a crucial idea and inventing that was a tremendous accomplishment - but the C++ stdlib, which may be called "STL" and is in this article, is not a very good realisation of that idea. Indeed I'd argue the C++ programming language isn't a good realisation of that idea either - it was possible to write the STL for C++ and so that's what was done, but knowing this is a good idea you'd design the language differently. And people have.
The last paragraph hints that there are more modern languages than C++ which implement generic programming in a better way, truer to its original conception?
The breakthrough with C++ templates, was to provide a macro system which creates an AST for a static language, at compile time, using (surprise!) a template of the desired code. Contrast with the C preprocessor, which works on the token stream (there's more to it if you're curious, but in a way that makes it worse, not better).
Of course, a macro system operating on the AST is an older idea than C, this is one of the essential features of Lisp. But Lisps have always been dynamic languages. In some other timeline, there was an s-expression language for writing static programs, which used Lisp-style macros, and thereby avoided the problems with C++ templates. But we don't live in that timeline, it's a road not taken.
This template system enabled C++ programmers to write code which was generic over not just types, but features of types, and much else, it's all rather complex.
The problem, in essence, is that it was thought that template-expansion could be a decidable process implemented using a sub-Turing-complete language. So it was built using a second syntax, which was intended to be restrictive but flexible. Uh-oh!
C++ templates are Turing complete, as it turns out. So what C++ has is a second language. You could write your entire program in that language, and execute it at compile time. For everything you can do in C++, there's a second way to do it in templates, with its own syntax and semantics. Invariably, this language is much more verbose, and just... weird, in big and small ways. The sum of this is a system which is difficult to understand, and which generates notoriously terrible errors. This isn't intended as a complete summary of C++ template's shortcomings and weird features. That would take longer.
Zig just lets you run Zig at compile time. It's much, much better. Instead of trying and failing to make the problem decidable, Zig brute-forces the question by providing a branch quota: this is similar to gas in the EVM, or what Piccolo, a Rust implementation of Lua, calls "fuel". Comptime comes with a complexity limit, after which it quits, and which can be boosted for operations where the quota is insufficient.
So there aren't two languages to learn, just the one: Zig. That glosses over some things, like loop and switch constructs have inline equivalents for comptime, but the general premise holds.
There are other things which C++ makes easy which are less easy in Zig, like runtime polymorphism. But comptime is just a massive improvement over templates.
From what I understand, a disadvantage of this templating capability in C++ is that it makes the language "unpredictable" in terms of static analysis, so that, for example, intellisense in code editors cannot be implemented without having a full C++ compiler. ..But then again, maybe that's true of any language with macros, where the code can dynamically modify itself.
That's an interesting point too, about how C++ templating is a second language that runs at compile time, making the code more complicated to understand. Whereas in Lisps, macros are written in the same language, working with the same AST in a unified way. It's conceptually simpler, a more elegant way to achieve the same result.
I'd heard of comptime in Zig, but didn't know that there was a "complexity limit". Fascinating. It makes sense how it's related to the concept of "gas" in the Ethereum virtual machine, and "fuel" in Piccolo. (I remember seeing the latter language recently.)
https://ziglang.org/documentation/master/#setEvalBranchQuota
https://kyju.org/blog/piccolo-a-stackless-lua-interpreter/#f...
https://ethereum.org/en/developers/docs/gas/
..Whew, many tempting rabbit holes there. :) I'd been interested in Zig, reading about it here and there, but your comment convinced me to get more serious about studying it. Thanks!
https://tour.dlang.org/tour/en/basics/templates
https://tour.dlang.org/tour/en/gems/compile-time-function-ev...
As far as I can tell, D's templates and CTFE are as powerful as Zig's comptime (not sure if they're exactly equivalently powerful, but if they're not, the difference is not very large in practice or I'm pretty sure I would've noticed).
You might very well be right. I didn't realize that D's templating was much different from C++, I don't know much about the language. I'll also note that the sibling comment you refer to wasn't posted when I wrote my reply.
I'll note that a careful reading of my post, and the one it was responding to, will reveal that I didn't claim that Zig was the only, or first, language to improve on C++ templates. Only that it's an example of a more modern language which has done so. But I'm also largely ignorant of the ins and outs of D.
It seems that you, however, are not. Here's an example of Zig building a data structure at comptime:
https://ziglang.org/documentation/0.12.0/std/#src/std/compti...
Is this an example of something D could do as well? If so, I'll be sure to give the language credit the next time the topic comes up.
About comptime, it seems to me it's trivial to do that in D.
I started a little implementation of StringMap based losely on the Zig example, but I just did enough to show how D's templates and associative arrays could be used to implement that:
https://godbolt.org/z/o7GoobbKb
You can see that the creation of the map does not generate any Assembly, so it's run at comptime:
// comptime values!
static immutable entries = ["foo": 1, "bar": 2, "zort": 3];
enum map = mapOf(entries);
The Assembly seems to be almost all related to the associative array implementation generated by the D compiler.
EDIT : if you want to try to continue the impl, have a look at the D's template tutorial: https://github.com/PhilippeSigaud/D-templates-tutorial/blob/...
D templates can do pretty much anything.
> it has associative arrays
Zig also has a variety of generic data structures in the standard library for this purpose, HashMap and ArrayHashMap and variants. The main limitation in the current implementation of comptime is that it can't handle heap allocations, so these can't be used at comptime yet. D has a garbage collector, which makes that sort of thing easier. It's also about fifteen years older, which also helps.
That limitation won't be there forever, though, this is just the immaturity I referenced in my first post.
But fair enough, you've convinced me: D is also a language which has improved on C++ templates using compile-time execution.
Looks like C++ itself is another such language! The last time I used it was well before the later constexpr and immediate mode function features described here existed. https://en.wikipedia.org/wiki/Compile-time_function_executio...
The libc qsort is type erased and only works on arrays. So to sort Widgets we note down how big a Widget is and how many there are, then we forget these are Widgets and we provide qsort with just a type erased pointer to the array [of Widgets], the size and count, and we provide a function pointer (we'll need to ensure it's suited for specifically sorting Widgets) to a custom function we wrote which receives a pair of type erased pointers [to Widgets] to decide which comes before the other.
A lot of performance optimisations are left on the table when we do this, and of course because it's type erased it also loses type safety. We can't reliably apply our algorithm to any other collection, and we can't easily re-use the comparison functions either.
The STL's generics mean you preserve type safety and you can use this for any (conforming) collection type, and you can re-use comparisons because those are generic too. Along the way you pick up the performance optimisations that were lost with type erasure.
Better still, all of this is optional. You can perform the type erasure explicitly if, for example, you have a thousand different types and want to emit code to sort them all even if it's a little slower, rather than emit a thousand different sort implementations, if the code size is crucial for example.
The one well-known "generic" algorithm in libc is qsort. But qsort only works with pointers, and so can only sort continuous containers. You can't sort a linked list with qsort. In contrast, the STL has a more general concept of iterators that know how to fetch their next element (and some iterators can do more than just that). This allows the STL to have a std::sort algorithm that works on arrays, linked lists, trees, and whatever other data structure you want to come up with.
So the crucial idea is that the STL provides a large set of algorithms that work on all containers. No matter what kind of weird container you have, as long as you can provide an iterator for it, you can use most of the ~100 different standard algorithms on it. You can sort, find specific elements, partition, find the max element, filter elements, etc.
And the idea also goes the other way around. The STL also provides common containers that you'll need, such as linked lists, growable arrays, associative containers, strings. Of course, all the STL algorithms work with the STL containers.
Then, if you want to write your own algorithm, you only need to write it once in terms of iterators and then you can use it with all the different containers. No need to write the same routine twice.
The last big accomplishment is that all this can be done with almost no runtime overhead. All the types are resolved at compile-time, and generic algorithms can be specialized if you have a more efficient implementation for a specific container. This means that if you wrote a manual implementation of an algorithm that specifically matches one container, it would not be much faster or consume fewer resources than just using the generic STL algorithm. At the time the STL was originally incorporated, I think this was unique and no other language was actually capable of doing this.