Is C++ fast?
zeuxcg.org
zeuxcg.org
What does this have to do with C++?
This is more like “is C++’s STL fast for me”.
However I'd also say this is something already well known by most C and, particularly, C++ developers. Yes, you can get better performance by reimplementing much of what C++ gives you, and that is typical of writing C code instead of C++. But there's a fair bit of productivity trade off in doing so.
A few years ago for debugging purposes I wanted to hexdump a large amount of data. Turns out that the standard hexdump tool was pretty slow at it, writing a simple C utility that would format the dump "by hand" instead of using printf was about an order of magnitude faster. You have fully static code instead of the interpreted format string, you can unroll all the formatting, you don't have to rewrite the characters that don't change in every line (spaces, delimiters etc...). It's lightning fast but of course not very flexible.
Yes, of course it is. That's why I took it as an example. Since I work on RealTime systems I usually have to use plain 'write' instead. But that is because 'printf' is an incredibly complex function, not because 'C is slow'.
- printf is a standard function in C's stdlib.
- Because of the nature of the function (using a special purpose format strings and varargs instead of some kind of generic system) the function can't really be aggressively optimized and inlined without compiler magic. This limitation is caused by C's own limitations when it comes to generic programming and type introspection, printf needs to be told explicitly through a "side-channel" (the format string) what types to expect because C lacks the infrastructure to do that by itself. It also means that you can't extend the function for custom types and that you can't type-check the parameters without compiler magic.
- Even though some compilers do implement special magic around printf to perform more aggressive optimizations (like replacing it with "puts" if there's no formatting going on for instance) it's still generally trivial to beat its performance in non-trivial case with handwritten code.
Given all of the above I would say that makes this particular corner of C rather slow indeed. Or at least slower than it might be compared to an hypothetical language that made it easier to optimize string formatting.
int x;
cout << x;
ios::sync_with_stdio(false); #include “studio.h”
int x;
printf(“%d”, x);
(The #include is essential. With it, the compiler can know what printf does, and, in theory, optimize it to a int-to-string conversion and a puts system call. Without it, it cannot, because it doesn’t know what the printf function does (in this case, it is easier to generate fast code for the compiler if it doesn’t have the source available than when it would see source code for a function called printf)
Utterly pedantic: no. As given, both are undefined behavior, and it is trivial for a compiler to discover that.
That is true for certain format strings (containing only %s or %c conversions), but for %d GCC and Clang don't seem to want to call a print_int function, possibly because there is none: https://gcc.godbolt.org/z/rltFse
I agree with you about the title, but I also think that if you fix the title the author is making a good point. It's not groundbreaking, but the move "down" from C++ to C does run parallel with the move from more general purpose to more special purpose, in a lot of ways. C dominates embedded after all.
It's been one of the things that have most strongly shaped the evolution of the language and the library, for both good and bad, so absolutely, it is very much well known.
You might say that compilation time is nothing.. But it is C++, compilation time has a huge impact on big codebases.
My point stands. The performance trade-offs you're paying for with various C++ features are extremely well understood, and better documented for C++ than most languages.
That doesn't mean there aren't surprises here and there, and obviously there are compiler difference, but the point was that the language is designed so that these tradeoffs are known and documented and for the most part comes as a surprise to people mostly when they don't know the language very well.
It is not a surprise that you can improve on things like std::vector by rolling your own that does exactly what you want; for starters if you know your requirements chances are you can speed up initialization, because you don't need to abide by standard semantics. For starters, if you know you're going to assign a fixed number of elements, you don't need to track size and capacity, and you can often avoid the need for bounds checking.
As for compilation time, that too is very much a matter of paying for what you're using. You can choose to disable debug info, you can choose to not include headers that are bound to result in lots of time spent on parsing templates.
That he's not using many advanced features is irrelevant; his starting point very much had made trade-offs in using functionality that comes with known costs.
Or if performance is basically your only goal (HFT) then you just need to spend this time on a lot of parts of your code.
But then in 99% of code it's a waste of time.
In that case the author is showing what has been widely known by C++ developers for decades. In fact, the "don't use the STL because it's slow" is a mantra which, along with the old "replace STL's default allocators with your customized ones", is repeated ad nauseum in gamedev circles.
> This is definitely an area where libstdc++ and libc++ could improve in the future - I don’t think it’s reasonable to force users of C headers to pay for the C++ baggage.
On big machines a copying collector with a nursery is much faster and safer if you can deal with the pauses.
Memory safety is still an issue I heard.
It's a variant of the observation that, although in C++ you aren't paying a direct abstraction cost, you are often paying a generalisation cost. This cost is obscured by the abstraction and may not be warranted in your application. In other words, an indirect abstraction cost.
#include <cstdio>
void call_func(void(*func)()) {
func();
}
int main(int argc, char* argv[]) {
// Lambda can be converted to a function pointer
call_func([]() {
printf("hello as a function pointer\n");
});
}I am fairly certain I have had error messages and disassemblies that suggest otherwise. This may however be an implementation detail.
> You can use them as just function pointers, too.
Yes, I know this. It only works when there are no captures. And AFAIK once there are captures you start getting object code that looks a lot like you put those captures into an anonymous class. (When not inlined.)
And I still maintain that most people treat class libraries (such as STL) as given and do not inspect how they are implemented and do not consider what constraints / alternatives do exist. One example: the std::dequeue is a dynamic array, and you can't quite tune the size of the inner array easily by means of template or instance parameters - no one cares. Also many people happily use the shared_ptr class and don't even know that it takes very expensive atomic operations to do the reference counting; I would have expected a template parameter that could determine if shared_ptr is meant for multithreaded use or not. Most people don't care and happily use the shared_ptr. Also there are no intrusive linked lists in STL (well, there are in boost)
I can go on and on about the STL (and other widely used libraries if you care) and I still suspect that they get away with these "issues" in a widely used library because people are used not to question library implementations and treat them as frigging black boxes.
And STL really isn't based on an OOP mindeset.
You could achieve good release build runtime-speeds in modern idiomatic C++, but you have to trade off your compile times and Debug-build runtime-speeds for that.
Well, other than the "zero-cost" features that increase code size. For example careless use of heavy template classes can generate a lot of code.
In other words, in C++ it's very easy to generate a lot of code from seemingly innocent constructs.
Instruction cache misses, page faults (and to lesser amount TLB misses) are not zero cost.
It would be nice to have a tool that could compute the size impact of a given line.
[1] https://gist.github.com/zeux/bf847986e0474cf48f61bb5749da38e...
Given that most new C++ features in the "modern day" are implemented as std::whatever, in the "everything is a library" way, it's extremely relevant.
I still have to google every time I need to sleep the current thread ; and don't get me started on iostream!
However, I'm happy to be able to return a buffer without copying it (move semantics), to precompute frequency tables at compile time (constexpr), to be able to pass around functions without having to create a dedicated class (lambdas), not to have to worry about having an extra conversion (auto) ... And soon, I will be happy to write serialization code without relying on dirty tricks (introspection).
I write in C++. I had a coworker who wrote in Ruby. He wanted to learn C++ because C++ is "fast". So he wrote two identical programs: one in C++ and one in Ruby. Then, he ran both programs though `time` to determine how long each took.
His C++ program took longer.
Why was that?
He didn't write his C++ program using any efficient algorithms whatsoever while his Ruby program was well-designed. He came to me to try to tell me that C++ wasn't as fast as Ruby.
So I rewrote his C++ program. My C++ program then ran almost instantly while his Ruby program hadn't even begun reading or writing; the C++ program finished while the Ruby program was still initializing the Ruby runtime libraries.
Is C++ fast? Yes, it can be. But it's only fast if the developer understands how to make it fast.
Don't use C++ because it's "fast". Use C++ because you understand what its intended purpose is: to be close to the hardware.
I suppose the question that many people would like to ask, but it's hard to test is:
"Can we rank the speed of languages, given each are used by a domain expert to take full advantage of the features of that language"
or some such.
i.e. if a masterful C programmer and a masterful C++ programmer and a masterful Java Programmer and a masterful Ruby programmer all write something, which is faster.
I'm sure the answer is "it depends", because that's always the answer.
What gets overlooked in language performance is that you pick a language for what are, in essence, placeholder constructs. The performance matters only in that you need it to be fast enough "off the shelf" that you don't have to start writing a compiler. But you generally aren't prevented from using that approach to boost your existing code, and high performance systems often converge on that strategy in some degree, because it lets you keep maintainable source code and add toggles for debugging and optimization that you won't get from treating the application and compiler code as separate entities.
The article doesn't mention what versions of each compiler was used. I'd also challenge the author to use Compiler Explorer to discover compiled-to-assembly-level differences between their C and C++ code.
Given those requirements, a custom allocator would speed things up. A custom allocator could allocate even on the stack (small buffer optimisation) if possible or use a memory pool.
Embracing C++ features or reading the docs often gives you more than going back to C or reinventing the wheel.
However I think this demonstrates one of the many lurking problems of C++: It does not enforce the right way of doing things nor makes it easy. Instead it makes the right solution awkward or cumbersome to write.
but what the author is doing here, while being "the right thing" for his own project with specific performance requirements, is absolutely not the right thing to do for the average project that has a few std::vector of 15 widgets, a dozen strings and six and a half pixmaps, or does the occasional web request.
Also I'd like to see other optimization levels, since at least GCC adds a lot more potentially slow (to compile) optimizations at -O3. Maybe -ffast-math. Static linking too could conceivably make a difference if he calls into external libraries frequently.
And where does perf say the time is going? Maybe it's all lost in branches or cache misses so a different algorithm that branches less or uses denser data could help.
How about vectorization? I've now looked at the code and it's all vectors, arrays and matrices so surely this is an application for AVX, maybe even a GPU.
Anyway I think what's clear is this doesn't have much to do with C vs C++.
They write, "We don’t need resize or push_back in our code, all arrays are initialized with the right size." But if you're not doing any resizing or anything, and you're reserving the right amount, std::vector basically is a thin wrapper over a dynamically-allocated raw array -- which is what they tout as their replacement.
I guess I'm surprised that the overhead from default initialization was so large.
With vector, it seems there are always tradeoffs: resize and you have some initialization overhead, but only reserve and you will have a bounds-check-maybe-realloc every time you need to push_back. Of course, since you reserved, the realloc never happens, but you still have to check each time.
I wonder if something like generate_n or copy_n into a back_inserter can avoid the bounds checks?
Not exactly. A dynamically-allocated array doesn't necessarily save its own size (the system allocator can do optimizations), but std::vector does (you could call size() on it). So std::vector has to save this size somewhere, which takes space and time.
The overhead is thin but non-zero.
Doing "push_back" requires checking if "size < capacity", so this operation has a lot of overhead even for std::vector instances that never reallocate storage.
- code size is bigger, taking up space in the cache and memory;
- instructions need to be decoded (whether this affects performance depends on surrounding code);
- this branch will take a slot in the branch predictor state (same here).
Apparently you can prevent this by using custom allocators, which might be a fine solution but makes the code look less idiomatic.
The algorithm discussed also involves matrix math. It’s not object oriented, it’s not data processing. So, there’s just naturally not that much difference between C and C++ in this domain.
I only write occasional C, and no C++ since college, but I think this Arena pattern is drastically underused. I'd love to see it in web frameworks, where every allocation inside a request comes from the same place, and at the end you just give it all back. It seems like you could even offer that in Ruby. Does Rocket use anything like this? It seems like the kind of thing Rust folks would take advantage of.
Postgres memory allocation follows an Arena pattern, where you use palloc to grab memory, but you rarely have to free it, because it is all freed when the "context" completes. There are nested contexts for things like the connection, the transaction, the sql statement, a function call, etc., so as long as you're allocating from the right context, you can practically ignore cleaning up after yourself. (This is from the perspective of a frequent extension author but someone who has only barely contributed to the core project, so perhaps a more expert contributor can correct me and/or elaborate.)
EDIT: Btw, I really enjoyed the article; thank you for sharing your experiences with us!
When I understood this, it was an aha! moment for GC languages in stateless server contexts. I could see that for a properly written server - ideally keeping any long-lived mutable memory like caches off-heap - GC overhead could be negligible.
Anything requiring large semi-long-lived intermediate buffers would need a different approach, I agree. Similar to caches; you don't want a mid-life crisis in a generational GC.
- Why O2? Why not O3? You're tuning your code for maximum performance and then leave the optimizer halfway? That'll of course disadvantage code with more abstractions.
- Compile times are improved by dropping STL headers - that's why precompiled headers exist. Having that factored into the comparison would indeed be interesting, sadly this was not done.
- MSVC is slow in debug mode because it's doing every debug check it can. If that's not what you want, you can just change the iterator debug level. I was hoping the author would figure as much after finding the gcc/clang switches (and complaining that they were off by default - very ironic) with note 6 but no dice.
- Regarding the choice of title: Almost all performance improvements were of algorithmic/data structure nature. That's completely natural for the level of performance tuning here. And I wouldn't really care if those specialized implementations are in C++ or C, do what you want. Just don't implement them in C and then draw conclusions from comparing them against the C++ STL under the flag of "Is C++ fast?". I think a better title for the article would've been "Are defaults fast?".
The language is just a tool; USE the tool, don't be a tool.
Here’s an example where a fast linked list implementation (not mine, `CAtlList` from ATL) approaches or outperforms std::vector: https://github.com/Const-me/CollectionMicrobench
Fortunately, in C++ we don’t pay for features we don’t use. If you want fast code, no need to go back to C, you only need better C++ containers.
If you want fast performance, you use a Release build. The purpose of a Debug build is that you can detect the source of as many bugs in your program as possible. So for Release build, it is a priority to improve performance (and in my opinion Microsoft did a good job). For the Debug build, it is a priority to improve the detectability of bugs in your program. Here also in my opinion Microsoft did a good job.
If you want it to be as fast as other implementations, ask it to do less of this. They provide [_ITERATOR_DEBUG_LEVEL](https://docs.microsoft.com/en-us/cpp/standard-library/iterat...) for exactly that purpose. I think their choice of default (Debug means you want it to catch as many bugs as it can, and Release means it should be fast) is appropriate.
If they hadn't provided the knob to adjust this I'd agree they had a problem for some use-cases, but if you want the faster-but-less-thorough mode you just have to ask for it.
In a way it is comparable to the overhead of the sanitizers available in other compilers.
IIRC MS chose to enable it by default in debug mode as part of their effort of fortifying unsafe C and C++ code against security bugs, but of course it has a large penalty.
libstdc++ has a similar debug mode, but it is disabled by default, as, in addition to be more expensive, it also breaks the ABI.
I have a hard time to get the rationale here. Is the author implying that using memset is inherently better than initializing an std::vector with the constructor that takes a count and a default value, or resetting it with the same variant of .assign()? Skimming through the code that's linked in the article, it looks like there's no use case where either the former or the latter wouldn't fit.
#include <vector>
#include <cstdio>
struct MyFoo {
MyFoo() { printf("default ctor\n"); }
MyFoo(int i) { printf("Initialized with %d\n", i); }
MyFoo(const MyFoo& foo) {}
};
int main() {
std::vector<MyFoo> myvec;
myvec.reserve(100);
printf("resize(2)\n");
myvec.resize(2);
printf("emplace_back()\n");
myvec.emplace_back();
printf("emplace_back(1)\n");
myvec.emplace_back(1);
printf("push_back(MyFoo{1})\n");
myvec.push_back(MyFoo{2});
return 0;
}
resize(2)
default ctor
default ctor
emplace_back()
default ctor
emplace_back(1)
Initialized with 1
push_back(MyFoo{1})
Initialized with 2
C++ is no different. If you want it to be blazing fast, you need to tell it exactly what you want. What data structure do you want, where do you want it, where should it ask for more memory if needed?
If you don't do that, there's some sane defaults but the compiler doesn't know whether you are going to relocate the structure or how you plan to access it.
> dropping C++11 requirement allowed me to make sure anybody can compile the library on any platform, removing std::vector substantially improved performance of unoptimized builds
What platform doesn't have C++11 support? And why is the performance of an unoptimized build interesting? -O2 is basically a standard compiler flag...?
In any C++ conference talk about performance the speaker's almost guaranteed to mention that their project uses their own optimized library instead of the STL.
It starts with the comparison of a general-purpose hash table with specifically-tailored one.
Then it throws in a switch from a comparison-based sort to a radix-based one.
Then, the stunning discovery that std::vector value-initialises its elements (fixable with 3 lines of a custom allocator, if you don't want that behaviour).
In the middle, it preaches about portability while at the same time advocating the use of __builtin functions vs standard functions, and type-punning floats to int and back (undefined behaviour in C++).
And all of this for a 572ms -> 320ms performance improvement?
And the issue with vector value initializing it's elements is also well known and programmers have been asking for a solution (more user friendly than a custom allocator) for a long time.
Want a vector that does not initialize its elements ? Ask and thou shalt receive: https://github.com/aguinet/pector
The whole point of C++ is to make it easy for people to write generic libraries beyond de standard base provided by the stdlib. So why won't you use those ?
Still it is fair to critique the specific tradeoff done by the standard library.
You do not have to pay for what you do not use.
What is unfriendly about custom allocators? Note that in this specific case you don't need to write any allocation function, you just have to re-implement the construct() function with something that does not value-initialises if no construction arguments are provided.
I disagree.
1. Everybody knows Python is a scripting language, and trying to do heavy computation without handing it off to a library is going to be slow.
2. A vector is the de facto data structure for this in c++. In Python, numpy is effectively the de facto approach for math.
1. "Everybody" knows that unordered sets/maps in C++ must de-facto use chaining due to the requirements imposed by the standard.
2. numpy.array is not part of the Python standard library. It is an extra component you still need to install separately. As someone who deals regularly with novice Python users (often coming from Matlab & the likes), one of the very first points of pain/confusion is the fact that I need to teach/convince them not to use the facilities of the core language to represent "vectors".
Also, IMHO tying allocation with initialization in the allocator concept is not one of C++ brightest moments.
There have been continuous talks about adding an unsafe_push_back and an unsafe resize for these and other reasons.
The same technique vastly improves map and set too.
It wouldn't say it "advocates" for the use of builtins. The author demonstrates a problem they had (including math.h made their compile times 2x slower in c++17 mode) and shows their solution, and advocates changing libc++ so that c headers don't pull in c++ stuff.
In any case, the snippet you object to uses the builtins if they are available, or falls back to math.h. It's a hack to avoid math.h when it's slow to compile, but stills seems pretty portable?
And yes, it's only a 252 ms improvement, but that's a whopping 44% time saved. If you need to do lots and lots of mesh optimizing, that might be very well worth it.
I thought it was ok. Asking "Is C++ fast?" is a little like asking if internal combustion engines are fast. The actual answer is, "It depends." The content is ok for this site, which also has people who are just starting out with optimization or who might not be that familiar with C++.
"Is [lang] fast?" is a typical clickbait tactic which works by nibbling at a programmer's pride. Here's the thing. Either one is a bit naive, the pride is hurt and the clickbait works, because one isn't seasoned enough to know that optimization is a complex subject within a complex world of tradeoffs or you're so invested in making things fast (possibly so good at it) that you can't resist taking it apart.
And all of this for a 572ms -> 320ms performance improvement?
78%? "Not tea bag!" as AvE would say.
>One number that we haven’t moved substantially yet is the time it takes to run this code in debug using MSVC. While it’s natural to expect unoptimized builds to be slower than optimized, they have to be fast enough. Sometimes you want to debug your problem on a non-trivial input dataset. Sometimes you want to run the debug build with full checks through your tests to make sure they don’t trigger any bugs that could disappear in release. Sometimes you are trying to debug a different part of the program, but you still need to run the rest of it.
The library, by the author, is https://github.com/zeux/meshoptimizer. I presume it will be used in graphics pipelines. Debug builds working at a speed that allow graphics to be tolerable by humans sounds like a good goal to have.
C++ has a lot of generic programming solutions like std::vector and algorithms, [and we think of these things as being fast even though they are generic because of templates and “zero cost abstractions”] but we show that we still pay for their being generic (because they must support operations which make everything else expensive, or are doing more work than we need done), and so [for situations like this] we prefer the trade-off of implementing specialised functions that may be similar to other functions elsewhere to gain runtime and compile-time performance improvements. We demonstrate this thesis with a case study of a mash simplification algorithm.
I think this is a reasonable topic to write about, and I think the article does an ok job of it. I think it is silly to get hung up on specific cases (e.g. you need to do some magic custom allocator thing to make a c++ thing about as cheap as a c thing that achieves the purposes you have, or that you might want to replace an exact sort with an approximate one (note that in this case, using a full radix sort would have you pay 30ms instead of 10, so 572 -> 340ms which is still a large improvement, and one that will become larger with larger inputs)). I think one should instead treat it as a case study which shows some general ideas like “you often do pay for what you don’t use in c++/stl” or “often one can use a more specialised algorithm with much better performance then a general one so when one is writing performance sensitive code, having generic algorithms to hand may not be useful”
The library is supposed to be part of a graphics pipeline. It will be called many times a second, and it will be part of what the eventual framerate is.
In this context, the Arseny's article should be titled: "How to be old-school and tailor algorithms & data structures to your needs". I have this haunting feeling that people nowadays are afraid of "reinventing wheel" even though it'd pay off. A recurring argument is that the standard library or boost or something available on github is better than a custom solution. But as we see, there are cases where saying "nope, I'll do better job myself" has sense.
It is already a somewhat funny first joke saying that the subset of modern C++ that's a nice language is just the C part and the "class" paradigm ("C with classes").
It is another second somewhat funny joke to say that actually the nice part is only an even more restrictive subset, "C with structs". Yes, structs and classes are virtually equivalent in C++, but let's assume they mean structs as in "classes without member functions". Which were already present in C. Which is why it is funny.
Jokes are never funny when explained :/ Source: I'm a somewhat experienced stand-up comedy practicioner, apart from a very experienced software developer.
I wholeheartedly recommend reading Stroustrup's "Design and evolution of C++", gives you so much valuable and interesting background.
From day 1 it was a full compiler that happened to generate C at the back end. At the time this was novel, and was easier for people to understand as if it were a preprocessor like RATFOR. Nowadays, generating C is an extremely familiar approach to bootstrapping a new language, and nobody pretends your compiler is "just a preprocessor". (Also, nowadays, you might better generate LLVM. But that option is new.)
So, saying "C with Classes was a preprocessor" amounts to simple slander.
Why is he optimizing for compile times when his compile times are only 1/2 second the start out with? Interesting article though.
Your code will be compiled only once for your customers, yet run millions of times. Clearly runtime is millions of times more important than compilation time.
If compilation is taking too long, start compiling in the cloud.
(yet)
There’s a supported way to speed up debug builds by a huge factor, for the price of less runtime checks.
https://docs.microsoft.com/en-us/cpp/standard-library/iterat...
You can workaround by undefining DEBUG, _DEBUG, defining NDEBUG, etc., to make headers think it's a release build, but then it won't be a debug build anymore.
If you want sanitization instead of stepping through the debugger, use that.
The people who are best at debugging avoid debuggers entirely, and use logging statements inserted in release-built code, and little proofs. It is hard to persuade people who don't think that way, but it is the only route to mastery.
At the same time I also use logging a lot. These two things aren't always mutually exclusive.
Only sometimes they are. Some platforms don't have good debuggers. For other stuff it's the opposite, e.g. you can't log from code running on GPU (very hard to accomplish, borderline impossible) but there're specialized GPU debuggers.
I don't think people who use just logging, or just debuggers, are the best.
Just the fact that we're asking ourselves "is C++ fast" should be frightening, for a language that has always rivalled C closely.