The Hidden Cost of C++
rachelslabnotes.com
rachelslabnotes.com
This is more a question of knowing what you are doing and less of C++ sucking. C doesn't solve this - you still have to know what you are doing there or you end up shooting yourself in the foot. I much prefer having an interface with virtual methods to a struct with function pointers.
For a game shop who is worried about performance there are many little things you can have your developers do to prevent the compiler from doing many of these things and you should have it be part of your normal API reviews to make sure that developers follow them. A simple example
class Box { public: Box(int x); };
class Foo { public: Foo(Box a) };
... Foo(1); ...
Pop Quiz: How would you change Foo (or the whole api) to cause Foo(1) to cause an error?
Put another way, the performance details we so often care about in projects for which we're using C++ are often non-local to the calling site. In order to know the performance characteristics of "a = foo(b, c)" I need to know a lot of things about the types of a, b, c, and foo, in addition to knowing what "foo" does inside its curly braces. In contrast, while it's hard to reason about the instructions being executed in Java in the same way we do in C or even to a large degree in C++, the types of a, b, c, and foo don't really matter, since passing in b and c is always a pass by reference or a primitive value copy, and foo is pretty much always going to be dispatched the same way (and constructors are preceded by a "new" operator at the call site, so we don't have to worry about that, either). Now, the only thing I care about is what's inside the foo method.
I am of course not disputing that you need to know the language you're using. It's just that even when you know C++, if you can't trust everyone who ever touched the code, you need to check in a bunch of different places to reason about the performance characteristics of this one piece of code.
Also, reasoning about the performance of C is all well and good right up until you realize that your compiler is better at translating C to assembler than you are and also, some jackass wrote a preprocessor macro called "foo" which does 18 different things. At some point, writing high-performance code is not about assuming you can guess what will happen but about seeing what's happening and trying to make something better happen.
Disclaimer: it's been a good 4 years or so since I worked with Java, and on top of that, I've never cared too much about Java performance, so please take this with a grain of salt.
The difference being that in this case, the code performs better than I assumed. Nobody minds that. In C++, the usual results are that it performs worse than assumed. Significantly worse.
> some jackass wrote a preprocessor macro called "foo" which does 18 different things
Yes. There is that. Thankfully, they are rare. (I'd rate the preprocessor as by far the worst part of C. Amongst other things, it costs us any number of tools that could help us reason about code)
The alternative for performance-critical applications is not Java, but C. C remains surprisingly tough competition after all these years for those rare pieces of software where programmer time is cheap compared to hardware resources. Having had non-trivial experience of both languages, reasoning about the performance of a C program is much, much easier than reasoning about non-trivial C++ programs' performance. Guesses based on local code inspection have an order of magnitude less uncertainty attached to them, which is basically what Rachel is saying here.
...which is why we have profiling tools that make it easy to detect performance hotspots. For the other 95% of our code, the expressiveness of C++ makes it easier to do more work in fewer lines. This is a far more important metric to any working developer (and it's a big reason why languages like Python and Ruby are popular.)
Also, you're exaggerating. I wrote C++ for well over a decade, and I wrote in C before that, and used it again in graduate school for my research. While I can count on one hand the number of times that I was affected by "hidden" C++ costs, I was routinely blindsided by bad C code, where pointer errors, type safety gotchas, and inefficiency due to aliasing are de rigeur. You'd be crazy to write a non-trivial application in C today that wasn't an operating system.
I am not talking about "business apps" (whatever that is). I'm talking about down-to-the-metal resource management, task scheduling, etc.
I could probably live with those issues in the higher logic layers (although I'd argue a scripting language would make you much more efficient there). But the underlying "OS" as well as computationally intensive tasks do not benefit from C++ that much, and it causes a lot of collateral damage.
No. C++ has some pretty huge advantages for performance-critical software. Generic programming makes it possible to write extremely succinct, high-performance code. Operator overloading allows the creation of vector and matrix libraries that look like real mathematical expressions, while evaluating to code that's as fast as anything you'll get from Fortran.
I didn't write "business apps" (not sure where you got that). I wrote software that simulated proteins -- in other words, high-performance computing -- and except for FORTRAN, C++ was the best choice for the job. Doing the equivalent optimizations of a library like Boost::MPL in C is a nightmare of pointer arithmetic.
When you throw in the fact that C++ makes it easy to avoid the memory leak, corruption and type safety issues that plague C code, you can easily see why C++ is becoming the language of choice in the HPC world.
And yet, the HPC world looks for a new language: http://www.cs.sandia.gov/Conferences/SOS10/presentations/Tue...
And a lot of the HPC world still writes FORTRAN or C. (I.e. BLAS, LAPACK, et.al.)
But then again, I don't spend that much time in the true HPC world. Maybe I'm missing a trend there. But from the outside, it looks like C++ is not quite what people wanted.
And hearing C proposed as an alternative to C++ is mind boggling. I don't want to match malloc/free calls, strcat strings together or realloc pointers. I don't want to declare a bunch o function pointers in a struct and call it OOP. I don't want to use macros because C99 has poor adoption rate and C89 doesn't have inline. I don't want to watch the compiler blissfully convert unrelated types to one another just because it can. I don't want to pass types around as void* and lose any useful type information I might have had. I don't want to fumble with return codes.
And in C++ I don't have to. I can use Qt, the STL and Boost and be very productive. Telling me to program in C is not even funny.
C++ is simply at the wrong abstraction level.
Oh, and I don't want to manually track resources myself, either. But guess what, somebody has to do. And on a game console, that's you. There's not much of an OS to speak of.
You did notice I was talking about game development, right? ;)
Anyway:
* smart pointers don't need an OS.
* for me, C++ is at the exactly right abstraction level to allow me to use only C++ instead of "C++ + other language" or "other language and C for speed". Most C + HLL proponents underestimate the logistics overhead of using multiple programming languages.
I'll concede that as a valid concern, especially for smaller projects where everybody touches many areas of the code. As projects grow, a separation becomes easier.
Add to that the fact that most games need scripting support anyways, and you'll see that the logistics overhead is at least not much different.
The mappings of C/C++'s type system into an HLL is almost intractable. What if there's a union somewhere? You could have distinct HLL and LLL representations, sync'ed either lazily or eagerly, but that's really, really complicated.
Unfortunately, you can only apply "C* + HLL" when there's a nice, clear, never-shifting boundary between the HLL and LLL parts. So, language selection is not a false dichotomy; you really do have to choose "mostly one."
(edit: typo, non-sense sentence. Sorry folks, running a fever.)
We allowed large parts AI/gameplay (not generally what I would think of as "kernel") to be pushed out of C++ into a HLL, and it was extremely painful bringing it back when the performance began to hurt us.
I'd certainly be interested to hear about this, because I'd love to know what the breaking points were.
Yeah, us old fuddy-duddies. We also like to reason about the correctness of our code while we're constructing it, instead of writing all of it and testing it. Crazy, huh? Like deep correctness, a performant system does not come from sprinkling some tools over at the end; it's an emergent property of attention to detail at every step.
The performance problems a profiler can show you are the easy ones. Hard performance problems come from users, where you can't touch, feel, and measure the code's dynamic behavior. Somewhere out there, in some customers' hands, software you barely remember writing three years ago is crawling along, and the customer is too livid to clearly express what she's doing to make it slow.
Just like debugging a tough correctness bug from the field, you are stuck alternating between deduction and induction. You try to piece together what the user did, and with luck you can reconstruct the bug! Yay, profile it, deduction worked! But I was almost never this lucky. I was left stuck using my bottom-up understanding of what the source code does and how it could have interacted with the user's setup. Identify a possible critical path for this workload, and stare at that motha with a critical eye: "I think you're slow sometimes. But obviously not always. Why? What could make you slow?"
C++ makes this part much, much harder, because even having narrowed it down to a candidate critical path, you can't look at the source code to know (as reliably and quickly as with C) what the hardware is going to do. Notice the qualifiers there (as reliably, as quickly). I'm sure a C++ expert like yourself could rattle off the whole flying circus of default constructors, copy constructors, operator overloads, and destructors that the compiler spits out, but to do so you will need non-local information in a way that you do not for C.
If these problems don't arise in your practice, awesome, good for you. Your marginal user doesn't consider performance problems bugs. That's a great situation to be in, because it lets you focus on features. But it does lead to question of why you are using C++.
> I don't want to match malloc/free calls, strcat strings together or realloc pointers.
If you don't occasionally really need realloc, that's cool, nothing wrong with that, but what kind of resource-intensive software are we talking about, then? It's an important tool in the toolbox when trying to implement data structures that occupy significant fractions of your address space.
a = func(b,c);
....
Is it a function call, a member function call, or is it an anonymous constructor? Are b and c implicitly invoking copy constructors for other classes as part of type coercion? Is that a normal assignment, or an assignment operator? Is there a cast operator involved?
If it's such a complicated expression, then the equivalent C code would be correspondingly large and hard to understand, and it would be just as complicated to figure out its cost. The not-so-hidden cost of C is that even simple things end up being many, many lines of code. C++ was invented because certain kinds of C programs -- programs that were already being written in C -- were painfully verbose to express and complicated to change.
If writing your code in C would be simple and clear, then you would have to be stupid to write it as complex, obscure C++. When your C++ code gets complex and you start banging your head against a wall, imagine re-expressing it in C. If the result would be an improvement, then you screwed up. If you blame C++ for screwing up, then you're a language feature junkie. Admit it and check yourself into rehab.
That is the hidden cost. The mental model for a simple function call became incredibly large and complex, and every function call is potentially as complex. Which makes reasoning about performance a rather hard thing to do.... All that translates ultimately into either worse performance or longer development time. Neither one is something you like to hear about.
I don't know how it is a "hidden cost," because it's quite well understood that any reasonably expressive language can say more in a single line of code than C. Anyway, the size of the mental model of your program is what you should be worried about. C++ doesn't give you worse performance or longer development time. It gives you more choices for how you express your program. It means more different problems, because you can write C-style code and have C-style problems, or you can use the possibilities C++ offers and have different problems. If you already know that the choices forced on you by C are usually the right ones for your domain, then by all means apply that knowledge to how you write C++. For instance:
Worse, it makes profiling harder than necessary. All the type coercions that happen at the API level will show up as separate functions, not attributed to the callee, but the caller.
If you want them attributed to the callee, it's pretty simple. Do the coercions in the callee, just like you would have in C.
I don't blame C++ for screwing up - I blame it for being an ill-designed language that makes it easy to screw up.
Any of the high-level features of C++ are well-implemented in any number of decent languages that don't obfuscate your code and incur horrible link times. And my argument (for game development) is that C++ is indeed the wrong language for the domain. In fact, I'd argue it's wrong for most, if not all domains. And I'm not exactly alone - I can't recall any prominent figure that actually thinks C++ is a decent language, except Bjarne Stroustrup. (Correct me if I'm wrong - I'd love to hear about it!)
<blockquote> If you want them attributed to the callee, it's pretty simple. Do the coercions in the callee, just like you would have in C. </blockquote>
The issue is that in many instances, I own the caller, but not the callee. And the person owning the callee can unintentionally make my code perform worse by simply changing the API. Or adding a destructor.
I think people are mostly blaming C++ instead of bad programmers.
Let me ask you this: Have you ever worked on a large scale C++ project that didn't have performance issues? Or memory issues?
APIs can easily lose performance when refactored for correctness or code clarity. Large projects have large project issues!
And yes, large projects have large project issues. It is kind of telling that C++ needs an entire tome on large project issues, though... (Lakos, Large Scale C++ Software Design)
I don't understand this at all. The performance of a C function can change without any change to the function signature. Even the destructor scenario can happen invisibly in C code for any type that already has a cleanup function. No matter whether the cleanup function is defined as "int myproj_FooCleanup(struct Foo* foo) {...}" or "myproj::Foo::~Foo() {...}", you won't notice implementation changes until runtime.
Also, just adding a virtual destructor (without any functionality) adds cost. And unfortunately, you need to if your class happens to become a base class.
The issue is that in many instances, I own the caller, but not the callee. And the person owning the callee can unintentionally make my code perform worse by simply changing the API. Or adding a destructor.
This is just bad programming practice on their part. The performance cost of a destructor shouldn't be a surprise to the person who writes it. If someone is messing with the performance of types you use or removing functions(+) you call without consulting you and without themselves taking responsibility for calls to that functionality, then obscure features of C++ is just one of the many ways they're going to screw you on a regular basis. How would those programmers screw you in C? I don't know, but I know they would.
Nobody who knows C++ well enough to use it effectively would call it a "decent" language in the sense of "not obscene," but it is definitely "decent" in the sense of "adequate." It's easiest to define C++'s deficiencies with respect to other languages (for instance, memory management in C++ is much more intrusive in source code than in Java) but comparing it to C is way too simple. After all, the primary problem with C++ is that it doesn't remove any of C's dangers; it just gives you better ways of abstracting them away in some cases. The second problem with C++ is that it's extremely complicated and takes a long time to learn. And that's it. Every other supposed misfeature of C++ (relative to C) seems to stem from people hitting the second problem without realizing it. In your case, your coworkers should be more conservative about messing with things (such as defining expensive automatic type conversion functions) when they don't understand the consequences (such as those type conversions actually being invoked.)
(+) This is the most plausible explanation for different type conversions suddenly being invoked. It could also happen by adding a function that invokes a more expensive type conversion to a type that is more closely related to the type declared by the caller, thus making the new function a better match than the old one, but it's unlikely that a conversion between two types that are more closely related would actually be more expensive.
That was Stroustrups reasoning: "Only pay the cost when you use it". In retrospect, believe that that's a bad choice for language design, because there's a good chance somebody will use a feature without fully understanding the ramifications it has over the entire code base.
C opts for the opposite and uses annotation to treat certain code segments as "special" and make them faster - "inline", and (in the distant past) "register" come to mind.
(or at least the cons)
I believe I do get the pros as well, at least to some extent. I've been using it since CFront came out ;) And I'd still advocate it for use in a small team (<5 people?) of experienced programmers - you can make it sing.
But gamedev engineering teams are at least 15+ people, with the occasionally less-experienced ones thrown into the mix. C++ is, in a sense, like a professional power tool. It sure can get stuff done, but the wrong person uses it and its a blood bath, and it's not appropriate for single small home repairs. (Gah. Bad analogy, but I can't come up with a better one right now)
This is just bad programming practice on their part. The performance cost of a destructor shouldn't be a surprise to the person who writes it
Possibly. I'd argue that it's hard to always keep all performance ramifications in mind. But let's for a moment say it's indeed simply bad programming practice - the issue that makes this a problem is that my code pays the cost, not the offending code. Which is a rather indirect.
C++ changes often have tendrils all through the system. It is easy to make inadvertent mistakes.
* but it is definitely "decent" in the sense of "adequate.*
It clearly gets the job done, yes. We are shipping games occasionally, after all. I'm looking for better ways to ship games, and I think that abandoning C++ might bring gains.
The second problem with C++ is that it's extremely complicated and takes a long time to learn.
That's what my hidden cost is referring to - C++'s complexity makes it hard to understand. You certainly can write code that performs well in C++, but it is arguably harder to get it right than C.
I like to think there's a better solution than either one hiding somewhere, but I haven't found it yet.
The not-so-hidden cost of C is that even simple things end up being many, many lines of code
>C++ was invented because certain kinds of C programs -- programs that were already being written in C -- were painfully verbose to express and complicated to change
That is certainly true, but I think C++ has outlived its usefulness and is coasting on momentum.
> Anyway, the size of the mental model of your program is what you should be worried about.
If you can keep 2.5 million lines of code in one single mental model, congratulations. I can't. I don't know anybody else who can. Game code bases, for better or worse, are fairly large, so we need to reason about subsets of it.
This is where the C++ problem comes to bear - I have no direct control and knowledge of all classes involved in a particular computation. Yes, I can look them up - but not everybody does. And I'm concerned with building shipping products with a normal team, not some hypothetical team of superstars.
This is where the C++ problem comes to bear - I have no direct control and knowledge of all classes involved in a particular computation. Yes, I can look them up - but not everybody does. And I'm concerned with building shipping products with a normal team, not some hypothetical team of superstars.
There's no difference in time or difficulty between looking up a C function and looking up a C++ member function, constructor, or destructor. Working on a codebase full of C++ language constructs and acting like only C language constructs matter is passive-aggression and should be treated as a morale problem, not a programming problem. You don't need superstars, just normal programmers who are willing to learn the language features used in the code they're hired to work on. C programmers who are happy to take a job using a different language but refuse to actually learn the language is such a 1990s problem. It shouldn't be tolerated anymore. It was a big problem in the Java community, with predictable results, and resulted in all kinds of ludicrous complaints being leveled against Java (lack of macros was raised many times as a fatal objection to Java -- there were so many things you simply couldn't do in Java because of the lack of macros), but the problem was solved -- people learned how to recognize and not hire those people.
There clearly is. In my hypothetical C example, there is one function I need to look up. In the C++ example, there are up to four classes:
* Anonymous constructor class * Class for each in-parameter * Class for the return value.
just normal programmers who are willing to learn the language features used in the code they're hired to work on
The problem is that the set of language features in C++ is huge. The issue is not programmers who don't want to learn, but programmers who haven't learned the whole language yet.
I can't solve that in the hiring process unless I exclusively rely on senior-level programmers. And those are hard to come by...
magic()
What's that? It's a function call. What does it do? Anything. Everything. Why does it need a complex grammar? It doesn't. Spend your time thinking about the problem, not the language. What a concept!
Do simple things take many lines of code in C? Yes. If you only have a few data structures and algorithms in your mental toolbox, simple things will take many lines of code in C, or C++, or in any language. Why do people advocate baroque languages by claiming that you cannot do simple things simply except with baroque languages, and then blame people for complicating their programs by using that baroqueness?
Yeah, C++ programmers are feature junkies; the language is designed by a feature junkie and promotes language features as the solution to every problem. If you go against that, you are going against the whole culture.
If PL/I was the fatal disease, C++ is the shambling corpse.
In fact the entire language is designed with care, which should be obvious to anyone who has read The design & evolution of C++ or has followed C++'s evolution.
But kidding aside - the constraints imposed on C++ during its design (C backwards compatible) combined with a desire to have every feature under the sun available has led to an overly complex beast.
Don't tell me you looked at e.g. C++ lambda functions and thought that was "good design". It gets the job done, but that's the best you can say about it.
Yes you have to know what you are doing in C++. But this is not different in other languages. Understanding and optimizing code is a touch discipline. No matter what language.
There is also another side to this story. C++ can absolutely generate highly efficient code. Code that performs much better than C equivalents.
For example, I've looked a lot at generated (iPhone/ARM) code for cases where (function) templates are used and C++ wins most of the time. Doing compile-time configuration of code is usually much more efficient than having runtime-configured code. By using templates and inline functions the compiler can do some seriously cool performance tricks.
Sire, there may be more code (bigger text segment) but that is a small sacrifice for what you gain in speed. Specially these days where a 8MB fart app is easily downloaded over a 3G connection without hesitation.
The problem with C++ is not that you have to know what you are doing - you also have to know what everybody else is doing.
And the "small sacrifice" of a bigger text segment is a big one in this particular domain. We only have 512M available, and a tremendous chunk of that is eaten up by graphics. An executable seriously exceeding ~20MB would spell trouble.
And I'd argue that AAA games are slightly more complex than a fart app ;)
Instead, we follow modular programming practices. If some interactions are too slow, we know who didn't optimize his section and we kick the problem back to him.
So it's not always that easy to just find a guilty party and blame them.
This post about not knowing how a function is called is really splitting hairs, bordering on irrelevant imo -- sure, it can affect the time taken to call a function by ten times, but this is not a large performance hit at all for 95% of your code, and even less since most high performance items are on the GPU. If it ends up that it does affect performance in a critical area (found via profiling), the author even suggested how it can easily be fixed.
I don't see what the big deal is. If you're a game programmer, change your coding conventions so that you don't get into problems like this.
You don't write sloppy code and argue that you can find bugs in the debugger, either, do you?
And no, most high-performance items are not on the GPU. Most games are already GPU-bound, so you want to take load off of it. (Also, with the arrival or LRB et al., the GPU/CPU distinction becomes moot)
If those issues were irrelevant, I wouldn't care about them. There's one thing I do care about: Shipping high quality games, developed as fast as possible. C++ hurts that goal.
And no, the 95%/5% rule does not hold true in game development. But don't trust me - read, for example, Tim Sweeny, the guy behind the Unreal engine. According to him, there are not hotspots. Another Unreal programmer: http://lambda-the-ultimate.org/node/1277#comment-14238
"Dan Vogel, was fond of pointing out that UE2 had a basically flat profile"
That is the big deal. In a flat profile, things like calling overhead matter everywhere.
Shipping high quality games, developed as fast as possible.
If this is what you care about, then you should care about optimizing AFTER you write the initial version and identify hot spots.
You don't write sloppy code and argue that you can find bugs in the debugger, either, do you?
Sloppy code vs. "slow" code are vastly different. I would much much rather have slow code, since 99% of the time it isn't even noticed by the user. Hence, it was worth not thinking about how to improve it.
most high-performance items are not on the GPU.
I agree that you want to take load off the GPU. However, CPU speed has not been the limiting factor in games for a LONG time. In fact, in conferences such as the GDC it is rarely discussed any longer, only in the context of using more CPU if available for non-engine tasks such as AI.
And no, the 95%/5% rule does not hold true in game development.
There is a big difference between what is essentially a graphics engine (you reference the Unreal engine) and a video game. If you look at modern games, do you think it is a waste that they are based on interpreted scripting languages? They can do this because as I said above, CPU isn't as limiting a factor as before.
Not really. Some amount of up-front investment can avoid a lot of pain at the end. Do I obsess about every single call? No, of course not - but I try to avoid obvious performance issues.
While you can optimize them after the fact, it's more time you're spending.
_CPU speed has not been the limiting factor in games for a LONG time_
Yes, actually, it is. It might not be raw speed - we're talking cache misses instead - but you still have to be careful with your resource usage.
_There is a big difference between what is essentially a graphics engine (you reference the Unreal engine) and a video game_
Yes, and no. Unreal is more than a "graphics" engine - it's an entire game engine. (There's way more than graphics there.) And CPU performance still is discussed at GDC. See here, for example: http://www.scribd.com/doc/15118967/Hitting-60Hz-in-Unreal-En...
And the games built on top of it use some scripting. And often dive into C++ and even the engine itself to make the game actually perform well.
But that's exactly the model I'm advocating. Write the kernel in a low-level language (preferrably not C++, for many reasons, some of them mentioned in my post), and do the game logic in a scripting layer on top.
The faster you can get the kernel, the more resources you can devote to a language that makes life much easier for the rest of the world.
So yes, I wasn't exact. When saying "game development", I was referring to "engine development". The point still stands - I think C++ is inappropriate for low-level tasks (too much "magic") as well as high-level tasks. (Incredibly convoluted language. No, I'm not saying Stroustrup is a bad designer. I'm saying the constraints he imposed on himself forced C++ to head that direction. And design by committee helped.)
If you look at modern games,
I do. Every day. Intimately. I work on them. That's why this topic matters to me. It'd be kind of stupid to complain about C++ usage in game development without knowing about it ;)
If you think C++ is not overly complicated, just what is a protected abstract virtual base pure virtual private destructor and when was the last time you needed one? Tom Cargill
You either know the code you are working with or you don't. C++ or not.
Calling func(x) in C may as well be causing x to be cloned or have some other nasty side effects that are not immediately obvious from looking at the code. Similarly how one should always keep in mind the existence of macros when reading C code, one should also keep in mind the existence of constructors and casting operators with C++. That's hardly the hidden cost.
It is easier to write messy or obscure code in C++, especially for a novice programmer. But this has less to do with the language per se and more with established coding practices within specific team. If there is a hidden cost to C++, it is the fact that it requires higher level of competence. Not that it has constructors.
What I can't tell is how much extra code the compiler will generate at this point, and that is the problem.
> If there is a hidden cost to C++, it is the fact that it requires higher level of competence. Not that it has constructors.
I didn't say the cost is in the constructors. They're a symptom of the cost. The cost is creating the mental model of your app.
Sure, if the entire team consisted of C++ superstars, we might be able to get away with this. Given the team size on current games, (and the fact that superstars are rare) that's unlikely to be the case.