Clang vs GCC benchmark on UnrealEngine3
plus.google.com
plus.google.com
Things change with -Ox, but then again, who really cares? Generating optimized binaries is not something I do routinely, except when tuning for performance.
When I'm generating a binary for release, I couldn't care less if the project actually took 2x times to compile. In fact, to use feedback-directed optimization (which on some projects may make a marked difference, usually larger then the generated binary at O3 between gcc/clang), you actually have to build it twice, and _very_ few people actually bother.
clang used to be much faster at building, but then again the binary was never equally on-par performance wise. As clang is getting better at optimizing, the compilation times are getting more and more similar.
This is again, all hardly surprising for me.
I _have_ worked on projects using C++ where build times where incredibly slow. They all shared the common attribute of #include <megaheader_with_all_class_definitions> and "using namespace library".
For other projects, as long as rebuilding an object takes 2 seconds or 10, does it really make a difference? In those cases, linking actually takes disproportionately longer, to the point that compilation times are insignificant.
It's a genuine question.
But my perception reading these articles is that the tone is different. There's seems to be a "huge problem with compilation speed in c++", that's also "a detrimental factor to using c++ itself" that I _never_ really experienced, except, like I said, in badly-organized projects.
It always felt to me, by programming, that writing and debugging were _always_ the problem, never compilation itself. By reading these articles, it feels like I'm either too slow, or there is some magic that I would need to be doing to use the compiler 20x more than what I'm doing now.
It happened to me _very_ rarely to change a global header that would cause a massive rebuild. Most of the time I would rebuild just the current object I was working on, taking anywhere from 10 to 60 seconds.
Yet, linking took massively longer (around 4 minutes as I recall).
For CI builds ~ 10 mins is nothing (we're currently hitting 3 hours and managing fine with a rats nest of csc and cl compilers being invoked galore).
What really brings the shit to the table with clang is simply this:
Compiler error messages are beyond awesome as is checker.
That is all. Productivity is way up for me. I do my own clang builds of stuff because it's a lot more helpful as a compiler.
I think not.
The difference will be between 1s and 2s, given overall difference in speed. I suspect that your time to think about problems will two orders of magnitude more.
Mine is - minutes between builds.
AFAIK Clang and GCC are now roughly neck-and-neck in terms of performance though (at least as of December) - http://www.phoronix.com/scan.php?page=article&item=llvm34_gc...
At least I'm not aware of being able to type in c/c++ in gdb on the fly like you can with lldb contexts.
So what exactly is "hip and cool" about having your debugger actually use the same parsing engine as the compiler infrastructure? If you have a technical point make it, otherwise stop being the discussion equivalent of acid.
Even being able to reliably evaluate C++ expressions is far from what I want: I want to be able to execute arbitrary C++ statements (including loops), use templated objects that weren't instantiated at compile time, etc. I feel like this might be close to possible using, say, the LLVM JIT, although I haven't seen anything that implies anyone is trying to build it.
edit: for example (stock gcc/gdb on Ubuntu 12.04):
#include <iostream>
int main() { std::cout << "foo\n"; }
(gdb) p std::cout << "foo\n"
Cannot resolve function operator<< to any overloaded instance
(lldb) p std::cout << "foo\n"
error: no member named 'cout' in namespace 'std'
error: 1 errors parsing expression (lldb) expr for (int i=1; i<5; i++) { printf("%d\n", i * i); }
1
4
9
16
It still fails on a lot of things for me (including your above example and the std::cout example, although variants work), which I'm guessing (hoping?) is just a lack of maturity. Looking forward to see how it progresses.
What is this test really trying to determine? How the performance of the core of each compiler impacts productivity? Shouldn't the test parameters be adjusted to real world conditions, as seen by the test creator, then?
Like most benchmarks, you could fabricate scenarios that favor one or the other all day long. Some closer to reality, some completely synthetic.
250MB of output adding 5mn of compile time would mean 800KiB/s. That's about what a 5400RPM HDD gets on random writes. Possible, but means GCC has an utterly terrible I/O architecture and flushes to disk after more or less every write call.
> What is this test really trying to determine?
Nothing? It's just trying to infirm or confirm the initial off-hand remark: https://twitter.com/icculus/status/448281959538888704
> Like most benchmarks, you could fabricate scenarios that favor one or the other all day long. Some closer to reality, some completely synthetic.
I'm not sure how you can get closer to reality than "I spend my days compiling Goat Simulator with Clang and GCC, let's compile Goat Simulator with Clang and GCC".
Very possibly IO, yes.
And it is twice as big for a debug build, whereas a non-debug build is smaller in gcc - 69 MB vs 74 MB in clang.
Perhaps more importantly, not all debug builds are equal, so the comparison is not apples-to-apples without saying whether the gcc debug builds contain more useful information. gcc also has various levels of debug info, perhaps a different one would provide less info for a smaller binary that would be more of a parallel to clang.
Is this all program code or does this include resources?
I wonder what performance penalties arise from that.
The code size would still be massively bigger than the CPU code cache, but the working set that actually matters for performance does not have to have any relation to the size of the total code itself, as long as time-critical sections are small and localized.
C++ templates, while being absolutely lovely to write and use, tend to do that. Every single different type you instantiate the template into produces a separate compiled copy of the whole template. Write a sorting function and use it for 5 types -> you get the same sort in assembler 5 times with differences that relate to the types they use, alongside with 5 massive symbol names (C++ template symbols tend to be really large).
To be fair, this is also why the performance of C++ std::sort beats C qsort hands down. Calling the comparator function with a function pointer is expensive compared to doing an integer comparison. Additionally, the boundary between translation units (ie. object files) inhibits a lot of optimization that could otherwise take place (and does in the C++ case, when the function calls are inlined).
There are also techniques to avoid C++ template bloat, such as type erasure. Although this makes the template code only a thin shim that provides type safety.
The code for our linux game server compiles to an 11MB stripped binary with 608 megs of debug data.
The code for the windows game client contains the majority of that same code, but in addition also contains the entire renderer, UI, and a pile of other client side garbage and comes in at only 7MB with 86MB of debug data.
The 70MB for UE which would presumably be a much larger code base seems reasonable to me in that context.