One of the features of Go is suppose to be compilation speed. Can anyone compare to C, for example, or discuss compilation speed in large projects (e.g. >100k loc)?
C compilers are fast, but they aren't that fast. Go compiles so fast that it changes the way you write and organize code; you would never consider compilation overhead in any decision you made in a Go source tree, where you might do that in a C codebase. For instance, Go programmers routinely compile code, run it, and throw away the compiled binary! You wouldn't think it would be possible for compiler speed to be so much better than C that it would make a difference, but Go's compiler manages it.
Okay, Go is fast, but it's not that fast. My 1000 LoC program takes ~1 second to compile. While it's certainly fast enough for me, it's not imperceptible.
If I wanted to spend the time I could do something like append the blob to the end of the program executable instead of just clumsily jamming it into DATA, but this all exists in a library that doesn't need to change often so the fact that it takes ~10-15 seconds to compile is not a problem in practice.
At least, Go team will be aware of your pathological case and have a chance to speed up it.
Maybe it'll work for your use case.
that sounds pretty interesting, is there a GitHub page I can look at ? thanks !
A nicer, more accessible (python/ruby/etc folks seem to like it a lot), faster compiling reasonable alternative to C is a worthy goal - I really don't get why some people get into flamewars over it.
I would think that, even before I had used Go. Ignoring the smaller scanning benefits of Go that come about from the more rigid syntax for braces and such, the C preprocessor system introduces a complete nightmare when it comes to requiring multiple passes over the same code, especially for code that is naive about include guards, pre-compiled headers and other "best practices".
C (and C++ is even worse) has a lot of very large low hanging fruit to cull when it comes to writing languages with faster compilers. Whatever speed C compilers have now is due to the geniuses who wrote the compilers and decades of shoulders to stand on, the language is actively working against them at nearly every turn when it comes to compilation speed.
That aside, I agree. Go compiles very, very fast. So fast that it never occurred to me that "go get" did a compilation until it was pointed out to me that it did.
I look forward to seeing the speed boost that C and C++ get if/when modules are available: http://llvm.org/devmtg/2012-11/Gregor-Modules.pdf
Sure, there are compilers that do a poor job at optimizing it, and it's very easy to make it slow on memory starved systems, but on a system with enough RAM to keep your headers in memory, and a compiler that doesn't totally disregard compilation speed, the overhead of skipping over blocks of include files using include guards is low and for code that doesn't, a compiler that can process multiple source files at the time can trivially cache large chunks of parsed headers.
As I've pointed out elsewhere, Bellard demonstrated TCCBOOT that used TCC to boot-time compile the Linux kernel in 15 seconds on a P4 - it's sufficient to demonstrate that it's difficulty in pre-processing or parsing C quickly that is the block for fast C compilation, but that to the extent it is even a real problem it is architectural decisions in the major C compilers and/or optimisation.
It's quite possible Go can be compiled faster, but so far I've not seen any compelling evidence that it is even roundly beating readily available C compilers. Certainly the numbers bandied around in this thread are not impressive.
I did some ocaml work a few years ago and had a similar revelation. Its compiler is just as fast—my short little programs were more or less instantaneously compiled. Fast compiling is really, really nice, even though it seems like a silly thing to tout.
C compilers that are written to be fast rather than flexible can be extremely fast. E.g. see Fabrice Bellards "TCCBOOT", that he demonstrated compiling the Linux kernel at boot-time in 15 seconds on a 2.4GHz Pentium 4: http://bellard.org/tcc/tccboot_readme.html
TCC can also be treated much like an interpreter like your go example.
It all boils down to having the parse each include file every time it appears, because the preprocessor might change the meaning of the file in each inclusion.
Yes, you need to pre-process and potentially parse them again, but in practice most C include files use include guards and are low cost to process if done even remotely properly.
This is why Plan9 C compilers have the strict rules that header files don't include other header files and you as a user need to include them all.
I need to look into TCCBOOT in more detail.
It is also perfectly possible to cache token streams or even pre-parsed (with some caveats) versions of the headers specialised by the value of used pre-processor definitions, which in most cases will prevent re-parsing of most include files (only causing re-parses when the pre-processor definitions actually change).
The pathological worst case for C code can be horribly nasty, but real-world code is generally fairly well behaved and easy to speed up processing of on modern systems that aren't terribly memory starved. Even on systems that are memory starved, there have been plenty of C compilers that can "pre-compile" headers to speed up compilation.
I assume you are aware, but tcc can be treated exactly like an interpreter:
#!/usr/bin/tcc -run -lgmp
#include<stdlib.h>
#include "gmp.h"
/* all this in eg. fac.c */
void factorial(mpz_t r, int n)
{
unsigned int i;
mpz_t tmp;
mpz_init(tmp);
mpz_set_ui(r,1);
for (i=1;i<=n;i++)
{
mpz_set_ui(tmp,i);
mpz_mul(r,r,tmp);
}
mpz_clear(tmp);
}
int main(int argc, char** argv)
{
mpz_t r;
mpz_init(r);
int n;
if (argc == 2)
{
n = atoi(argv[1]);
factorial(r, n);
gmp_printf("%d!:%Zd\n", n, r);
mpz_clear(r);
} else {
exit(1);
}
exit(0);
}
chmod a+x fac.c
time ./fac.c 1000
1000!:4023872(... snip for formating ...)000
real 0m0.007s
user 0m0.004s
sys 0m0.000s
edit #2: indentation/block formatting of code. edit #3: missing "time" prefix...
% make clean
% time make make-listener make-sender make-diag make-ssl...
3.42s user
1.05s system
105% cpu
4.240 total
% find src/ -name '*.go' | xargs wc -l | tail -1
17870 total
% make clean
% time make -j4 make-listener make-sender make-diag make-ssl...
5.75s user
1.99s system
382% cpu
2.020 total
% time make -j4 make-listener make-sender make-diag make-ssl...
1.56s user
0.73s system
209% cpu
1.090 total
(I should have not counted the tests in the above analysis. There are 7,290 lines of tests and 10,580 lines of code).
> Yes, we wrap the go build tools with make.
i.e. instead of issuing "go build foo/mylib" and "go build bar/myotherlib" in parallel, just issuing "go build foo/mylib bar/myotherlib".
I say this because I remember there being logic inside the go command to parallelize builds already, see the "-p" flag in "go help build". In particular, the go command will be able to avoid statting all the files, comparing mod times twice, recomputing the DAG, etc. Although, considering how fast it runs anyway, this speedup might be negligible...
: time go build -a
real 0m3.980s
user 0m5.012s
sys 0m1.140s
: time go build
real 0m1.854s
user 0m1.760s
sys 0m0.332sMy home server is an AMD FX-4100 (quad core, 3GHz; this is one of the early Bulldozer CPU's and definitively not a speed daemon - I picked up the CPU and motherboard for <$150 a year or two ago).
I just tested a 6k C program that compiled in 1s flat with gcc 4.7.2, or .44 seconds with -j4, and a 13k lines one that completed in 2 seconds, or .88 with -j4.
Gcc isn't exactly renowned for being a fast c compiler.
A "no op" build of the 13k project takes .004 seconds, though, perhaps that's the reason - if your make setup is horrendously complicated perhaps it's slowing your numbers down a lot. And it's certainly possible to make C compilation incredibly slow if you do lots of nasty pre-processor hacks.
But then again whether 6k, 13k or 18k lines, both your project and mine are tiny.
edit: s/tpl/tpu/
btw, the tp linker - http://turbopascal.org/linking-modules-and-creating-exe-file
http://prog21.dadgum.com/47.html
edit: in fact a number of the things that Rob Pike has brought up in talks reminded me of that article