Hacker News Reader

Comment by simonsaidit | Hacker News Reader

7 seconds slow.. I was happy when I decreased the 32 hours build to 4 hours.

7 seconds for a rebuild after a small edit is already a little unergonomic when you are iterating heavily. But it all depends on the project. Simple C-style code should compile at ~1M lines of code per second on today's machines without heavy optimization, and maybe compiling on 4 cores or so. (Not possible with common infrastructure like C, gcc/clang/msvc, large headers per compilation unit, many small files, object file style linking! and the awfully slow linkers (I wonder if they could be faster))

I knew one guy who used C++ but hated STL and most of standard library. He usually implemented data structures in-place as he need them (linked list, growable array, etc). While it seems strange to reimplement similar data structure over and over again, it worked for him. And one particularly wonderful thing was how fast his projects compiled. He had one project with few libraries, around 100k lines and it compiled very fast, something like second from the scratch and it was 10 years ago.

That's the way I learned C++. While I mostly use the STL now as it's much more convenient to write, there is an elegance to writing one's own tools.

you'd also be shocked, SHOCKED at how inefficient some of the STL implementations of basic data structure algorithms are

I've almost purposely avoided studying what the STL does because of that. I have some of my own notions on how some of the data structures ought to be built in order to be efficient, and sometimes I implement them myself to be sure I get the behavior I want.

Of special note is the unordered_map/unordered_set implementations.

https://news.ycombinator.com/item?id=7849213#7849607

imagine all the CS research that relies on the STL expecting it to behave optimally

> ... should compile at ~1M lines of code per second ... > Not possible with common infrastructure

If it is not possible with common infrastructure, what do base your estimate on?

My own experiments. I have recently bought a new PC with an i5-7600 and my own pascal parser, written in straightforward unoptimized C, parses about 2M lines of code per second on a single thread.

Also I'm following Jonathan Blow's streams (Jai language). With his own trivial x64 backend he has/had a very real compilation speed of 80K lines/s, and only the parser was parallelized. I think he indends to improve that to 800K lines/s. Note that his language is quite a bit more work to compile than a basic C-like language (which is about as easy as you can get if you discount the parsing model).

The linkers could definitely be faster! I've seen reports of Rust debug builds going from ~6 seconds to ~2 seconds by moving from gold to lld

Seven seconds is already pretty fast. Might have warranted some profiling and a brief attempt to speed up the parts of the compiler that your usage hits, since that could potentially be useful to everybody and thus magnify the value of cutting one second off a build. But this? Was a cool learning experience, no more, no less.

7 seconds is plenty fast enough to make “watch” the cheapest and most reliable solution to the speed problem.

I'm not sure what you're talking about with respect to the C compiler. C compilers are the fastest around even before being paralllelized - and they can be.

C parsing is rather slow due to the preprocessor (also: huuuge include files) and due to context-dependent syntax (need symbol table). And compiling itself need not be slow, but the popular compilers are not exactly super fast even with -O0. LLVM in particular has a reputation of being somewhat bloated. And finally, due to common practice of using a standard object file format, which is slow to link (at least with common tools), the whole build experience is not exactly blazingly fast.

With clang/gcc/msvc, I think I'm more in the ballpark of 3K-30K lines/sec (-O0, 100-1000 lines/file, not counting basic std* includes).

gold is pretty fast. I don't have a source but those numbers seem very off to me. C compilers are fast. C++ much less so, but C++ is a different language.

I tried to follow gold's author's series on linkers once but gave up. Now that I've written an ELF-64 writer and know a little more about linking I should give it another shot. In any case, I think I've heard of still faster linkers than gold, and at the time I also found gold's object oriented architecture (as it was described in the series) questionable.

gold is sure fast compared to bfd.ld, but then you should try lld from the LLVM project which is much faster in general.

Do not measure other people problems by your scale.

Reply on news.ycombinator.com