How Much Does a Compiler Cost? | Hacker News Reader

How Much Does a Compiler Cost?

embecosm.com

129 pointsingve70 comments

As a previous compiler developer, its a pretty bad idea to look at the final lines of code to determine its cost.

Typically in compiler development, there's massive R&D time, many hours spent analyzing memory & CPU patterns to figure out which code to emit. You also take a look at disassembly of generated code and nowdays perform data-mining to figure out which patterns of optimize for (typically on non-kernel or hotspot-heavy type workloads).

The end result of alot of investigation can be few lines of code which have to be absolutely robust and production ready at the time of release under standard configurations (i.e -O1, -O2 etc...)

Its a costly piece of engineering which is why some companies still charge money for it.

I'd say the article's estimates are accurate for a simple compiler that emits results at say -O0 or -O1 level of efficiency, but anything higher will start to require more and more resources with diminishing gains over time. But this can be said about most things in engineering.

> LLVM is smaller at 1.6 million lines, but is newer, supports only C and C++ by default

Unless I'm misunderstanding "by default", this isn't true. LLVM is language agnostic and supports many languages - even the primary C-like frontend Clang supports more than just C/C++.

You are correct. The article is misinformed

Perhaps they mean that within those 1.6 million lines, the only frontend is Clang? But the line count for gcc includes e.g. gfortran as well?

The GCC addons (gcc-fortran, gcc-cobal, and gcc-go) just convert the existing language into C that is just ingested into the standard GCC. It was only around 2008 that gcc-c++ stopped doing the internal translation/name mangling dance and gained native first party support [1].

Much like how the older versions of GCC used to just produce ASM files to be ingested into GAS to create the binary objects.

[1] https://lwn.net/Articles/542457/

The link you posted has nothing to do with how GCC does codegen. It is about GCC transitioning itself from being written in C to being written in C++.

GCC IR is not C

Yeah, I thought that part of the point of the LLVM project was that front end language compilers (entirely language agnostic) could target non-platform specific LLVM IR code, and then let the backend LLVM compiler compile to multiple backend targets. It simplifies front-end compiler development by having a single IR target over multiple native target.

> LLVM is language agnostic

There exists no such thing as "language agnostic". Every intermediate representation contains properties (very often implicitly) that are specific at least to a class of programming languages.

If the intermediate representation is no more language-specific than x86, then it isn't really the compiler that's language-specific: it's modern computer architecture.

LLVM IR is fairly C specific. For instance, you better not be trying to write a language where null pointer access is defined...

We compile D with LLVM. Null pointer access is defined to crash, unlike C/C++ where it is undefined.

Are you adding explicit checks (possibly using fault maps to optimize them)? Because if not, you're depending on undefined behavior.

That is only necessary for single objects that are bigger than one page which, we dont currently do (but should). Arrays are no problem since we check the bounds, only raw pointers cause problems.

No, it is necessary in all cases. Look at the InstCombine code I linked in a sibling comment…

This post is correct. Load of a null pointer is undefined behavior in LLVM.

If you don't believe me, here is the code that optimizes based on this.

https://code.woboq.org/llvm/llvm/lib/Transforms/InstCombine/...

Or maybe another way to put it would be that the given language is hardware-antagonistic.

Sure, but the supported class of languages is very broad and certainly not limited to C/C++.

Those languages are however supported mostly by (accidentally or deliberately) being similar to C/C++. Not only does the IR need to be able to express certain constraints of the language semantics (IIRC, there have already been changes to better accommodate Rust), but the kinds of IR patterns the compiler frontend generates might not get optimized if they are too un-C-like.

LLVM is targeted by both Haskell and Idris - two of the most un-C/C++-like languages I've ever come across.

I was curious about how that works, so I searched a bit and found this description of how GHC's GC works in LLVM https://www.reddit.com/r/haskell/comments/3qix1e/how_does_gh...

Some highlights:

Now, before GHC generates assembly code (abiding by the calling conventions previously described), it also needs to optimize it. It optimizes a form called "Cmm", which is like a very low level compiler language for performing optimizations. This language does include functions.

When you use the LLVM backend, GHC translates every Cmm function to an LLVM function. In order for everything to work out, we patched LLVM so that functions can have an annotation saying they follow the GHC calling convention.

...

What this means is, we have completely side stepped LLVM's support for GC.

So the support mostly comes from the Haskell side, by compiling to the C-like Cmm (I think that name comes from C--, the idea being a language between assembly and C in abstraction level) and patching or otherwise working around the part where LLVM's feature set wasn't appropriate.

Note that mentioned LLVM patches are now upstream. http://llvm.org/docs/LangRef.html#calling-conventions documents "GHC convention", for example.

We have worked with LLVM a lot for own own compilers and DSPs.

LLVM Intermediate representation in practice is C++, AST is C++. LLVM do not support many languages,many languages support LLVM, which is different.

That is many people, or languages' designers, have made the necessary work to make their languages compatible with LLVM IR. If the language is similar to C++, this is a simple task, if it is very different, it is very hard.

LLVM does not support other languages, in fact, codebase changes a lot making maintenance painful.

Could you elaborate on the C++ resemblance?

Specifically, in my limited experience LLVM IR resembles typical hardware rather than C++. Modeling the hardware is a design goal for C++, of course, so anything hardware-like must be C++-like. Can you describe some ways in which IR resembles C++ that are not plausible ways to resemble hardware? I'm really curious.

In a decade Cygnus spent well over $250 million on GCC and its related tools -- well our customers did. Not sure what that is in today's dollars.

Can break down where the money likely went, how it was spent?

I remember writing code for Psion (later Symbian) Epoc32 in the late 90s and early thousands. It was a big deal that Cygnus had done the work to make gcc support ARM because the vendor compiler was (IIRC) some weird combiniation of expensive and not very good.

We now assume ARM as being a default necessary platform to support, 20 years ago it really wasn’t clear

This was between 1989 (when we started in my living room) and 1999 when I left. By 1999 we had about 160-170 developers working on gcc, gdb, binutils, glibc, Cygwin etc. The money was a mixture of 1> ports (typically paid for by the manufacturer), 2> maintenance (mostly paid for by the corporate end user -- for $100K/year we'd fix your bug whether we spent $100 to make a trivial fix or $300K on a major problem) 3> maintenance on very old versions that people had to freeze on for one reason or another and 4> the public tree (we wouldn't work on non-free software, apart from a few abortive experiments after I had left).

Making sure everything we wrote was compatible with the public tree was expensive. In the end, one of the last things I did saved a lot of money and improved code quality for everyone: I forked gcc from the FSF (and made the egcs tre the de facto, and eventually de jure version).

Nowadays forking is considered routine, and even beneficial, and projects have steering committees that are responsible for master releases. At that time, forks were considered a tragedy, and it took me over 6 months of discussion with various people to build a reasonable consensus (you can see the people's name at the bottom of the letter I wrote announcing it ( https://gcc.gnu.org/news/announcement.html ). It's been a good model. Note that RMS's name is not on the letter -- he was furious and was sure this would destroy the FSF. Instead it has strengthened it.

By the way, to compare these numbers: Red Hat went public with, AFAIK less than 10 engineers and when Cygnus and Red Hat merged, though both companies had about 200 people, Cygnus had over 160 engineers while Red hat had a couple of dozen. Of course we had a lot more revenue, and in the years since RHAT has invested heavily in free software development. But their company value at that time was something like 3X ours -- a very valuable lesson.

How did Cygwin come about?

http://www.toad.com/gnu/cygnus/

Maybe I'm misreading that, but it doesn't say anything about Cygwin, per se, just about Cygnus, the company.

Steve Chamberlain (our second employee) needed to run some code on Windows.

I find it odd that they did an LLVM port and even another blog about dealing with 16bit char types, but none of the target specific changes are going upstream. I can understand a company not wanting to help their competitors, but it sounds like they're still going to try to get the 16bit char changes pushed upstream, which is likely one of the more useful pieces for a competing DSP vendor with a different instruction set.

In 5 years we'll all be using risc-v with a nice DSP extension anyway... And after typing that I felt the need to google: https://riscv.org/wp-content/uploads/2016/07/Wed1000_dsp_isa...

Heh. Remember when you had to actually buy a compiler? :)

People still buy compilers. IAR is [1] still the gold standard in the embedded space.

[1] At least it was 18 months ago when I still working in the space, can't imagine that's changed since.

IAR is still very popular. However, if you need to support esoteric microcontrollers, there is usually no other options but to purchase it.

In the scientific / numeric space it’s not unheard of to buy Intel Fortran/C compilers for performance reasons.

Also, it’s been a few years since I touched IBM’s POWER hardware with AIX, but I’m pretty sure XLC and whatever the Fortran compilers were didn’t come free.

My organization uses the Intel Fortran compiler. I did a few rough benchmarks at one point and found it produced code nearly 3x faster than the same code in gfortran.

I suspect the exact speedup depends a lot on the workload, but the last time I supported scientists on HPC, they reported pretty substantial performance gains with ifort over gfortran as well. But then Intel has deeper pockets to pay for this kind of thing than the FSF.

I think I also recall something about gfortran deliberately choosing some more conservative optimization strategies, which could also explain some differences. Or I could be misremembering absolutely everything :-)

Much large scale computation on a typical general HPC system is quite insensitive to the compiler, with most time spent in maths libraries and/or MPI. Most scientists compiling code don't know anything about performance engineering, or even measuring things, unfortunately.

GCC does correct optimization by default (modulo bugs). ifort's default set of flags on GNU/Linux at least used to be non-standard-conforming (maybe fast-math?). Comparisons need to use equivalent flags for gfortran. I don't have the ones I once noted to hand, but they'd include -Ofast -funroll-loops -march=native. The ifort default also generates static binaries, which require static libraries and which you typically don't want so that you can do profiling, for instance.

Hmm... maybe it was the Intel linear algebra stuff (MKL?) that was the source of that speedup

Maybe. For what it's worth, OpenBLAS is at least as fast on all AMD64 micro-architectures before AVX512 that I've had access to. That's for DGEMM, which is what normally matters; MKL may well have optimized more kernels than OpenBLAS has. Unfortunately, many cling to performance myths in the face of measurements.

Yeah, my last experience with paid MKL libs / Intel compilers was some years ago now—before I ever remember seeing anything about OpenBLAS. I know OpenBLAS has been a big performance boon.

And to be clear, I’d love to see Free and Open Source options be the standard for HPC. It sounds like we may be much closer to that state today than we were years ago when last I worked with people highly concerned about performance in this area. That’s great news.

I hear this a lot, but if it's with equivalent optimizations it's quite pathological and should be reported as a bug; it's certainly not generally the case. They are typically similar at the 10-20% level in my experience (not always in ifort's favour). There are several significant HPC codes which recommend GCC where it matters. You also need to consider reliability; GCC is surprisingly better in my experience -- I've quite often solved users' problems with "just try GCC".

There used to be a published set of results from Polyhedron -- presumably to help to sell proprietary compilers -- which had a geometric mean favouring ifort by ~20%, if I remember right. However, the last I saw didn't use the latest GCC, required the same flags for each case (which didn't include profile-directed, for instance), and the cases with the biggest differences actually bottlenecked on libraries, not generated code. gfortran is infinitely faster on most architectures, of course.

Most benchmark results I see aren't useful because they don't supply the parameters needed to be reproducible and they don't have profile information to allow you to understand, and maybe improve, them.

I found that results from the Polyhedron benchmarks are online again <https://www.fortran.uk/fortran-compiler-comparisons/polyhedr.... Note that they compare gfortran from 2015 and ifort from 2017. I don't know how representative they are of the part of the workload on typical HPC systems for which the compiler might be important.

I wish Intel would make their compiler free, or work harder on integrating their optimizations into free compilers. They can't be making much money off it, and it just feels counterproductive to charge customers extra to get the best performance from their own hardware.

Maybe their optimisations reveal proprietary information about how their processors are implemented that they don't want to disclose?

Maybe people would flip the RUN_SLOW_ON_AMD flag to false. :-)

Anecdotally, changes to ICC are evaluated (by management) on two metrics: improvement on Intel CPU, but also improvement on Intel CPU minus improvement on AMD CPU. The end result is similar to RUN_SLOW_ON_AMD flag.

If you think this cheesy, consider this: you are a manager of Intel compiler team. Someone implemented an optimization that improves performance by 1% on Intel, but 10% on AMD. Do you think it is a good change? The change will be no-brainer on GCC, but consideration for ICC is necessarily different.

Source?

For ARM devs, Eclipse+GCC is taking over. IAR's IDE is horrendously primitive and crash happy.

Yes. I started using FreeBSD and Linux (which both came with gcc) because I couldn't afford Visual Studio in the late 90s.

I bought the Borland C++ compiler as a kid.

Now I want to write compilers and developer tools, but there's no money to be made.

> Now I want to write compilers and developer tools, but there's no money to be made.

It's sad. One the one hand it's great that we have access to so many amazing compilers for free. But now there's a monopoly of compilers - and by extension, programming languages - by large corporations. They're all kind of the same thing too - vaguely C-like statically typed ALGOL languages with OO and FP features bolted on top. Swift/Kotlin/C#/Java/Typescript/Go, it's all much the same.

I don't think anything novel (a smalltalk, prolog, haskell or scheme) could get any mindshare today, unless a big company lost their mind and tried to push something weird.

Microsoft's Visual Studio division and Jetbrains would probably not agree with you. Neither would Atlassian :)

I think there's money to be made, but there's a lot of competition and you need a very specific niche to break through.

Yes, I bought a few Turbo Pascal and Turbo C++ editions.

Nowadays my company pays volume licenses for MSDN.

Also, I would argue that when one buys an Apple computer, given the price difference to PCs, the price of the development tools is included.

I remember myself and others learning all some terrible practices like avoiding stack usage in c++ because the free versions had limited stack sizes.

I also remember dad having to drive me to a university in a bigger city so that I could buy VC++ from the university there.

I wish I knew about linux back then.

> For a commercial system, the compiler has to be completely reliable—whatever the source code, it should produce correct, high performance binaries.

Well, that's certainly the ideal case. I can only wonder what historic defect rates have been for commercial compilers.

Not that it answers the question, but here are some VC++ bugs found in Chromium development: https://randomascii.wordpress.com/2016/03/24/compiler-bugs-f... . HN discussion at https://news.ycombinator.com/item?id=11361151 .

Red Hat tend to hide bug reports on their commercial (free software) compilers in bugzilla, unfortunately; I don't know what Adacore, for instance, or other commercial providers do, and such bug reports will be only a small part of it. You could obviously study the defects in released versions of GCC on which commercialized versions are based, though, to the extent they have PRs.

A great benefit of GCC was always enabling learning and research, though there are more opportunities these days.

Posts like this make me think of this: https://github.com/oriansj/M2-Planet A C compiler with support for inline assembly, gotos, structs and weighs under 2000lines and is self hosting.

I was hoping this was an analysis on how much time and money is spent while waiting for n number of lines of code to compile by various compilers. I find waiting for things to build, deploy, install dependencies, etc as the most mind numbing aspects of the field. I can hardly stand it.

I wonder, are there languages that are cheaper yet have similar features?

For example, how much is rust? python?

It's weird how a compiler can cost so much, yet it can still produce unsafe code because the language is not safe by design. In that optic, rust might save a lot of money.

You cannot produce a Rust or a Python compiler without a working C/C++ compiler first anyway, so the point is slightly moot. If there exists a C/C++ compiler for the platform you want to build for, just build Python for it and you're good to go.

You could produce a Rust or Python compiler without a working C compiler if you wanted to.

Yes, but then you are running into all the same issues that writing a C compiler would have, you're just skipping the middle step.

Does anyone know of any resources to learning about compilers and building a small, simple compiler for beginners?

https://en.m.wikipedia.org/wiki/Principles_of_Compiler_Desig...

I watched a couple videos from this and was good to go. https://lagunita.stanford.edu/courses/Engineering/Compilers/...

My strategy was to start with a very small grammar and grow it.

C/C++ compilers are different from other compilers. They are forced to support all kinds of complicated, too-many-cooks-spoil-the-broth, design-by-committee standards. If you just wanted to do another portable assembler, your new compiler would take just a fraction of the effort.

Reply on news.ycombinator.com