So you want to be a compiler wizard (2016)
belkadan.com
belkadan.com
* Lambda calculus evaluator
* Hindley Milner typechecker for lambda calculus
* Stack based calculator (extend with variables)
* Play around with macros
* Work through Crafting Interpreters
But really in my experience the best way to get better at compilers (I can't claim to be a wizard) is to just build a goddamn compiler. Start by writing a parser (you can crib from Crafting Interpreters), writing the simplest possible typechecker for arithmetic (crib from Hindley Milner), then a code generator to whatever target you want. Code generation is tricky, but if you're generating arithmetic, it's really not that bad.
If at any point you get confused or have no clue what to do, take a deep breath and guess! Yep, just guess how to do it. I've done this quite a few times and later learned that my guess was really a dumbed down version of idk, solving the AST typing problem, or lambda lifting, or whatever. If that's too hard, scroll through other compilers and figure out what they do.
Once you get something working, celebrate! It's pretty cool seeing even arithmetic end up as code generated and run.
Then start adding other features! Add local variables. Add nested functions. Add whatever.
The secret is to treat a compiler as a challenging software development project. Because that's what it is. Start writing code and figure it out.
Granted this is not great advice if you want a usable compiler at the end. I'm just trying to learn, not make something for people to use.
I'm writing up a tutorial for my students right now. It is just a stripped down dialect of BASIC. I think seeing the entire compiler process from start to finish on a tiny language is what helped me the most.
The article is written with amazing (these days) clarity. Kinda coincided with my current interest in LLVM. Nice to see this article also be coming from UofI Urbana-Champaign grads.
Reportedly the article series misses implementation of one component - run-time, which is needed in order to fully replicate it now. Well, anyway, reading on!
Btw: HN thread on Tiny Pascal from 2018 https://news.ycombinator.com/item?id=17220507
Yup. In fact, the best way to start is to build a desk calculator, where you type in strings like:
4 + 6
The hairiest thing is usually overload resolution (both operator and function, if implemented). Determining the set of applicable overloads may depend in the types of the arguments, scoping rules, or generic instantiation if it's in the language, and determining the best overload depends on weighing up coercions, subtyping and polymorphism. Simple rules may lead to rejection of overload selection as ambiguous in situations where programmers don't want to define more overloads.
Here's code if you prefer that to slides: https://github.com/gergoerdi/hm-compo
I'd even go as far as saying the literature on the subject is overrated or out of date.
And not everything is set in stone. Not all compilers have a single symbol table. Sometimes it makes sense to have a type/global table and local table, and check the current class scope on the fly. Sometimes not. It doesn't really.
Abstraction doesn't make sense until you actually have multiple implementations. And you might have rewritten it four times by then.
My advice like above. Just start writing
And is there a large (e.g. exponential) performance penalty for large lookahead?
Performance penalty only comes if we backtrack in our hand written parsers. I don't think we usually do that.
Any glimpse of top-down parser would be simply too slow.
Handwritten recursive descent has a lot going for it, error messages in particular.
It can also support variable-length lookahead, which is really nice.
Front-end is not where the challenges are in any production compiler. The hard work is in the optimizer. Lexical and syntactic analysis are homework problems for sophomores. That is not where any of the value in a production-worthy compiler lives, no more than the value of your C++ code being found in the semi-colons at the end of the statements.
I am going to guess back-end, because there are a lot of self-study tutorials for front-end that you probably have already found. Unfortunately, I am not an expert in back-end, although at one point I did manage a group that was involved in compiler validation -- but I was pointy-haired, it was my team that knew the innards of the compiler.
Advanced texts in compiler construction are going to get into data flow analysis and liveness testing, and talk about basics of code motion. These are all elementary topics and barely touch on the state-of-the-art, but are foundational. Also, get good at reading the assembly language for the machine of your choice and look at the .S files.
Sorry I can't be of more help, but maybe I gave you some search terms.
I think there are a couple things in play here. Folks working with text, semi-structured data, synthesizing from disparate sources, etc will be front end heavy. Tokenization, lexing, is important outside of more than compilers, like loading binary formats from network or disk into memory.
For backend work, being able to extend or modify existing backends is important for languages targeting different runtimes (Spark, Beam, Impala). This can be in targeting new architectures or for predicate pushdown into data processing pipelines. Lots of different applications to use those skills.
Compilers and Database systems are an incredible microcosm of many areas of CS.
Areas of self study I think are nice are
MAL - Make a Lisp https://github.com/kanaka/mal
Nand2Tetris, project 11 https://www.nand2tetris.org/project11 (one should start from zero and make your way here, it is journey not a destination)
An educational software system of a tiny self-compiling C compiler, a tiny self-executing RISC-V emulator, and a tiny self-hosting RISC-V hypervisor. http://selfie.cs.uni-salzburg.at
LLVM is a huge system, libFirm is a much smaller, simpler system that includes a c front end. From their site
> libFirm is a C library that provides a graph-based intermediate representation, optimizations, and assembly code generation suitable for use in compilers.
There's plenty of interesting work to do in the frontend too but a lot of it is past the parsing stage.
Language architecture and writing a compiler are very difficult tasks if the intent is to come up with a production ready compiler.
I suppose that if you want to do a toy compiler for a very small and limited language, there are simpler ways than getting into lexers and parsers. You can str replace with assembly language or C instructions and function and compile to C or assembler.
It's an extremely complex problem in itself, because most frontends strive for supporting autocompletion of broken/half-finished code with good performance and type hints.
How many are really happy with their IDE's? I remember Borland Pascal IDE and Visual Studio 6 being shining examples and after that... I don't really know what happened. Tried a Scala project with VSCode, takes around a minute for it to pick up my variable changes and 100% CPU, even worse with SwiftUI and XCode. And the computers back then had what? 600mhz?
I'd say, move some c++ graybeards to the frontend team to fix this mess.
You can see the source code for a small Lisp interpreter in 81 different languages.
It’s a very clear introduction to creating a language and building parser, interpreter, compiler, and VM for it. The book uses Java and C, but you can use pretty much any language you want (ex: I used Swift).
Peter Norvig's lisp interpreter series (Python):
https://norvig.com/lispy2.html
Thorsten Ball's interpreter and compiler book (Go):
For whatever it's worth, I have a PhD in compilers, and I work as a compiler developer. I have done nothing listed in the article except learn about regular expressions.
Also, I disagree with your disagreement with the last part of the article.
What resources have you found helpful for your Ph.D.?
In your opinion which journals/articles are classic/must be read in this field?
Is there any topics (or books covering these topics) that are a "must know"?
If there is one "cross-cutting concern" in compilers, it's the importance of program representations. The correct representation will allow you to do things that you wouldn't be able to do otherwise, or only at much higher cost. So some more concrete things to look into are SSA form and the Sea of Nodes representation (for the latter, Click: "Global Code Motion/Global Value Numbering", https://courses.cs.washington.edu/courses/cse501/04wi/papers...). Some general graph algorithm stuff (depth-first search, cycle detection, dominance) is useful.
One surprisingly commonplace, simple, and very useful thing to know is the Union-Find data structure (https://en.wikipedia.org/wiki/Disjoint-set_data_structure). I've used it in various settings in compilation. Once I was doing something in GCC and needed a union-find implementation; poking around, I found three or four of them. None were reusable, so I added one more at the place where I needed it :-(
As for journals/articles, much of the seminal compiler work appeared in the proceedings of PLDI, but that doesn't mean that it makes sense to methodically go through 30-odd years of historical papers. ACM Computing Surveys (https://dl.acm.org/journal/csur) can be quite good, if they are not too old. If you are looking at a specific area and see a reference to a survey paper on that area, definitely follow it. But all this doesn't mean that you should only focus on certain conferences or journals. If you want to dig PhD-level deep, it's very much about following references in general.
Good luck! Let me know if you have further questions.
The majority of people those references will learn toy-compilers, that are surely important but a completely different league than production-grade compilers, e.g: LLVM.
Talking specifically about LLVM, does someone have their go-to references to start and have a sense of the infrastructure, or even some specific reading about a part of the (huge) infrastructure?
https://llvm.org/docs/tutorial/MyFirstLanguageFrontend/index...
Not by any means a tutorial but Rust has a guide for understanding their LLVM compiler frontend. It has some useful insights into what actually makes up a real "production grade" compiler outside the stuff in the LLVM tutorial.
Awesome jobs by Rust to keep something like that (hopefully updated too), since the lack of updated information usually is the bigger barrier of entry for contributing/working on stuff like that. I struggle to find something similar and in-depth for clang, but I guess the bigger complexity makes it more difficult.
This makes the learning process self-serve.
Ref: https://github.com/tanin47/lilit/blob/master/playground/READ...
On a side note, I pretty annoyed by the prevelance of 'So you want to be a <insert something here>' titled articles. It's so common around now and just doesn't seem to express good intent about what is actually going to be in an article at this point.
"Write snprintf in C. For those who haven’t used C before, snprintf is a function that produces formatted output based on an input string and a variable number of arguments. Doing it in C forces you to deal with constraints you may not have had to deal with in a higher-level language. Don’t skip out on writing unit tests! (And don’t bother with floating-point numbers; just handle %d and %s.)
const size_t bufferLength = 128;
char buffer[bufferLength];
snprintf(buffer, bufferLength, "%s %d %s", "first", 2, "last");
assert(0 == strcmp(buffer, "first 2 last"));
Write snprintf in assembly…for the exact same reason. Pretty much no one programs in assembly any more, and that’s generally a good thing, but this will (a) force you to learn a new and very suboptimal language, (b) get you to learn a little about your CPU2, and (c) help you later on if you ever need to debug a compiled program without debug info. Bonus points if you can get your assembly version to work correctly with C."
It starts from the "build a calculator" angle which I think is a very good way to get into compilers, since it emphasises the recursive nature of things, and extending the calculator to a full programming language becomes easier that way.