CS 6120: Advanced Compilers: The Self-Guided Online Course
cs.cornell.edu
cs.cornell.edu
It's "par excellence", and it feels weird in that sentence structure anyways. A better way to put it would've been "my university has the low level computing systems curriculum par excellence".
Unfortunately, in my experience, computer architecture and even systems programming are domains that schools/universities appear to systematically increasingly de-prioritize, presumably because it is seen as too technical.
That knowledge however is instrumental in landing some of the best jobs in the industry.
As an aside, I also tend not to get along with the “systems programmer” types because they tend to make knowing these kinds of extremely specific factoids their entire professional personality. You end up with people that can write assembly but have no idea what a functor is.
But I agree it isn't really necessary to write a compiler. Compilers tend to be pretty much the worst case from a microarchitectural point of view anyway - full of trees of small objects etc.
ISAs are still very similar, and so is the relative performance of the various operations.
Registers, caches, pipelines, superscalar execution... it's all still mostly the same, and still drives what the codegen should look like.
And yet you have so many people that don't know the considerations of what makes a simple loop fast or not.
Granted, the reality is that most people are doing the equivalent of digital plumbing, not necessarily solving computational problems.
Writing a compiler with shallow understanding of the computer you're targeting is definitely possible, but ultimately wasteful.
I think we need software engineers coming from electrical engineering, mathematics, humanities, and all walks of life. If you’re not interested in compilers, maybe the comments for a link about compilers isn’t your place to be.
Like most ideas in programming based on theory, it's ultimately a very trivial thing, whose theoretical foundation is quite unimportant outside of helping design a minimal set of operations for your programming language.
It's not particularly important in the grand scheme of things. People even build and use monads with no understanding of the background behind it.
And then they miss the common pattern between them, so they miss the opportunity for abstraction and a common interface over the pattern. That’s the whole point of category theory: recognizing large classes (in the mathematical sense) of objects by their universal properties. It’s useful in programming language design because it gives you the ability to build useful, generic, water-tight abstractions.
A lot of abstractions built without any regard to the mathematics behind them wind up being leaky and difficult to apply appropriately because they rely on vague intuitions rather than simple mathematical properties.
That case is due to history and path-dependence. The theory and abstraction of the lens library was developed long after Haskell the language was designed. If Haskell were rebuilt today from the ground up with lens in mind you wouldn’t have that mess. Unfortunately, fixing it now would be too much of a breaking change.
Agreed. Likewise, you end up with different names for the same concepts across different facets of the industry which actually makes the profession as a whole harder learn and makes communication across different communities harder.
There are enough complexities in the problem domain and the system itself, so KISS is king.
Good software is usually built by focusing on the actual problem being solved, and only generalizing solutions once sufficient amount of specific ones have been built and their commonalities identified to be lifted in a generic implementation.
The most impact language purists tend to have is when some of features end up adopted and adapted by practical languages (C++ or even C#).
Concepts without names are difficult to reason about.
I think everyone who knows what they are would agree that the definition of "functor" can be learned in ten minutes. Recognizing the same concept being applied in different situations is the value. (As my sister comment says.)
What kind of insights this means for programming is quite up to interpretation and how you choose to formally describe its semantics (and most popular programming languages don't have formal semantics).
Telling people about the power of hydraulic presses when they just want to make omelets is pretty unhelpful.
The idea is that the type of elements is a parameter, called A, say. Then you use functorialty to change all the elements to something of a different type (or a different element of the same type).
For instance List(A) is the type of lists of elements of type A. If you have a function f taking input A and giving output of type B you can “apply the functorial action” of List to transform a List(A) to List(B). For lists “the functorial action is simply mapping the function on each element. But being able to abstract over all functors can give very general implementations of interesting algorithms and patterns which then applies to many situations.
Your attack on systems programmers holding specific knowledge as holy while pointing to their ignorance about another bit of knowledge you consider more relevant seems kind of ironic to me.
New challenge to self: write a functor in pure assembly.
1) I have written many functors in my life without describing them as such
2) "Pure assembly" functor doesn't make much sense. Functors, both in the CS and mathematics sense, require the concept of type. Which is absent once you are dealing with stacks and registers. My best idea is something when you have mixed-width registers - eg, say you've got some 128 bit and 256 bit registers. Your functor could apply a mapping from one to the other (and would probably be nothing more than a move instruction with the appropriate sign extension).
The kind of ppl who always when arguing about programming langs or compilers starts referring to C or CPP standard or its ecosystem as a holy bible
Thank gods that's good thing.
In my opinion one of the biggest industry advancements we have had is that compilers are now available without such level of low-level detail. There is still so much work to be done on a compiler level that really shouldn't concern themselves with the (micro)-architectural level of computers.
LLVM allows hardware manufacturers to more easily provide mainstream language support for their platform than before, but the problem here was mostly that GCC was hostile to modular design, not really theoretical advances.
In terms of frontends, I guess we're seeing more languages reach C-level performance thanks to LLVM again.
But in terms of optimizations driven by theory? There were some significant advancements in generic auto-parallelization for imperative languages, about a decade ago I think. And it it doesn't magically solve the codegen problem, and remains hampered by language semantics that are not always parallelization-friendly.
There were a bunch of improvements which were driven by making languages more hardware-aware, e.g. the concurrent C++ model in 2011 which was widely copied to other low-level programming languages.
We're also seeing more and more libraries that are specifically designed to target hardware features better.
So ultimately it looks like most of the advances are driven by better integration of how the hardware works throughout the compiler, language and community.
If they made JavaScript as fast as C, the software industry will become a cheap perversion of what it once was, and I'm picking up my toys and going home.
Could you elaborate what you think are the best jobs in the industry and why?
Compiler optimisations give you faster/cheaper execution of every program.
Type systems and linters detect errors without writing or running tests.
Programming languages are tools for thought. Matching one to the domain makes the domain easier to reason about, lets people solve bigger problems.
Whether that correlates with "best jobs" is subjective. The value proposition of making other programmers more productive is really good.
If you think that's an exaggeration, consider writing a web browser in machine code.
The surprising thing about this, I think, is that hardware is generally expected to be quite underpaid around here, I think, compared to programming.
But to get "the best jobs in the industry" you also need a demand/supply imbalance and I wonder if that exists and where, or whether something different was meant by OP (i.e. not just plain $$)
The statement is correct: universities are de-prioritizing systems programming, but not for the reason you state.
Since I've been working with Universities, I've come to appreciate their fundamental problem. There's a lot of potential material to cover in a finite number of hours, which is horrifying short if you're the one to allocate them.
The amount of information one could know in our field grows exponentially with time, and we're well past the point of overflow. So yes, systems programming is getting less class time in the general track, but that's only because it's less relevant to more and more students every year, so it makes sense.
Things seem to have changed recently.
I mean, Haskell is also over 30 years old at this point.
Now functional logic languages like Verse, that is definitely new material.
[1] https://github.com/rohanpadhye/jqf/wiki/Fuzzing-a-Compiler
Just going by the new stuff that you could have taught back in 2004-2006 that you likely didn't, there's SSA construction, SLP vectorization, automatic peephole superoptimization, and that's just things I could name off the top of my head (and were papers for my qualifying exam :-P).
What hasn't changed is the compiler textbooks, which tend to have way too heavy a focus on how to build a parser generator and almost nothing on how you actually architect a compiler, let alone designing modern computer architecture. But this is a gripe which has been around for decades.
That makes compiler teams especially keen to hire people who have already spent ages building compilers. However that obviously has a bootstrapping problem so the larger teams also hire graduates who look like they might make it over that curve. That's roughly how I got into it.
If you're experienced in general but not with compilers, the obvious play is to join a company doing whatever you're used to which happens to also have a compiler team, which roughly maps onto "largish software company", and aim to move laterally.
So, is it very advanced then? I don't think I'm at PhD level when it comes to CS.
Just try it, if you can do it, you are ready, and if you can't, you aren’t.
It was very engaging to put up PRs for small issues in the Bril IR, and work with the professor to fix them.
As referenced in another comment, Simon Peyton Jones has a 1987 book on compiling functional languages, The Implementation of Functional Programming Languages. Follow that up with some of the referenced papers contained therein and you will have a base level to start at. Following the FP academic boom of the 80’s I would look to the following (in no particular order): papers of Stephanie Weirich, Simon Peyton Jones, Simon Marlow, Odersky, Philip Wadler and other authors you would find being referenced in their papers; the webpages for the Koka language, Idris, the Granule project, Haskell, and Ocaml all have references/publication sections that will contain a wealth of material.
It is kind of a shame there is not a more ‘one-stop-shop’ for functional language compiling, but the research is so broad it would be hard to condense. You aren’t going to go wrong starting from SPJ’s book as the basis. Jones has a talk on YouTube about ‘Fitting Haskell into Nine Terms’ which is based on a paper that discusses the compilation strategy for GHC regarding the push to keep the internal System Fc language small and clean that is a good watch, along with the Jones talk on Compiling without Continuations also on YouTube and gives more internal views of GHC.
Sorry the list is a bit scattershot, if you need more specifics I will certainly try to find a more narrow selection.
The really gnarly part of compiling languages is dealing with load/store. If you translate a functional language to SSA form, what you get is an IR which doesn't have load/store in it. I.e. that's easy mode.
I'd suggest compile to SSA with implicit memory semantics, then somewhat later splice in the part of the language runtime that deals with memory management and now you have an imperative SSA form, continue as usual.
That is, they're the same problem really.
Current compilers do not guarantee that re-running the optimization passes a second time is idempotent. Every pass just does some "useful" transformations, it's best-effort.
Drop in a modern AI as the optimizer and every single line of C++ code can be considered undefined behaviour.
It's a bit difficult to use because it makes extensive use of templates.
Most compilers and operating systems are built in C++ nowadays, it's the industry standard for any systems-level programming.
C++ is the industry standard for existing software but it would be dubious at best to start a new project in C++. Especially something that has essentially no dependencies like a compiler. GUIs and games, sure.
Believing that all new software should be written in Rust rather than C++, and that this situation is inevitable anyway, is one of the core tenets of this church.
I bet you were one of those people that dismissed anyone that liked the iPhone as being an Apple fanboy.
You know people really like Rust because it's good right? I'm curious what other reason you think people have for promoting it?
Even the White House says you should use Rust instead of C++. Are they Rust zealots?
Anyway, no point in cargo culting the use any further.
Very easy to use. Build a parse tree from it, deal with the ambiguous nonsense of your language in a translation to AST.