Really cool article. Posts like this always make me wonder what the state of the programming would be if browsers hadn't sucked up almost all of the world's compiler optimizers.
But considering that optimized scalar code performance has moved, what, maybe 40% over the last two decades, I'm going to say "not much". Compilers are sexy, but they're very much a solved problem. If we were all forced to get by with the optimized performance we saw from GCC 2.7.2, I think we'd all survive. Most of us wouldn't even notice the change.
But in the context of JIT compilers for dynamically typed languages, in particular the space involving runtime inferred hidden type models, there is a TON of work left on the table.
It hasn't been paid much attention to in academia, IMHO largley because of a historical perspective on optimizing dynamic languages as "not classy" work among language theorists. I hope that perspective changes over time.
Not for all of the other widely-used languages that still have incredibly simple interpreters. Think how much energy could have been saved if Ruby, Python, and PHP were all as fast as your average JS engine.
I would think we'd see even better improvements if developers would move towards statically typed languages as well.
Here's an old slidedeck that talks about it: http://www.oracle.com/technetwork/java/jvmls2013wimmer-20140...
My god can't believe it was 3 years ago i read about it on HN.
That's the currency I deal in.
In fact, looking at the market as it stands right now I'd say that performance concerns are almost entirely uncorrelated with programming environment adoption.
I actually think that the reason why JS is so odd is that it is so expressive. That tends to happen with kitchen sink languages like C++.
Relevant: "There are only two kinds of programming languages: those people always bitch about and those nobody uses."
And you're gonna apply that to the darkest corners of ES6? Whee, language theory nerd paradise!
I can't wait for the compilers targeting that using all sorts of really odd idioms that in effect work like fuzz testing.
https://www.destroyallsoftware.com/talks/the-birth-and-death...
Javascript is low level, primitive and clumsy. There is absolutely no excuse for it being so crappy.
If you still think it is "expressive", you never seen an expressive language.
Are you aware who Filip Pizlo is?
You seem to have an intense dislike of JavaScript. I'll the first one to admit that JavaScript has a lot of problems. But not being expressive enough is not one of them. JS is, if anything, too dynamic and expressive.
And, no, JS is not expressive. It got strict limits on an available level of abstraction (unless you go into eval, of course).
JS perf is important to users though. Faster execution means fewer watts spent rendering and interacting with your favorite web page.
(Fun fact: B3's backend contains a machine description language that gets compiled to C++ code by a ruby script, opcode_generator.rb. We use Ruby a lot.)
So, naturally, we want to avoid building JSC twice. ;-)
EDIT: I wonder why the positive effect to re-discovering that not all compilers need to be like C and C++ compile speeds and that it was once upon a time mainstream, is worthy of downvotes.
Unfortunately, they didn't learn about the stuff between Oberon and 2007 that would've been nice to have in a modern, app language. ;)
The remark is tailored to those that think compiled languages can only be slow as C and C++ compilers, since they never used anything else, and then jump of joy when they use Go.
Yet if it wasn't for the VM detour of the last 20 years, that experience would probably be a current one, instead of being re-discovered.
Ahh. Ok.
"Yet if it wasn't for the VM detour of the last 20 years"
You mean the C++ and VM detours? ;)
No. I'm claiming it doesn't perform anywhere near the level of optimization of GCC and LLVM.
> I'd think Java can certainly be considered having highly optimized compiler / runtime. But it has almost same performance as Go and much higher memory usage compared to Go.
I disagree with what seems to be your implication that compiler optimizations don't matter (and almost everyone else who works on compilers would also disagree), but I don't really want to turn this thread into a critique of the benchmarks game, so let's just leave it at that.
Please don't jump to the conclusion that huge differences in the default memory allocation of tiny 100 line programs means there will be similar huge differences between ordinary large programs.
Notice that even for those tiny 100 line programs, the difference can be more like 2x when memory actually needs to allocated for the task.
† Which can be seen as a runtime cost for interpreter+JIT languages, but that's a different issue. We're talking time-to-steady-state when the interpreter is fed a file (which is something Javascript engine devs worry about a lot), not benchmark-speed-at-steady-state.
Compilers are sexy, but they're very much a solved problem.
For sequential languages the problem has shifted: how can I get a performant compiler easily. The most interesting answer to this question is PyPy's meta-tracing, and that's work is from 2009, and far from played out.
A 40% decrease in optimization is enough to drop framerates from 60fps to 30fps easily, so I'm pretty sure we would notice it.
I'm not convinced. Raw single-thread number crunching performance is somewhere around _two to three fold_, clock-for-clock, on Intel x86, over that of 10-15 years ago. What methodology do you use to attribute only a fraction of those gains to language optimizers? And even if you are correct, why is it meaningful? Who is going to have invested energy in optimising the shit out of mundane codegen when hardware performance will have just come and stolen your thunder a few months later?
The problem we have now is that CPUs are gaining ever more complex behaviour, peculiarities, and sensitivities. I'd say compiler engineering is far from a "solved problem", even for statically-typed languages.
With mainstream CPUs, exactly the opposite is happening. CPUs are getting more complex under the hood, but less sensitive to code quality. For example, a lot of the scheduling hazards in the P6 microarchitecture have been eliminated in subsequent iterations. Branch delay slots are a thing of the distant past, so are pipeline bubbles for taken branches, indirect branch prediction is extremely capable, even the penalty on unaligned accesses is minimal.
No they're not, and won't be for long (ever?). However it does not matter because they are "good enough".
Compilers are driven by heuristics which provide "reasonable" results in most cases for common architectures. But they still leave a lot on the table. Compiler writers have to trade compile-time with execution-time. Now we're not talking about an order of magnitude, but rather ~20%-30% in some workloads. When it matters (I guess for people like Google/Facebook/Amazon/... it translates in electricity bill and a number of racks to add to the datacenter) people may have to get down to the assembly level for a very small (and hot) part of the program.
Python, Ruby, PHP, and Perl aren't "the rest of the world"? As far as compilers are concerned, all of those languages have more troublesome semantics than JavaScript does.
> And only the web world is so obssessed with smart compilers compensating (impressively, but still far from being sufficient) for multiple deficiencies in the language design.
You have no idea how much compilers have to compensate for the deficiencies in C and C++'s design.
And no, thank you kind sir, but I've got a very good idea of what compilers are doing wrt. C deficiencies, I was writing OpenCL compilers for 6 years at least. Besides aliasing stupidity and byte-addressing there is nothing really bad to compensate for.
asm.js and Web Assembly.
> Besides aliasing stupidity and byte-addressing there is nothing really bad to compensate for.
Aliasing issues, C++ heavy reliance on virtual methods, too many levels of indirection in the STL, overuse of signed integers due to "int" being easier to type interfering with loop analysis, const being useless for optimization, slow parsing of header files...
For example in JS, once you prove that "o.f = v" is not going to hit a setter (or you put a speculative check to that effect), then you know that this effect does not interfere with "v = o.g" (provided you check that it's not a getter). JS VMs are really good at speculating about getters and setters. The result is that alias analysis, and its clients like common subexpression elimination, are super effective. It's only a matter of time before we're creating function summaries that list all of the abstract heaps that a procedure can clobber so even a function call has bounded interference.
This is totally different from C. There, making sure that accesses to o->f and o->g don't interfere is like pulling teeth. And the user can force you to assume that they always interfere by using -fno-strict-aliasing, which is hugely popular.
The fallback-to-C paths on the web aren't that great, though. At least not yet. Once you fall back to C, you're sandboxed into a linear heap with limited access to the DOM and JS heap. But we'll fix that eventually. :-)
As for virtual methods, it is a problem of a bad C++, devirtualisation may help, but nobody cares in general. We have a cool curiously recurring template pattern instead.
But for the integers you're right. And signedness is still the most annoying source of bugs in the infamous LLVM instcombine.
Headers are a frontend issue, nothing to do with the optimisations.
That's because (a) page performance is frequently gated on things other than JavaScript, so people don't go through a lot of trouble to write C++; (b) many modules that would be written in C in Python are provided by the browser itself; (c) JS itself is usually fast enough, since the gap between JS and C++ is much less than the gap between Python and C++.
> As for virtual methods, it is a problem of a bad C++, devirtualisation may help, but nobody cares in general.
Huh? Tons of people care about devirtualization! Much of the reason Firefox builds go to the trouble of PGO (and it is a huge amount of trouble) is to get devirtualization.
> As for virtual methods, it is a problem of a bad C++
So I could say the same thing about JavaScript: if it's slow, you're writing "bad JS". But you would rightly reject that as invalid: if the code people write in practice is slow, then the problem is with the language encouraging people to write slow code. The point is that the same thing applies to C++.
As for C++ - there is a choice. With JS the choice is much more limited. You either rely on non-standard asm.js behaviour, lose DOM access with web assembly, or tolerate the stupidity and limitations of the language that far overgrown its tiny niche.
Btw., I am yet to hear how a language with a fixed syntax and no macros can be perceived as "highly expressive".
Also, what do you mean by level of indirection in the STL? Pointer chasing or, I suspect, layers of layers of tiny functions that need to be inlined for reasonable performance?
Which ones are these? All of the statically-typed and well-designed languages I can think of are, at the least, hard to compile well, if not hard to compile in the first place. (Haskell and Rust both come to mind; there are few Haskell compilers other than GHC, and no Rust compilers other than rustc.) The languages that are easier to compile are either not statically-typed in a useful way, or not well-designed, or both.
Edit: I was really referring to language families, so including things like OCaml, Modula, etc.
The frontend is usually the hard part IMO. In WebKit, we have 100,000 lines of code for our "frontend" (i.e. the DFG compiler) and 50,000 lines of code for our "backend" (i.e. B3). That split is really interesting.
Far as Ada, I figured it would be difficult due to the large number of language features and complexities of analyzing a program for safety w/ all its rules. And any interactions that might emerge in that between features and rules. Could be easier than I think but my default belief is that it's not easy.
Frontend was awful and I never finished it, but the LLVM-style backend wad very easy to build.
Is that really the case? It seems to me that a Prolog-like resolution system, coupled with a constraint solver, would get most of the job done with little effort.
There are certainly many rules to keep track of, but in some cases the newer rules are strict generalisations of the older rules. For example, Haskell's original type classes are just a special-case of modern multi-parameter, flexible instances/contexts, etc. type classes. Likewise, we can implement constraint kinds, type class instance resolution and type equalities using one constraint solver.
My usual approach to implementing a type system is to derive a flat list of Prolog equations out of the AST and leave it to Prolog for solving. If you ask what to do with error messages, I've got a comprehensive answer, but it is not for a mobile phone typing.
My approach to the typing error reporting is quite compatible with the Prolog codegen. What I do: for each AST node which produce one or more type equations I record the location data (cache it and give it an index), and then issue a Prolog term like this: with_location(Id, Term), for each of the terms generated by this location.
with_location semantics is simple, but may depend on the presence of a backtracking. If it's a plain Hindley-Milner, there is no backtracking, and it simply do call/1(Term), continue if true, store Id and the failed term with all the current bindings and fail/0() if false.
If there is a backtracking, I store a stack of location ids and failure reasons instead, but this required patching the Prolog execution engine a little bit (luckily, I'm using my own Prolog for this).
Now, the next step is to decipher the failed term. Most of the equations are in a trivial form: equals(A, B), so I simply report that prolog_to_type(A) failed to unify with prolog_to_type(B). For the other (rare) kinds of equations I define custom pretty-printing rules.
To me this is the difference between knowing that your program will probably work after it compiles, vs not knowing until after its deployed.
What I think people really enjoy in dynamic languages like JavaScript is the instant feedback, just reloading the browser. This can be accomplished with decent IDEs for most statically typed languages.
Also, many of the improvements done for JS will certainly trickle down to Python, Ruby, PHP eventually.