Comment by ubernostrum | Hacker News Reader

Comment by ubernostrum | Hacker News Reader

Parent Full thread

Both Herb and Miguel seem to be writing on a flawed assumption, which is that JIT will only ever apply optimizations that can be proven safe. As I understand it, this is true for CLR implementations. And if that were the case generally, there would be no significant-enough difference between the set of optimizations available with AOT and the set of optimizations available with JIT, and JIT would never be "as fast as C".

But this disregards one of the major selling points of profiling + optimizing JIT: you can apply optimizations that cannot be proven safe and you can get away with it. This means you can not only be "as fast as C", you can implement a JIT that will mop the floor with C.

Good JITs are already taking advantage of this, and the very first comment on Miguel's article points this out. This (four-year-old!) article by Charles Nutter explains it a bit more in the context of the JVM:

http://headius.blogspot.com/2008/05/power-of-jvm.html

I keep on hearing that it's possible to write a JIT-based vm that's "faster than C". I have yet to actually see one where that is true for anything other than one or two very specific micro-benchmarks.

Edit: the old C2 wiki actually has something useful to say about this.

http://c2.com/cgi/wiki?AsFastAsCee

http://c2.com/cgi/wiki?SufficientlySmartVirtualMachine

It's actually not that difficult to find programs where a JIT is better. Things that benefit significantly from profile directed feedback often run faster in JIT than static compilers, because JIT's tend to do this dynamically, whereas static compilers require training runs.

For example performance critical code that has a lot of conditional branches, but only a few are taken at runtime, will often work faster in a JIT.

(Of course, the typical solution is not to write performance critical code with a large number of branches, i'm just giving you examples)

Sure, but even in those cases the saving has to win back the overall cost of the runtime profiling and dynamic compilation. The crossover point seems to be high enough in the graph that it's not reached very often in practice - especially since, as you say, programmers tend to learn early on that hoisting conditions out of inner loops Makes Code Faster.

The crossover point seems to be high enough in the graph that it's not reached very often in practice

I remember a study posted here a few months back that supported the notion that most web apps have a certain "steady state" that they reach after they start up, where almost all the types of variables are known, and most of the funkier "dynamic" changes don't happen. If one can get a JIT to remember the tracing information from session to session, it should be able to run as fast as or faster than C.

But if they reach a steady state, then profile guided optimization can be used to make an AOT C compiler generate faster code too.

True, but let's say the implementation Lang in question has optional types. Consider the unfilled type information to be technical debt. If the tracing information can be exploited to automatically fill in the type info, then this is technical debt that magically repays itself. That's something startups should be interested in.

"Winning back" the overhead of the profiler + JIT is not terribly hard right now for long-lived processes. And unlike the assertions in Herb's original article this is not something that requires a mythical "sufficiently smart compiler" to make even better -- it just requires people to implement and fine-tune things we already know.

Yes, you can perform unsafe optimizations, but optimizations take time, so if you are wrong enough, you lose.

Anyway, as I said below, there is nothing that stops you from implementing a JIT for C, it's just not cost effective from a "where do i put money to get better performance" perspective. If you did this, these JIT's would do just as well as the ones you think will "mop the floor" with C.

Something that seems to be missed here is that the whole point of compilers (and JITs) having intermediate IR is to abstract away the language to a point that you can optimize and codegen stuff.

So talking about a "statically compiled" or "dynamically compiled" language is well, silly. Most popular languages could be compiled either way[1], it just happens that we don't because it's not "better enough' in the real world.

If you want a real world example, LLVM will happily JIT C/C++ code for you, for example. It doesn't care that it's C/C++. It will even run it interpreted if you like.

[1] Some languages would still require a support library with an interpreter/jit/compiler if you statically compiled them, in order to handle dynamic class loading.

Polymorphic inline caching can be proven safe; it's just proven safe through dynamic checks of the cache, rather than statically. Miguel is right here, he's not working with a flawed assumption.

Reply on news.ycombinator.com