Jitterdämmerung
blog.metaobject.com
blog.metaobject.com
http://insightfullogic.com/2014/May/12/fast-and-megamorphic-...
Worth noting also that Azul's Zing ReadyNow technology for AOT (and indeed their no-pause C4 collector) address some of the specific gripes raised here around the JVM...
The driving force behind ReadyNow (as I understand it) was that many performance sensitive systems needed to be fast out of the gates on start up. This means that an interpreted->compiled->optimised-compiled transition was not acceptable.
Developers would try to solve this by running dummy data through the system before it was open to real traffic. But this had the unfortunate consequence that the JIT would optimise for dummy data only, including clever inlining at, apparently, monomorphic call sites etc. When real traffic flows into the system the JVM sees that its optimisations are no longer effective/valid. This would trigger a de-compilation/re-compilation of many code sites causing a very noticeable stutter.
Now we have ReadyNow. If you are really committed to Java or the JVM and you don't like these JIT stutters this is your solution. But this is an extra layer of complexity and another thing to be managed and another thing to fail. This is on top of your JVM, jar file soup you may already be struggling with.
I would prefer a good AOT compiler to remove this concern and give me quite good predictable performance. YMMV of course.
Sources
http://www.azul.com/products/zing/readynow-technology-for-zi...
https://groups.google.com/forum/#!searchin/mechanical-sympat...
PyPy's meta-tracing JIT-compiler framework [1] allows you to generate reasonably fast tracing JIT compilers for languages essentially by writing an interpreter. Laurie Tratt has written a nice description of this, and the advantages this delivers [2]. This make the development of new programming languages much simpler, because you don't have to invest a lot of time and effort into producing a reasonably fast compiler (whether JIT or AOT) at the beginning of language development. Yes, in many cases you can build AOT compilers that beat JITs produced by meta-tracing an interpreter, but not easily so.
Tratt's team has used this approach for building impressive multi-language tools [3] that allow you to write programs in heterogeneous languages like Python, Prolog and PHP at the same time.
[1] http://stups.hhu.de/mediawiki/images/f/f5/Tracing_JITs11_tra...
[2] http://tratt.net/laurie/blog/entries/fast_enough_vms_in_fast...
If not, why not?
You can very easily create a dumb one - you just copy-paste interpreter loop essentially (which is what e.g. cython does if not presented with annotations), however the results just aren't very good
[1] http://stefan-marr.de/downloads/oopsla15-marr-ducasse-meta-t...
This can apply to x86, too. Not sure if all your users have TSX-capable CPUs? You're now shipping two binaries. Not sure if your users have AVX? Now you have at least three binaries.
This is the biggest advantage of intermediate languages - your code can at the very least execute everywhere with a single image, and in some cases automatically take advantage of platform features such as AVX.
I think ART takes the correct approach: ship IL and AOT it on the device. Hopefully some day we can get the same type of system for LLVM IR.
The Intel C/C++ compiler creates different code paths depending on the CPU's capabilities. So at least there's no duplication of non-cpu-specific code.
It also does or at least used to check for the CPU's capabilities by using the CPU's vendor ID string, resulting in AMD CPUs executing the slowest code path: http://agner.org/optimize/blog/read.php?i=49#49 (Intel got sued for this by AMD: https://en.wikipedia.org/wiki/Intel_C%2B%2B_Compiler#Critici...)
*Edit:
>Hopefully some day we can get the same type of system for LLVM IR.
Right now the LLVM IR isn't platform-agnostic, e.g. when it comes to calling conventions.
Also when Apple transitioned from 68000 to PPC in the 90ies, then from PPC to x86 and recently from 32 bit to 64 bit, they used "fat binaries" where a single executable file contained code for several platforms. That way one and the same application could run on old Macs with PPC, newer ones with 32 bit x86 CPUs and really new ones with 64 bit capable x86 CPUs.
Wow. That's a really slick way to solve the problem.
Edit:
> Right now the LLVM IR isn't platform-agnostic, e.g. when it comes to calling conventions.
That's unfortunate. Either way I'd love to see some form of native IL that can be shipped instead of binaries.
WebAssembly could fit the bill, although it's still in early development.
But it does give you the same benefits in terms of CPU capabilities. You can't ship intermediate code that's agnostic about e.g. x86 vs x64 (C and C++ just aren't amenable to that, no matter what your intermediate representation looks like), but you can ship intermediate code that expects some form of x64 but postpones instruction selection until the program is installed, so you can take advantage of new instructions just when they are available. As I understand it, Apple is going to use it for that purpose.
OS/400, now IBM i, is probably the last living mainframe doing it.
Using USA CS speak, I guess I majored in systems programming with a minor in computer graphics, and my university had a very nice library.
So I got to do lots of programming language and OS research at that library. When I got into the university, we still had Gopher. :)
NGEN is there since day one, but they always kept it quite simple in optimizations and only allowed for dynamic linking (no static binaries). Also it required signed binaries, which many people did not bothered with.
Windows Phone 8 adopted the Singularity toolchain with AOT compilation in the store, with a dynamic linker on the phone. .NET Native just moves the WP8 linker out of the device and produces static binaries instead.
You configure a system-wide flags specifying your cpu, optimization level, the features you want to enable and disable (many libraries and apps have flags to compile support for some features or not), etc. There were also app-specific flags IIRC.
When you install a package it downloads sources and the meta-package from portage, then it is compiled on your machine with your flags added for your specific processor with only the features you enabled. It was supposed to be much faster than "bloated ubuntu", but I suspect now it was mostly placebo :)
On the other hand installing Open Office on my Celeron 2000 laptop with 256 MB RAM took 10 hours, and if I wanted to enable some feature that turned out to be useful after all - I would need to wait another 10 hours.
I used it for 2 years when I was on university but then it was too much to wait for hours when I needed sth.
Gentoo predates Ubuntu by several years. I remember reading some "gentoo ricer" threads bike-shedding over ultimately inconsequential optimization flags. But don't underestimate the simple benefit of `-march` and similar. At a time when binary distributions were pushing x86 binaries which ran on i386, compiling for a newer architecture could give a substantial increase in registers available, instructions available, and instruction scheduling quality. In aggregate, this definitely can improve performance.
Since then, I believe Debian moved to an i686 base which narrows the gap.
Sure it has a clunky default interface, and IBM artificially cripple most systems with licensing restrictions (owning a multi-core Cell processor on which I can only access one core? WTF?), but the virtualization capability, backwards compatibility, stability, reliability and performance really leave standard "consumer" grade server architectures for dead.
I can even run PASE, or Linux / AIX under an LPAR if I want to. It's pretty crazy what these things can do.
I find using them to be almost like living in some alternate steam-punk universe. It's like putting punch-cards into a quantum computer!
Three code paths, not necessarily three binaries. It's easy enough to compile different variants of performance-critical routines and then select between them at run-time.
I think ART takes the correct approach: ship IL and AOT it on the device. Hopefully some day we can get the same type of system for LLVM IR.
This isn't necessarily optimal though. Having certain CPU features may make algorithmic changes useful -- e.g., switching between lookup tables and direct computation depending on whether you have the CPU features which make the direct computation fast. There's no way an AOT will be able to handle that for you.
Phoronix did a test where compiling with march=native sped up programs a substantial amount. 20% for one application. It astounds me that we throw away that sort of gains.
I'm also not sure if "scripting languages" truly don't need a JIT. The development of LuaJIT for example has been sponsored by various companies, so there seems to be a need for the fast execution of Lua code: http://luajit.org/sponsors.html
This approach is great for some things, but it does sacrifice the flexibility and expressiveness of the language. Anything not part of the core language (which is a lot compared to most other programming languages, e.g. anything related to OOP, or more advanced data types than strings, floats and tables) has to be re-invented and/or bolted-on, which IMO makes it unsuitable for most kinds of applications.
This should not be interpreted as critique about Lua the language bu the way, I'm a big fan of Lua for embedded scripting and I generally love tools with a narrow focus (as opposed to kitchen sink technology). I would not choose Lua for anything besides embedded scripting though.
Usually I end-up using Python for smaller things that aren't mission-critical or need maximum performance, C++ for almost everything else, and Objective-C for OS X/iOS stuff. Maybe Java for things where both performance, safety and ease-of-development/maintenance matter (the latter would be mostly for other people who would have to work on the same projection who are less skilled in C++, as I'm not at all a fan of the language myself, but I recognize it has some properties that make it a suitable choice for many kinds of applications ;-). I don't have enough experience with any other programming languages that can be deployed easily across Linux and OS X, so I can't comment on those. Rust seems to have some good ideas so I may want to learn more about it in the future.
If I wanted to write something for Windows platforms I'd most likely gravitate towards C#, from what I know about the language it appears to have all the good things of Java without its downsides.
Having already quite a good experience with memory safe languages when Java came into the world (mostly Wirth and ML derived languages), I never understood why Sun went with the JIT approach. Other than trying to sell Java to run in the browser.
There were already quite a few solutions using bytecodes as portable executable format that were AOT compiled at installation time, e.g. mainframes and research OSes like Native Oberon variants.
The best is to have both on the toolchain. A interpreter/JIT for development and REPL workflows. Leaving the AOT for deployment/release code.
Not if everything else includes long-running servers that may require various kinds of fiddling at runtime, like injection or turning off and on of monitoring/troubleshooting code, nor if you want to have zero-cost abstractions that are actually high-level abstractions (as opposed to the other kind of zero-cost abstractions, that are basically just weak abstractions that can be offered for free).
However bearing the scars of the pain of "kinds of fiddling at runtime, like injection or turning off and on of monitoring/troubleshooting code" actually means in practice, changed my mind.
It is so damn hard to tune them, that there are consulting companies that specialize in selling JIT tunning services.
And in the end they offer little performance advantage over using PGO or writing code that makes better use of value types and cache friendlier algorithms.
This was one reason why Microsoft introduced PGO for .NET in .NET 4.5.
So yes, I really mean everything else.
If you really need careful tuning of optimizations (which is never a walk in the park), it will be made much nicer with this: http://openjdk.java.net/jeps/165
[1]: Just like Erlang, only fast! ;)
One just has to look at Haskell, OCaml, Common Lisp, Ada, Eiffel, SPARK, Swift, .NET Native,....
The fact that for the last decade mainstream AOT was reduced to C and C++, kind of created the wrong picture that one cannot have performance in any other way.
I always dislike how they present some of the language features (stack allocation, structs and arrays) as if there wasn't any other language with them.
For monitoring, although one doesn't get something as nice as VisualVM or JITWatch, it is possible to bundle a mini-monitoring library if really needed.
Something akin to performance counters in Windows.
I use both Java and .NET since they got out, there is the occasional C++ library, but they are the core of my work. So I know quite well their eco-systems.
Some of the mentioned scars were related with replacing "legacy" C++ systems, while keeping a comparable performance.
[Citation needed]
final boolean runF = ...; //In *this* program run, it works out to be false.
...
if (runF) {
f(x);
}
if (cond) {
myVar = f(x); //f(x) has provably no side effects
}
It surprised me when I saw all this happen, but the JVM's JIT is actually pretty smart and works very well for long running server processes.
In the second, what do you mean by "run f and in parallel check cond"? How that would translate to the machine code? You want to run and synchronize cond and f(x) between two separate CPU cores or what?
I don't know why you believe such things are trivial. On one occasion I altered some code which broke this, and it added about 10-20 seconds to a 15 minute HFT simulation. If that code made it to prod, those 10-20 seconds would be distributed around the hot parts of an HFT system. That's not remotely trivial, at least for performance critical applications (HFT, RTB, etc).
In the second example, basically what happens (if you get lucky - I haven't found the JVM JIT to be super consistent) is that computation of cond and f(x) will be put into the pipeline together, and only the assignment to myVar will be conditional. This is strictly NOT something the CPU can do by itself, since there are side effects at the machine level (allocating objects, and then GCing them).
The benefit of this optimisation isn't correctly predicting the branch, which is the only thing the CPU branch predictor can do, it's removing the untaken branch from the optimisation pipeline.
if (x) {
y = rand();
} else {
y = 14;
}
y * 2;
y * 2
return y * 2
This in theory allows faster code to be generated. Than ahead of time compilers can. In practice this is destroyed by the pointer chasing default datastructure in java.
If you are executing "C" using JVM techniques then a research JVM JITing C code gets within 15% of GCC on a reasonable set of benchmarks. (http://chrisseaton.com/plas15/safec.pdf).
These benchmarks are favorable for the AOT GCC as they have few function pointers, startup configuration switches and little dataflow variety.
I suspect that specific compiler optimizations on the C code for the dense matrix transform in GCC are bigger part of why GCC is faster than the fact that it is AOT instead of JIT.
There are also a number of AOT compilers for Java e.g. excelsior. But I have not seen superior performance with that over an equivalent hotspot release.
They'd be surprised to learn that considering GCC supports profile-guided optimisation.
But this, too, is a tradeoff. Some JITs (like JS's in the browser) require fast compilation, so some optimizations are ruled out. Very sophisticated optimizations require time, and therefore are often useful when you have tiered compilation (i.e. a fast JIT followed by an optimizing JIT), and tiered compilation is yet another complication that makes JITs hard to implement.
That is not AOT. That is compiling some code AOT and some JIT[1]. It's not a contest. Those are two very common compilation strategies, each with its own pros and cons.
> In practice it almost never makes sense to do this for a language that isn't full of gratuitous dynamism.
What you call "gratuitous dynamism" others call simple, general abstractions. BTW, even Haskell/ML-style pattern matching qualifies as "dynamism" in this case, as this is something a JIT optimizes just as it does virtual dispatch (the two are duals).
[1]: Also, a compiler doesn't generally insert huge chunks of static code into its output. That's the job of a linker. What you've described is a system comprised of an AOT compiler, a JIT compiler, and a linker that links the JIT compiler and the AOT-compiled code into the binary. Yeah, such a system is at least as powerful as just the JIT component alone, but such a system is not called an AOT compiler.
Really, in almost all cases where JITs are effective it's because of a language where you have to use a dynamic abstraction when it should have been a static one. The most common example being virtual dispatch that should have been compile time dispatch. Or even worse, string based dispatch in languages where values are semantically string dictionaries (JS, Python, Ruby). A JIT is unnecessary in languages with a proper distinction between compile time abstractions and run time constructs. Names should be a purely compile time affair. Instantiating an abstraction should happen at compile time in most cases. There goes 99% of the damage that a JIT tries to undo, along with a lot of the damage that a JIT doesn't undo. Sadly most languages lack these abilities. Rust is one of the few languages that has a sensible story for this, at least for the special case of type abstraction, traits, and lambdas. I'm still hoping for a mainstream language with full blown staged programming support, but I guess it will take a while.
They would optimize it if one particular branch is picked all (or even most of) the time but not statically knowable at any given (extended) call site, and by "extended" I mean up to any reasonable point on the stack. That is not a niche case at all. Think of a square-root function that returns a Maybe Double. At most call sites you can optimize away all matchings, but you can't do it statically. Statically-knowable is always the exception (see the second-to-last paragraph).
> Names should be a purely compile time affair.
That doesn't work too well with dynamic code loading/swapping... But in any case, you're now arguing how languages should be designed. That's a whole other discussion, and I think many would disagree with you.
> There goes 99% of the damage that a JIT tries to undo, along with a lot of the damage that a JIT doesn't undo.
At least theoretically that's impossible. The problem of program optimization is closely related to the problem of program verification (to optimize something you need proof of its runtime behavior), and both face undecidable problems that simply cannot be handled statically. A JIT could eliminate any kind of branching at any particular site -- be it function dispatch, pattern matching or a simple if statement -- and eliminating branches opens the door to a whole slew of optimizations. Doing branch elimination statically is simply undecidable in many, many cases, regardless of the language you're using. If you could deduce the pertinent information for most forms of (at least theoretical) optimization, you've just solved the halting problem.
Whether or not what you say is true in practice depends on lots of factors (like how far languages can take this and still be cheaply-usable, how sophisticated JITs can be), and my bet is that it would end up simply being a bunch of tradeoffs, which is exactly where we started.
Even then you assume the profile resembles the real program input. Not always true.
So GCC supports PGO but nearly no one uses it. On the JIT side at least all programs runing on the common JVMs and .NETs use PGO.
Even with PGO, one can't make all the optimizations one can in a managed runtime. e.g. null pointer exceptions -> dereferencing a null pointer is handled in the rare case in the JVM by segfaulting and dealing with it in a signal handler (after 3x this code will be deoptimized and turned into an if null check). This is an approach GCC cannot take as it cannot install signal handlers, so at best it can do the check and mark the segfault path as unlikely.
That's irrelevant to my objection. Also O3 isn't necessarily a gain over O2, so it doesn't make sense to blanket-compile everything as O3.
> Even then you assume the profile resembles the real program input. Not always true.
That is at least somewhat relevant, but your original comment stated that GCC can not make usage-based optimisations, not that it won't always make the right ones.
> Even with PGO, one can't make all the optimizations one can in a managed runtime. e.g. null pointer exceptions -> dereferencing a null pointer is handled in the rare case in the JVM by segfaulting and dealing with it in a signal handler (after 3x this code will be deoptimized and turned into an if null check). This is an approach GCC cannot take as it cannot install signal handlers, so at best it can do the check and mark the segfault path as unlikely.
GCC can do one better, given dereferencing null pointers is illegal it doesn't need to handle that at all, and thus has no need to optimise something which doesn't exist.
>That is at least somewhat relevant, but your original comment stated that GCC can not make usage-based optimisations, not that it won't always make the right ones.
There are things that PGO can not optimize away that an optimistic JIT+managed runtime can.
> GCC can do one better, given dereferencing null pointers is illegal it doesn't need to handle that at all, and thus has no need to optimise something which doesn't exist.
Unfortunately, null pointer deference do exist even if their behavior is "implementation specific" in C, segfaults are depressingly common. And if you take CCJ as an AOT compiler then GCC as compiler collection can not make that optimization.
I think this points to a philosophical in difference between JVM compilers and GCC+ICC compilers. The JVM optimizes things away because behavior is defined in a certain way, while GCC tends to say you are asking for undefined behavior so I don't bother to do what you asked at all.
So I would not call it do one "better, it really is do one "worse"
And you can do that in the reference JDK by setting the execution threshold to 0.
I read in some forum once, that it was political.
At certain point it became kind of heresy at Sun to suggest AOT compilation. Hence why all Java AOT compilers are only available from third party JVM vendors.
If someone from those days at Sun can confirm or deny it, please do.
I imagine it took Oracle some beating in high performance trading, from the companies wanting to move from C++ to Java, to really change their mind. This is what triggered the value types, JNI replacement work. So AOT was the next checkpoint in the list I guess.
At my last job I worked on Spark processes that took several hours to run. In research, performance is important but you do get to average. So I don't think JIT will go away for that case; it wasn't worth hand-optimising (most jobs were only ran a couple of times), but at the same time I was very glad it wasn't Python.
On the one hand, it intuitively makes sense that JIT compilation is inferior to AOT compilation for an application that will be packaged once on a developer's machine and then run on thousands or millions of devices. Doing the JIT compilation on all of those devices is a waste of energy; it's better to do the compilation just once on the developer's machine.
Also, JIT compilation requires warm-up to get to good performance. The kind of consumer who won't pay more than a few dollars for an app will also be turned off by an app that isn't snappy right away. So the first impression is everything, and a warm-up period before good performance isn't as acceptable here as it is in server applications.
On the other hand, on the major mobile platforms, native apps which can be AOT-compiled have to be distributed by gatekeepers, i.e. the app stores. On these platforms, the only alternative to the gatekeepers is the web. So I have a non-technical reason to want the last major JIT-compiled language on the client side, JavaScript, to be good enough for a variety of applications.
I was going to comment on the advantages of JIT compilation with regard to polymorphism, but then I found that the OP addressed that with his argument about predictability being more important than best-case or even average-case performance.
Finally, I'm curious if the author of the OP would use the same argument about predictability over best or average case performance to argue for reference counting over tracing GC. Maybe Android's ART and .NET Native would benefit from using reference counting in combination with a backup tracing GC to handle cycles, like CPython.
Reference counting is only helpful if directly supported by the compiler, to remove inc/dec pairs. Also it is worse than GC for multi-threaded applications.
Currently I think the only alternative to GC are Substructural type systems. Even C++ guys are now looking into this. The problem is making it palatable to the average mainstream programmer.
"Have tracing jit compilers won?" http://lambda-the-ultimate.org/node/3851
Having an environment (JVM + web server + our code) that is the same across the various hardware and company's teams is really something that helps.
So besides the AOT/JIT discussion, the value of the JVM in itself is really good for us.
A few commercial JVM vendors do offer AOT compilation on their JDKs.
JITs are best for when you want to use universal computers to run better universal computers.
You can easily annotate functions with types in Julia, and pre-compile specialized functions. Julia does very little runtime magic to speed up code, especially compared to PyPy. The speed just comes from a very clever type system + LLVM.
The slow warmup makes JITs a bad choice for quick command-line tools, and the increased RAM/CPU makes them a bad choice for battery-powered devices and those with very limited RAM. The better performance and optimization opportunities makes them the only performant choice for some languages that are hard to optimize AOT (those that rely a lot on dynamic dispatch and/or have dynamic data-structures, i.e. maps instead of class instances). The better instrumentation support makes them a terrific choice for long-running server-side application, where the drawbacks don't matter. On the client-side, unless the language requires a JIT for decent performance (like JS), there is no compelling reason to use one, and that's why Microsoft's decision makes perfect sense, as they've decided to focus on .NET on the client. This has nothing to do with JITs' great utility in general.
One of the biggest breakthroughs in compiler technology in the last decade is Oracle Lab's Graal[1], which can also be used as an AOT, but with less-powerful optimizations. E.g. Graal does this: https://twitter.com/ChrisGSeaton/status/619885182104043520
Graal (alongside its language-construction DSL, Truffle) has yielded implementations of Ruby, Python and JS that easily rival the state-of-the-art with far, far less effort, and also a very decent implementation of C (also with orders-of-magnitude less effort than the competition).
The third drawback makes developing JITs from scratch a bad choice for anyone but the most well-resourced teams, or those that are in no hurry, or those that target very simple languages only.
JITs have another drawback -- less predictable performance -- that matters mostly for hard-realtime code (or nearly hard-realtime), as deopts that momentarily slow-down performance as a better optimization strategy is sought. This is why hard-realtime JVMs offer a mixed JIT/AOT mode, where the hard realtime kernel (which values predictability over perfroamce) is AOT compiled, and the soft-realtime or non-realtime support code is JITted (which you want to run as fast as possible, but don't mind a rare hiccup).
[1]: https://wiki.openjdk.java.net/display/Graal/Publications+and...
Do you have a cite for that? Except for method inlining, which only requires a trivial JIT compiler, I haven't seen them beating normal compilers. The argument that you should be able to use the programs runtime data to perform specific optimizations seems to me to be oversold. Since the JIT compiler must be reasonably fast it eschews many optimization techniques that AOT compilers can afford to use.
For example, the V8 engine has both a JIT an AOT compiler. The engine compiles all code with the fast AOT compiler and then recompiles frequently used functions with the optimizing JIT compiler. If it instead compiled all code with the optimizing compiler AOT, the JIT part wouldn't be needed and you would get just as fast code.
JIT can only beat AOT if you can exploit patterns in the dataflow of the program. Doing that profitably (i.e. the optimization must save more time than it costs to perform) is incredibly hard.
I also don't understand what you're saying with regard to V8. Can the optimizing AOT compiler actually do enough optimizations on a language as dynamic as JS?
z = x + y
See here for a description of the V8 JIT: http://jayconrod.com/posts/54/a-tour-of-v8-crankshaft-the-op... It appears to be just "a plain old compiler" with inlining of method lookups.
In terms of pseudo Haskell types these two types would be:
TraitObject t => List t -> Result
List (exists t. TraitObject t) -> Result
What do you mean? There is no way such optimizations could be done AOT on Ruby/JS code. Similarly, see chrisseaton's comments example on this thread.
> If it instead compiled all code with the optimizing compiler AOT, the JIT part wouldn't be needed and you would get just as fast code.
So why do you think they do that? :) The reason is that many optimizations are not available to an AOT compiler. An AOT compiler can optimize something only if it can prove that no matter what happens, the optimization will preserve semantics. A JIT can do speculative optimizations, i.e. optimizations that may change semantics and may not always be true, but would be true if the program behaves as it has so far. A JIT can do that because if its assumptions are wrong, it can deoptimize. Many, many abstractions are candidates for speculative but not definitive optimizations, such as virtual function calls, pattern matching, if statements etc.
> JIT can only beat AOT if you can exploit patterns in the dataflow of the program. Doing that profitably (i.e. the optimization must save more time than it costs to perform) is incredibly hard.
That is what pretty much what all (good) modern JITs do. But, as I said, a JIT is indeed more complex than an AOT compiler. Thankfully we now have great frameworks such as Graal/Truffle and RPython that take almost all the pain out of creating a good JIT (or a good compiler in general).
Hence X-Dämmerung is "the end of X" for X = Gods|JIT in the above examples.
Yes, one of the problem with jitters is that they are ... jittery.
But then again, I could be being pedantic.
[1] Frankly, no words would do justice to describe the magnitude of Wagner’s historical opus.
Perhaps "Lord of the Rings, for adults"? ;-)
If someone eventually advances the state of distributed revision control systems then they'll get to write an article titled "Gitterdämmerung".
Not so good on the Hamming distance though...
Taking all context into account, it's the “dusk” variety of twilight.