For example, if Intel decided today that CPUs need a new "fast" hashing opcode (I don't know if they actually do), a compiler can't compiles to it, as programs won't work on older computers.
Is it like the API cruft in Android, where "new" Lollipop APIs are introduced for 10 years from now, when no one uses any phones from before 2014?
For example, you could have a function
void hash(char *out, const char *in);
void fast_hash(char *out, const char *in);
void slow_hash(char *out, const char *in);
void (*resolve_hasher(void))(char *, const char *)
{
if (cpuSupportsFancyInstructions()) {
return &fast_hash;
} else {
return &slow_hash;
}
}Normally, asm would do
call slow_hash
Of course the loader could walk through all uses of the pointer to slow_hash and replace them by fast_hash on loading, but that won't work for selfmodifying (packed, or RE-protected) code.
That's the point of doing this in the linker - if you were going to look up the right function every call, you could do that entirely without special linker or compiler support.
call's jxx's are expensive, hense CMOV ^_^
This is the main reason why Apple is now pushing for LLVM bitcode, Android still uses dex even when AOT compiling and WP uses MDIL with AOT compilation at the store.
So regardless of what an OEM decides for their mobile device, in theory, it is possible to make the best use of the chosen CPU.
This is actually quite common in the mainframes, with AS/400 (now IBM i) being one of the most well known ones.
The way AS/400 works, IIUC, is that the compiler compiles to an intermediate byte code, which has remained stable for decades. When the program is first loaded, the entire program is compiled to the native architecture, cached, and then executed like any other binary.
The reason why JIT environments aren't competitive generally with AOT compiling is because of all the instrumentation necessary. A JIT environment is usually composed of an interpreter[1] which, using various heuristics, decides to compile segments of code to native code. But the logic for deciding what segments to compile, when to reuse a cached chunk, etc is complex, especially the pieces that keep track of data dependencies. Also, each chunk requires instrumentation code for transitioning from and back into the interpreter.[2] For this and other reasons JIT'd environments aren't competitive with AOT environments except for simple programs or programs with very high code and data locality (i.e. spending most of the time inside a single compiled chunk, such as a small loop).
Large or complex programs don't JIT very well. Even if the vast majority of the execution time is spent within a very small portion of the program, if data dependencies or executions paths are complex (which is usually the case) all the work spent managing the JIT'd segments can quickly add up. Programs that would JIT well also tend to be programs that vectorize well, and if they vectorize well AOT compilers also benefit and so JIT compilers are still playing catch-up. (One benefit (albeit only short term) for JIT compilers is that some performance optimizations are easier to add to JIT compilers because you don't have to worry about ABIs and other baggage; you iterate the compiler implementation faster.)
I increasingly hear the term JIT used in the context of GPU programming, where programs are generated and compiled dynamically for execution on SIMD cores. But that's much more like AOT compilation. The implementation stack and code generation rules are basically identical to a traditional AOT compiler and very little like the JIT environments for Java or JavaScript. The only similarity is that compilation happens at run-time, but you can analogize that with, for example, dynamically generating, compiling, and invoking C code. Which, actually, isn't uncommon. It's how Perl's Inline::C works, and how TinyCC is often used.
[1] Mike Pall has said that the secret to a fast JIT environment is a fast interpreter.
[2] So, for example, calling into a module using the Lua C API is faster in PUC Lua than LuaJIT. LuaJIT has to spill more complex state, whereas the PUC Lua interpreter is just calling a C function pointer--the spilling is much simpler and has already been AOT compiled.
JIT compilers can beat equivalent AOT compilers by about 20%, or at least, that's the kind of loss you get in HotSpot from not doing profile guided compilation and doing it all AOT instead. So that point seems wrong. If you're comparing Java and C++ well, that is affected by many things and you can quite easily construct microbenchmarks where Java beats C++. In real programs though, it's usually either a tie or the other way around, mostly because Java has a quite low density memory layout (currently).
It turns out that the "hot spot" theory is mostly correct, in that most programs do spend most of their time in small parts of the program, and large quantities of code may only ever be executed once or twice. That's why you can write Java programs that are competitive with or beat C++ programs (see the TechEmpower server benchmarks for some examples of this happening) despite the fact that not all the code is compiled.
JIT compilers tend to be better at unguided vectorisation than AOT compilers because they know what CPU features are available to them at compile time. Again, you may be confusing multiple unrelated factors together: in practice, most C++ programs will probably use vectorisation more heavily than Java programs do because C++ has features that let the programmer explicitly invoke them so the compiler doesn't have to figure out where the instructions can be used by itself. But that is unrelated to when the compiler runs: you can add vector intrinsics to Java and there's a project doing exactly that.
reality seems to agree with the parent. Case in point there are no production level JIT compilers for C/C++.
The fact that a java AOT compiler is not competitive with HotSpot might have more to do to the maturity of HotSpot and the amenability of Java to AOT compilation.
> JIT compilers tend to be better at unguided vectorisation than AOT compilers because they know what CPU features are available to them at compile time.
runtime dispatching to specialized functions makes this a moot point point. I'm not aware of the state of the art, but last I heard HotSpot wasn't particularly great at vectorization.
The 20% comparison is HotSpot in AOT mode (it's being developed) vs HotSpot in JITC mode. So I think it's one of the best figures you're going to get. Speculative, profile guided optimisations aren't going to double your speed or anything like that, but a 20% win is big enough to matter: it's like getting a free additional core on contemporary machines.
Yes you can do runtime dispatching to different functions, but how many apps actually do so? I've seen a lot of software that doesn't bother, or only has a "plain vanilla" and a "low-rev SSE" version.
HotSpot has got steadily better at auto-vectorisation with time. The unreleased Java 9 version has had a lot of patches from Intel go in that make it better at using the latest vector units more frequently.
those that do care I guess? Games, video encoders/decoders. Most custom HPC applications are simply compiled for whatever architecture is running on the cluster and don't bother with anything else.
Regarding AS/400, doesn't the documentation refer to it as kernel JIT?
Thanks for the explanation.
Another alternative is to use the new instructions regardless, then trap illegal instructions and emulate them. This used to be a common way to handle floating point instructions, back in the days when FPU hardware wasn't universal. CPUs with FPUs would run the instructions natively, and CPUs without them would emulate them in software. This was, of course, unbelievably slow, but it worked.
But for the most part, you just generate different code for different CPU capabilities and dispatch, as the other comments describe.
We have in the past received different binaries from vendors to use depending on which processors we're using.
Gentoo linux uses this to slightly optimize binaries - because every gentoo user specifies these flags systemwide, customizes them for herhis cpu, and the whole system is recompiled taking them into account. Performance boost is negligible from my experience (but I used gentoo 10 years ago, maybe it changed).
> Going forward, as new silicon generations are introduced, they will require the latest Windows platform at that time for support. This enables us to focus on deep integration between Windows and the silicon, while maintaining maximum reliability and compatibility with previous generations of platform and silicon. For example, Windows 10 will be the only supported Windows platform on Intel’s upcoming “Kaby Lake” silicon, Qualcomm’s upcoming “8996” silicon, and AMD’s upcoming “Bristol Ridge” silicon.
https://blogs.windows.com/windowsexperience/2016/01/15/windo...
That makes more sense.
I remember that years ago, there was this tongue-in-cheek conspiracy theory floating around that Microsoft and Intel had some kind of deal that new versions of Windows would pretty much require one to get a new computer so it would run decently, in order to boost Intel's sales. Maybe there was something to it? ;-)
OTOH: "For example, Windows 10 will be the only supported Windows platform on Intel’s upcoming “Kaby Lake” silicon" - that does indeed sound like one could run into trouble trying to run older versions of Windows on new chips... I wonder if there are hard technical reasons for this. After all, both Intel and Microsoft had put a lot of hard work into preserving backwards compatibility.
For handcoded assembly, it's not uncommon for code to check a CPU's capabilities at runtime (e.g. OpenSSL checks at runtime if it can use the really fast AES-NI instructions,. If not, it fall backs to other assembly or C implementations). That way you don't need different binaries tuned for every possible combination of available instructions. I don't think any compilers do this automatically, but most of the time generating assembly for the lowest common denominator is fast enough.
Theres more here: