> No production Java/.NET runtime directly interprets bytecode, it gets dynamically compiled into machine code.
True, with the caveat that:
1. Compilation occurs while the program runs, using some CPU power
2. Compilation occurs on those parts of the code that pass certain criteria for being a hotspot
3. Because of #2, some CPU power has to be used for profiling continuously as the program runs
It can be faster than AoT compilation, but AoT can do much more aggressive optimisations because AoT compilation can use all available CPUs, for as much time as they want to. JiT compilers have to balance the processing power used for compilation against leaving some processing power for the actual program.
JiT performs very well on benchmarks because:
1. It's a small piece of code, run serially (thereby leaving one entire other core just for profiling and compilation)
2. That one small piece of code is run dozens of thousands of times, triggering the JiT compiler to optimise that "hot spot".
3. The "hot spot" is the only code to run so the entire program is very quickly turned into a native-code program, with the best optimisations that the JiT compiler can perform.
In practice (i.e. not benchmarking), the code is large, it doesn't run serially, it uses all cores (especially in performance sensitive applications) and the JiT compiler and continuous monitoring will effectively steal processing power from the program. In benchmarks, the JiT compiler is not using any power that the program would have used.
> And as a result it can dynamically _recompile_ it if it discovers runtime profiling patterns that mean a different compilation can run faster, it can dynamically inline functions into the compilation if pertinent, etc.
There's a lot of "if"s there. They all have to line up perfectly to get well-optimised native code.
> This does not mean that Java/.NET code _will_ always run faster than native static compilation (a quick gander at the real world shows that optimized production code is usually pretty close either way), but it explains how it _can_ run faster.
It can run faster, but that is rare outside of the benchmark environment - it's more likely to run as fast as AoT compiled programs, because the throttling factor in most programs is the data access patterns, not the computation.
In practice, for the usual type of program, you're not likely to notice much of a difference (other than startup time) between AoT programs and JiT programs.