Ask HN: What is the point of JIT compilation? | Hacker News Reader

Ask HN: What is the point of JIT compilation?

2 pointsmonlockandkey4 comments

For the longest time I have been struggling to understand what is the advantage of JIT (Java, C#) over the static compilation model such as with C++?

The argument goes that the JIT compiler can produce highly optimised code by using run time information of the code hot paths and applying a multitude of optimisations. It is said that is can produce code that can outperform C++.

That sounds great in theory but in reality JIT languages are slower than statically compiled languages. CPU cycles have to be dedicated at run time to analyse and optimise code.

I'm struggling to articulate but in essence my dilemma is the JVM is a marvel of engineering with many many hours spent developing it. However the end result (from a performance metric) is that JIT programs are not faster than statically compiled programs.

So what is the purpose of the JIT compilation model for statically typed languages over AOT compilers?

You make a blanket statement as if it is never (or not even often) true that a JIT that has seen the actual runtime usage patterns can beat C++ and other static compilation. Maybe you haven't seen it. That isn't the same.

One advantage of JIT (eg Java) over (eg C++) static AOT compilation is not needing to build (and test) binaries for each target, nor have the bloat that a static compilation can bring, especially with decent debug support or heavy optimisation. (I'm remembering impossible 2GB library binaries back in the late 90s IIRC.)

JITs support running cross-platform with a single often relatively-small image, faster than a pure interpreter such as Python or shell, often much faster.

I regularly build a small app and test it on my MacBook (Air M1), ship it to my RPi (2+) and run it unchanged on a somewhat different JDK version, and it's plenty fast enough. No cross-compilation needed, no build environment on the target needed.]

And that's before gettin into things that AOT cannot easily do, such as dynamically load or even generate new code at run-time.

Just an FYI, C++ compilers can also optimize utilizing runtime info., this is called profile-guided optimization. Basically you compile the program, start it up, run tests or scripts that simulate typical user behavior, then you take the output measurements/annotations and stick it into the compiler a second time.

An example of one such optimization that could be made, is branch prediction. Basically you guess the result (true or false) of an if statement. Guessing incorrectly often makes an if statement 5-10x (depending on the CPU design) more expensive inside of the CPU pipeline.

The big difference is AOT requires pre-generated PGO profiles while JIT-based VMs can do it in-flight, "turn it on and get free performance" way.

Today, this is supported in

- Java HotSpot and GraalVM Enterprise Edition JVMs (at least wikipedia says so)

- .NET 6+ Dynamic PGO. It's a feature toggle and may become enabled by default in 8 or later versions. Each release sees improvements to what can be PGO-optimized.

JIT works cross-platform with a single deliverable.

Reply on news.ycombinator.com