Hacker News Reader

Comment by Hypx_ | Hacker News Reader

Though I wonder... Some decades ago people made similar arguments with regards to C/C++ versus native assembly. That the only way compiled languages could approach assembly languages on performance was if the code was written in a way that the compiler could optimize. If it couldn't, performance would often tank.

But as compilers got better this become less and less of an issue. By today it has become very rare to need to write in assembly. So shouldn't history repeat itself? That JIT compilers will be so good that it becomes extremely unlikely that you'll hit the performance cliff.

View on HN

I don't think so. Compilers have gotten smart, but so have people. See for example stuff like this[0]. Hot path stuff still sees hand-written assembly even in this day and age because even modern compilers aren't able to provide enough overhead elision guarantees required for the really hardcore optimizations.

What really happened is that thanks to the advances in computing power over the decades, we can now afford to skimp on performance most of the time - putting effort in optimizing only the most critical code paths - and still be able to get away with it.

But when we're talking about squeezing every drop of performance, one of the big challenges with fighting against the JIT engine is figuring out when things deoptimize. This is because there are patterns (e.g. polymorphic code paths) that are inherently impossible to optimize perfectly (because halting problem). As soon as the runtime runs into a "weird" data type, deoptimization is forced to kick in and that can destroy performance. As a developer you have to be extremely mindful of this. Type systems like Typescript help, but you can still be effectively polymorphic (e.g. large union types, `any`, etc) if you're not being mindful of the underlying machine code.

And JIT has one major downside that these benchmarks often intentionally omit in the name of "reducing noise": they usually take a while to warm up to native-like speed. I'm talking in the order of thousands of iterations running at a couple of order of magnitudes slower before JIT speeds fully kicks in.

[0] https://github.com/jart/cosmopolitan/blob/master/libc/nexgen...

Compiler optimizers also optimize only the generated code, but not the data structure layout, which is usually what matters most.

Also, compilers often can't optimize stuff which is obvious to humans. E.g. here is an example of GCC not being able to delete useless code: <https://godbolt.org/z/655KeM33Y> (it's fixed in later versions of GCC, but this version of GCC also isn't that old). Another example is Clang emitting useless code out of nowhere: <https://youtu.be/R5tBY9Zyw6o?t=1620>. So, it's not out of the ordinary for compilers to be really stupid about the assembly they generate.

JIT compilers like V8 are able to optimize from Object hashmaps to structs, but the problem is that it has trouble figuring out how to group arbitrary object literals into the same bucket in order to get monomorphism guarantees (this is a very common pattern among projects that emit large data structures such as JS parser projects like babel for example)

This is partly a reason why virtual dom projects don't expose the ability to replace a virtual dom node constructor with inline literals despite it being a pure deterministic function.

That is why top JITs, do cache their findings between runs.

On Android and UWP's case, PGO data is even shared across devices.

And IBM has a JVM commercial feature where a dedicated JIT server gets used and serves the whole cluster nodes with heavily optimized native code.

Naturally JITs coded on long nights tend to lack such engineering.

Your comment was fine until it leveled a needless attack at other JITs.

Reply on news.ycombinator.com