JIT compilers are a thing, so this statement is just plain false.
Also, apples to oranges comparison. I’m not saying that in this particular case it is not true, but that can’t be deduced from this datapoint.
Do you know of any JITed languages that approach SIMD optimized native code in terms of performance and overhead?
Even the best ones (eg. LuaJIT) fail to come close to even moderately optimized C code, with much higher memory usage. On top of that, JIT optimizations don't always trigger and you fall back to interpreted levels, which are orders of magnitude more expensive.
And autovectorization is definitely done by HotSpot (I would guess also by V8, though not sure), but autovectorization in itself is very finicky both AOT and JIT, so there is now a stronger move towards user-exposed controls, e.g. Java’s new Vector API, but C# also had something like that for years. Funnily enough, C doesn’t have any standard way to control vectorization, only compiler-specific extensions.
Also, memory layout is a completely different problem space, don’t judge JIT compilers on that. On numerical code they are very competitive with AOT compiled, low-level languages, both JavaScript and Java. Of course if a given object could not be escape analyzed than the semantics doesn’t allow further inlining and somewhere that pointer indirection have to be paid, but don’t forget that besides benchmarks, most real life code in C/C++/Rust whatever will also happily chase pointers, so the difference is again, not as stark as some benchmark may tell you. Also, this is more related to GC+JITted languages, but are not necessary for JIT alone. With that said, I just realized that there is Julia in this exact niche, that also demonstrates my point very well.
> On top of that, JIT optimizations don't always trigger and you fall back to interpreted levels, which are orders of magnitude more expensive.
An optimization “triggering” is exactly similar to what an AOT compiler would do, only with a smaller time/resource budget. What you are getting at - speculative optimizations - are a different story, and are not a necessity in case of a JIT. And they get deoptimized when the assumptions they made for the optimization are proven incorrect (e.g. in java’s case virtual calls may have been replaced by static ones as only a single class implementing the given interface were loaded at the time - but upon loading a new implementation this will have to be deoptimized). You are right that deoptimizations can be expensive, when they go wrong, or in the worst case go into some cycles, but in practice with V8/HotSpot, these work well enough. The initial warmup is more likely to cause some jitter if any.
> Why would LuaJIT be the best?
It was one example out the group of best ones, sorry for not being clear enough.
> Also, memory layout is a completely different problem space, don’t judge JIT compilers on that.
Exactly; you're stuck with the memory floor the language provides you with, JIT or not. This means a fixed minimum cost (resources, heat, electricity etc) that would otherwise not be there.
> On numerical code they are very competitive with AOT compiled, low-level languages
How competitive? I've never seen a benchmark show a JITed language compare to the equivalent program in C, with the few exceptions where the C implementation was written in such a way as to enable the author to say "we're faster than C".
> most real life code in C/C++/Rust whatever will also happily chase pointers, so the difference is again, not as stark as some benchmark may tell you
Oh most definitively. On the other side of that same coin, most real life code in JS/Java/C#/PHP/Ruby consume orders of magnitude more resource than their lower level counter parts.
Often, a lot of that is due to writing idiomatic code in languages where idioms are expensive (eg. `[... new Set(arr)]` in JS), which often also means that underlying libraries and frameworks also use these idioms. Even worse when libraries like lodash/ramda become popular and you have another order of magnitude more overhead compared to the equivalent for loops.
You could of course optimize that, just like you could optimize your C/C++/Rust/Zig program.
> You are right that deoptimizations can be expensive, when they go wrong, or in the worst case go into some cycles
If what you're saying is true than that's great news! My own experiences (>decade ago) of observing other teams experiment with JITed languages showed more pessimistic worst case outcomes (embedded engineers trying out embedded scripting languages).
> Oh most definitively. On the other side of that same coin, most real life code in JS/Java/C#/PHP/Ruby consume orders of magnitude more resource than their lower level counter parts.
Java is actually within the compiled language’s groups’ results with regards to energy efficiency. A good GC can allow to be very power efficient and tradeoff memory for running the CPU — only has to run the GC when it really is close to filling up. After running, it can free the former region in one `nop` by declaring the former region as unused (at least in the moving GC kind).