And autovectorization is definitely done by HotSpot (I would guess also by V8, though not sure), but autovectorization in itself is very finicky both AOT and JIT, so there is now a stronger move towards user-exposed controls, e.g. Java’s new Vector API, but C# also had something like that for years. Funnily enough, C doesn’t have any standard way to control vectorization, only compiler-specific extensions.
Also, memory layout is a completely different problem space, don’t judge JIT compilers on that. On numerical code they are very competitive with AOT compiled, low-level languages, both JavaScript and Java. Of course if a given object could not be escape analyzed than the semantics doesn’t allow further inlining and somewhere that pointer indirection have to be paid, but don’t forget that besides benchmarks, most real life code in C/C++/Rust whatever will also happily chase pointers, so the difference is again, not as stark as some benchmark may tell you. Also, this is more related to GC+JITted languages, but are not necessary for JIT alone. With that said, I just realized that there is Julia in this exact niche, that also demonstrates my point very well.
> On top of that, JIT optimizations don't always trigger and you fall back to interpreted levels, which are orders of magnitude more expensive.
An optimization “triggering” is exactly similar to what an AOT compiler would do, only with a smaller time/resource budget. What you are getting at - speculative optimizations - are a different story, and are not a necessity in case of a JIT. And they get deoptimized when the assumptions they made for the optimization are proven incorrect (e.g. in java’s case virtual calls may have been replaced by static ones as only a single class implementing the given interface were loaded at the time - but upon loading a new implementation this will have to be deoptimized). You are right that deoptimizations can be expensive, when they go wrong, or in the worst case go into some cycles, but in practice with V8/HotSpot, these work well enough. The initial warmup is more likely to cause some jitter if any.