SIMD accelerated sorting in Java – how it works and why it was 3x faster

> In the JVM world, when are things added to the byte code as a new instruction, and when are they wrapped up in a native code call?

These aren't native code calls but compiler intrinsics. The Java compiler (the JIT compiler, that is) translates certain Java method invocations into specific machine instruction patterns.

> Java makes it so impractical to really reason about the code directly that users ends up relying a lot more on tooling for benchmarking and introspection (and hence the tools are top-class)

As others have mentioned, you can dump the generated machine code, but you are correct that Java is, by design, higher level than C++ in the sense that low level details are less under the programmer's control, but the compiler's. So when you see a `new Foo()` in Java, you cannot tell whether an object will actually be allocated on the heap, or perhaps "scalarized" and stored directly in registers. In fact, what happens is not even decided statically and could vary dynamically, depending on what other code does, as compilation is done JIT, and optimisations are decided based on the program's current execution profile. As a result, even though all instance methods in Java are semantically virtual, in practice they are very often inlined, and don't compile even to a call instruction, let alone a vtable call.

Unlike C++, Java's goal — generally speaking — is to get very good average-case performance for little effort rather than give the programmer a lot of knobs to control the worst case. Of course, sometimes there are knobs — just like the Vector API, which is useful in cases the JIT cannot automatically vectorise loops — but the balance is different from C++.

If you use Oracle's own IDE, it will support it out of the box, as it already did on Sun's days.

Then there are other ways depending on which JVM implementation is used.

On OpenJDK's case you can load runtime plugin to do it

https://github.com/AdoptOpenJDK/jitwatch

You can't really have bytecode instructions for the vector API functions because they're generic. They're instead methods, which the JIT compiler can handle specially if it so chooses to. Effectively, for pure computational operations (i.e. not ones doing complex stack operations or jumps), there's no real difference between a method call and a new bytecode (except that adding a new bytecode is a lot more work because everything must be updated to support it, whereas a new method is just a new method).

> You should for instance be immediately able to see if a function is being inlining while you're editing

But it’s a dynamic decision, except in the most trivial cases. How can you know it without running the program?

Android Java is not like proper Java, including how ART works. At least they dropped Dalvik which was comparable to early JVM implementations pre-JIT days, think Java 1.2.

Then there is the whole issue that Google isn't that up the game into making the same kind of optimizations as the Java community, and rather calls into C++ via native methods than optimizing ART more than good enough.

Finally the sore state of proper Java support comes from their beef with Sun and Oracle initially, and now their agenda to push Kotlin no matter what, and every modern Java feature that becomes one red less against Kotlin doesn't help.

Hence why most of their Java vs Kotlin samples are mostly based on Java 7.

If the Android team really cared, they would provide equal support for both languages and then let the community pick up which one they were found of.

Even the recently announced improved support for Android 13 is based on a Java 11 LTS subset, when the more recent LTS version is 17, and Java 19 is a couple of months away.

> The idea that you should allocate as few short-lived objects as possible has stuck with me.

I don’t know about Android, but since generational garbage collectors have become the default, the rule is that short-lived objects are dirt cheap, because they’ll be collected all at once in O(1) with the young generation. And since allocating them is usually just a pointer increment, they can be as cheap as stack objects.

I think this problem occurs at many places in the industry. Developer learns a thing, then keeps trying to apply it two decades later, long after it became completely obsolete.

That's how you get things like people unrolling a loop by hand. Compilers have known how to do it for ages, and have built-in data about what works best on what CPU. 99% of the time they do a better job than a programmer that developed a pattern back when they were writing for a 386.

These days it often works better to write very straightforward, non-clever code and let the compiler figure out how to optimize it.

Android’s Java is pretty much a bastard child of OpenJDK - it is so behind on features that it is truly a shame.

The buffer in this case is the input to a max() function; his point is simply that sorting the dictionary indices before looking them up is a good idea for common data structures like arrays, disk blocks, or anything with caching or readahead.