Quick Tips for Fast Code on the JVM
gist.github.com
gist.github.com
My process for writing fast code on the JVM:
1) Measure. Set up a benchmark so you know whether you're on the happy path or not, or whether you've fallen off a performance cliff, for whatever reason. Make this part of your testing suite with some kind of notification for regressions.
2) Start small, ideally with a do-nothing loop over the input. This gives you a baseline; you can't go faster than a do-nothing loop (presuming you can't skip part of your input, which is an algorithm problem, not specific to JVM optimization).
3) Incrementally build up your algorithm and pay attention to when it falls off the performance cliff, using your benchmark from (1). If and when you fall off the performance cliff, that's when you start bringing in tricks like avoiding new, ensuring call sites are monomorphic / bimorphic, avoiding boxing, reducing pointer indirections and other cache friendly code, etc.
Another trick to consider is playing around with inlining, but not in the way you might think: try pushing infrequently executed code (conditionals) one level deeper in the call stack (i.e. making the body of an if-block into a method and calling it). The idea here is to encourage inlining of method doing the calling. Inlining is where the JVM gets to specialize your code to the specific situation at hand, but the JVM is reluctant to inline big methods because it has a time budget. So you need to help it.
Hence the advice "Keep Things Monomorphic or Bimorphic" !
On hot paths, I often find it better to just avoid iterators in the first place than trying to clean up the mess later.
but from my own experience, you usually do not have to optimize that far, removing that 'synchronized' block, is far more common.
I'm not a pro of Android, so i wonder if the AOT system of Android (ART) has an option to provide a profile like you can do with gcc. It seems a better idea than to have to rewrite all the codes, third party codes included.
http://psy-lob-saw.blogspot.com.tr/2014/12/the-escape-of-arr...
1- The author does tell you to understand your hot path. However in Java it is often difficult to pin down GC related issues and just easier to prevent them in the first place. I've tried to go through code bases others have written top clean up allocation problems and they become so embedded in the code that it is easier to just rewrite. Or some issues like finalizer/weak ref issues are very difficult to find in your measurements.
2 and 3- I've never seen this work. This is a great way to get a bunch of faulty analysis or not even know you're going off track to begin with.
This article and many of the tips are written when you are trying to be "as fast as possible" and where each microsecond counts. I've seen a lot of projects go down in flames where they treated performance as something secondary the can be written into the code in a second pass. When you ate having to deal with issues like the article describes you should take a more first principles approach and keep your eye on the proper idioms inn the first place.
If you need mechanical sympathy throughout you still need solid measurements, you still need to know when you fall off performance cliffs, but you should be experienced enough not to be misled by articles like this one. That last bit was my biggest concern. Because very very few Java apps have performance bottlenecks throughout. That's more typically seen in game engines, where the whole app is a hot loop, and they're not often written in Java.
As to whether it works or not: I've seen it work repeatedly for scanning and parsing. I don't know your situation and can't say why it hasn't worked for you. But again, measurement is the most important bit, and you still need to notice the cliff dropping.
You won't ever notice yourself going off the track even after youre done you likely won't notice.
I do a lot of latency and throughout sensitive work, and far more often the bigger problem is losing a few tenths of a percent on each poor decision in your hot path. It becomes very very difficult to fix.
Also, I think the first rule of microbenchmarks on the JVM is don't do them. The second rule is to use a proper setup for these benchmarks such as JMH due to the complexities introduced by the JIT and GC.
Make use of JVM and CPU performance counters. They're amazing for discovery and understanding of performance matters.
Quick googling found these:
https://zeroproductionincidents.wordpress.com/tag/jvm-perfor...
http://www.peternier.com/projects/overseer/overseer.php
Make sure you also test in actual application context. Microbenchmarking runs into risk of consuming shared CPU resources (such as L1 cache) and exhausting them when running an actual application.
In other words, something that is fast in a microbenchmark can perform badly in an actual application.
Other than that, one thing no one mentioned is to arrange data in cache-friendly format. Sometimes it might mean ugly things like arrays of primitive types instead of arrays of objects...
False sharing is another real performance killer in parallel code.
IMHO, writing high performance Java code is a lot harder than high performance C/C++.
I said: reducing pointer indirections and other cache friendly code (just saying!)
I agree that it's harder. It's harder because the platform is unpredictable and capricious, it's liable to change from one release to the next, and if you have C-oriented ideas around mechanical sympathy, they can lead you astray. For example, manual loop unrolling is seldom a win in Java (often, the array access checks kill you). What you want is to make your code simple enough for the JVM to unroll it (it'll be able to prove to itself that you don't exceed the bounds and elide checks, especially if you help out with an explicit comparison up front). You're in a dialogue with a reasonably smart compiler, and it's a bit of a guessing game hunting for the happy path. That's why my step 2 and 3 involve incrementally building up your algorithm: you want to stay on the happy path, because if you stray off it, you need to get back on to it again before you change more stuff.
Usually the way to go is to write straightforward code (common idioms get more optimization effort from the JVM team) and keep it simple enough to stay in the optimizer time budget (splitting it up where necessary).
Just like practically everywhere else.
Besides, it increases code size, and can decrease performance just for that reason, L1 code cache spills.
Intel CPUs have increasingly improved loop buffers as well. If the loop qualifies, unrolling has no positive effect at all.
Low end microcontrollers are another matter, that's where unrolling can still sometimes yield significant wins.
I think the biggest issue I had with high performance Java is lack of memory alignment control and (at least last time I checked) vector intrinsics. JNI time...
Maybe you didn't have multiple of 4 uops in the loop? Or hit some other (?) loop buffer limitation.
Dependency chain splitting is also indeed a good reason to "unroll". Although it's a bit more complex transformation than simple unrolling.
I haven't had to write high performance (numerical) code in Java, but my instinct is to doubt that it's even possible. There are too many levels of abstraction and complexity, which are constantly changing. Does some JIT heuristic get triggered to specialize a method? Does array bounds-check elimination work in this particular case? Does the compiler allocate this array so that SIMD instructions will work? Will the next minor version bump change everything? With C/C++/Fortran you can look at the generated assembly and either beat the compiler into submission, or write the asm yourself. With Java, I don't see a viable path to success.
This is Lisp's "sufficiently smart compiler" problem with a 2000s makeover.
Anyway, this is how the Vectorization works in Hotspot: http://cr.openjdk.java.net/~vlivanov/talks/2017_Vectorizatio...
Java can be used for numerical work but it's not ideal for the reasons you cite. However this isn't fundamental to the technology or language. It's more a problem of incentives. Java isn't used for such work, so the applications that people care about and try to optimise at the JVM level don't tend to benefit from them, so the cost/benefit ratio of such optimisations seem low, so focus goes elsewhere.
C2 can do quite sophisticated auto-vectorisations but I suspect those optimisations rarely trigger because work that benefits from vectorisation was moved into C or assembly a long time ago. It's rather forward looking work. Graal doesn't even focus on it. They put all their effort into optimising large 'business java' type apps, along with language interpreters (Graal's party trick is turning interpreters into fast JIT compilers).
I’d say about 1% of my code has ever required me to return and optimize it.(I usually write apis that are utilized by end user apis). Most of the time my code is more than fast enough and user concerns are more around bugs related to requirements.
In the few cases I have had to optimize its usually been a super obvious problem. For instance there was a Scala API that looped over a list of lists and called the Scala built in find function. That function is linear and incredibly slow I just converted the first list into a hashmap took a call that took 30 seconds to under 3 seconds still slow but order of magnitudes faster. It’s usually been silly stuff like that.
All of this of course depends on the domain you are working in some applications require you to squeeze out every millisecond in which case most of this won’t apply to you.
When that is the case you need to think long and hard about performance, because it needs to not just be fast enough for you, on your machine, but on any conceivable end users machine, for any conceivable end users idea of what responsive enough performance is, for any conceivable end uer's work load, which might be vastly different or larger than anything you intended.
further I don't think it is always the case that faster code is less simple. Often the faster option is more simple. Avoiding unnecessary complexity is often a win for performance too, and understanding memory locality and how the garbage collector work allows you to make smarter default choices which often have no net impact on simplicity
A lot of people don't disagree with this. It's the basis of one of the most-repeated programming adages ever, with its own 40+ year literature of talmudic interpretations, guided meditations and contrarian-yet-ultimately-supportive takes.
It must be in the hundreds by now.
That is an optimisation. I've found that the simplest solution is usually the most efficient.
[1] If by "efficient" you mean time-efficient. If you pick your own efficiency metric, then yes, sure, I guess so...
If this was true, we'd all be using bubble sort and searching databases without indexes, row by row. I can think of very few algorithms or even situations where this is true.
An approach I've seen work is to have dedicated Java and native threads, communicating via shared memory. We have some funky in-house code for this, but I think Aeron [1] should be a good open source option.
What do you mean by pinning? If you mean pinning threads to cores, how does that relate to JNI?
def first[A](left: A, right: A): A = left
first(12, 42) // => 12
Edit: I'm wrong, the example still needs a @specialized version.
E.g. prefer final fields; don't share between threads unless you need to; there is little point micro-optimising a poor algorithm; don't do the same work repeatedly...
Having said that, in a specialised context, very interesting....
Also final doesn't help hotspot nearly as much as the pointers the article gives. Final doesn't help hotspot much if at all.
Specifically on the Java side of things, while not quite as low-level as what the post is talking about, I've seen situations where proper use of logging libraries has given perf increases. It's as simple as log.info("Foo: {}", "bar") instead of log.info(String.format("Foo: %s", "bar")) -- reducing the number of string operations dependant on your chosen logging level. Simple but I've seen it happen.
Shoutout to Brandon Gregg's blog [0] with some interesting stuff on performance, some of which is Java-specific.
We recently moved all logging stuff to use a constant lambda as first parameter, log.info(s -> "Foo: " + s, "bar") so if the logger is not enabled, the constants are discarded by the VM and otherwise the string concatenation is faster than any string interpolations.
And we are using the new shiny System.Logger [1] so the VM logs and the application logs are interleaved. We still have some wrappers to redirect log4j logs on the system logger because few third party libs are compatible with the system logger.
[1] https://docs.oracle.com/javase/9/docs/api/java/lang/System.L...
My solution was to flip the problem, making a separate logger instance for each log level. So something like
Logger warn = Logger.factory(Level.WARN);
warn.log( "message" );
(My actual implementation had one level of indirection, so logs levels could be updated at runtime.)
Then I got nutty optimizing printf, doing JIT class generation per template... But that's another story.
TL;DR: Logging in Java drives me nuts, so I made my own.
if(log.isInfoEnabled()) {
log.info("Foo: {}", someHeavyOperationThatReturnsString());
}Can someone provide some reference on how JVM optimisation works, or maybe JVM generally?
There’s also some old Oracle or Sun white papers on Hotspot, but I’ve never actually read them.
One nitpick to keep in mind: the major JVMs do many things in similar ways but have important differences. In fact, there is a spec that’s much less prescriptive, and JITs are entirely an implementation detail. Often when you read “the JVM does X” it means that the author is talking about Hotspot.
[0] https://docs.oracle.com/javase/9/docs/api/java/util/concurre...