Why low-latency Java still requires discipline?
chronicle.software
chronicle.software
I managed to improve some applications of ours from several garbage collections per second to several minutes between collections. That _really_ improves p99.
$ java -XX:StartFlightRecording:maxsize=10M,filename=dump.jfr -jar app.jar
$ jfr view all-views dump.jfr > report.txt
$ jfr print dump.jfr > all.txt
There's also async profiler. LLMs just use whatever. Lots of choices.
If you want to collect these, telegraf has a jolokia plugin. It's an incredible combination!
https://blog.vanillajava.blog/2026/06/testing-java-memory-ma...
https://blog.vanillajava.blog/2026/06/why-you-should-tun-cod...
Suggestions welcome, it can be hard to know what to include or not.
Secondly if we are talking about something like real time audio, or sending the maximum amount of triangles per second, then there is zero memory management, the code should be allocation free across all the critical path, regardless of the language.
I.e. all alloc should happen up front, functions should have predictable execution time, etc etc
A lot of the advice is good in general - keeping things simple, generating fewer objects, etc. Profiling with Yourkit or JMH to find and improve slow spots.
This let us build high performance software that had a vast difference in say p90 input size and p99. It involved rewriting third party libraries to let us 3x speed and substantially reduce GC. I think in the end we were around a p99 of 7ms and p90 well under that (the data spread was kbs to megabytes) across hundreds of millions of inputs per day.
Average go, rust, c++ and c will outperform amazing java programs, and the former will also be way way more easy to run, troubleshoot, interpret logs from.
Java is usch garbage in every stack.
C++ or go? Then you'll have to take a very closer look, because the java JIT is wonderful. A masterpiece of several hands, actually.
Of course we are talking generals here. Sometimes the above is acceptable and Java/JIT is just fine. Sometimes it is unacceptable and you cannot use Java/JIT. Know your domain.
Of course in all cases (C, Rust, C++), you have to understand the system and what it is doing. Every language has a standard library that will do things that are not low latency on you. You have to know which library functions will do what, memory allocation and copies are both things that code often does without thought that are incompatible with low latency. No matter what you need to know what your language does that is against you.
There's startup "AOT cache" via Leyden that speeds up startup. Isn't native speed up it's quite a big boost.
Then there's GraalVM that does give you a native image. Real AOT.
To avoid the compilation etc. hit, common practice is to do some "warmups" before serving users. (Another reply has other ways to avoid this hit.)
Handling exceptions is higher latency, but they can/should be optimized out, so you're not hitting exceptions as part of your standard workflow (or even your 1% workflow).
Don't read the above, as you can't use JIT or Java for low latency. I never intended to say that. However, you are going to be fighting the system if you do try to use them. fighting the system even more than C where you have more control
To be more specific, you don't want Java-style-JIT. Julia JIT (which functions closer to AOT in this case) is fine for low latency.
C and C++ are dangerous languages filled with security failings and footguns, and no modern app should be written in them.
It's been my experience that well-written low-level Java code runs at about 75% the speed of good C code. (Of course lazy coders write in cushy Java which is much slower). When written efficiently, Java's biggest slowdown lies in array access (C and C++ array access is fast because it is very, very unsafe). But Java makes up for this in having a GC which will coalesce related objects into the same page and so take advantage of cache coherency effects in ways malloc and free cannot possibly do. I have some allocation-heavy algorithms in Java which are, as a result, significantly faster than well-written equivalents in C.
I've done a lot of performance engineering in both C++ and Java. Every optimization available in Java also exists in C++ but the reverse is not true, which is why C++ is always faster. Every example I have ever seen of Java being faster than C++ was just poorly optimized code. The heuristic I use is that heavily optimized C++ is about twice as fast as heavily optimized Java at the limit. And this requires some non-idiomatic and ugly Java that isn't nice to maintain.
This doesn't necessarily make Java the wrong choice though. Few organizations prioritize absolute performance above all other things. There are practical tradeoffs.
@pron, who works on the JVM and is a C++ expert, has been writing a lot recently about low level languages becoming increasingly impossible to optimize with more LOC and more people working on it (dozens, or even hundreds of developers).
The idea being be a language with an aggressive JIT, moving garbage collector, and bump allocation is going to allocate/deallocte faster than malloc, and get the most important average optimizations at all times, where as the equivalent low level app will struggle to get the ideal allocation pattern the more people that work on it, or be forced to use slow dynamic dispatch. He wouldn't describe a GC as pessimistic, he'd say it's literally faster, and that's why he (a C++ programmer) works on it. The JVM was built by people saying "we are doing a handful of the same things in C++ all the time."
There are specific cases where it's good to have low level control, but they aren't everywhere.
Almost all optimization is architectural in nature. Properly performance-engineered code allocates no memory after bootstrap. There is no possible way even in theory for a GC to outperform schedule-aware allocation from a fixed pool. In most cases the GC is just generic C++ code anyway; that indirection is unnecessary.
An important caveat is that I mostly deal with throughput-optimized code. It isn’t latency-sensitive and this article is about low-latency code. Nonetheless, the schedulers require predictable execution at ~1µs granularity for throughput optimization purposes, which is difficult to guarantee in Java.
I can write highly optimized code in C++ in a straightforward way that doesn’t really have a Java equivalent because equivalent guarantees are not provided as a practical matter.
TBH, even when I wasn’t trying to performance-engineer code I’ve never seen Java run as fast as the equivalent C++.
You mean in the absence of any reference-counting overhead? That doesn't sound like a fair comparison.
> In most cases the GC is just generic C++ code anyway; that indirection is unnecessary.
You can't use the JVM's GCs from C++. C++ heavily relies on RAII, so it doesn't seem like a good fit anyway. I'm not aware of any serious efforts at a high performance GC for C++.
As Herb Sutter points out, moving GCs require non-stable pointers, which C++ isn't suited to. [0]
> the schedulers require predictable execution at ~1µs granularity for throughput optimization purposes, which is difficult to guarantee in Java
Can't say I know much about that kind of issue, how does it work in C++ code?
[0] https://herbsutter.com/2011/10/25/garbage-collection-synopsi...
https://www.youtube.com/watch?v=M_HCG1JPMQE (or any podcast application)
always CAN be given one is a sufficiently experienced C++ developer, with sufficient time and effort (both for the first iteration, and for every future maintenance).
The stars rarely align well on all counts and one may very well be better off with a close-enough, cheaper Java version.
Not true. Many benchmarks have shown otherwise. it is at least competitive in many areas.
> and the former will also be way way more easy to run, troubleshoot, interpret logs from
No language will save you from poor logging practices. If you log every debug log it's not Java's problem. No 1 says you have to log the full stack trace if that's your concern. You can configure / strip / do anything. Learn to use the stack.
May be true between late 90s to late mid 10s. Both Java and JVM has had enormous of work going into it. 2026 the JVM is pretty damn good pieces of engineering.
Everyone that has a minimal experience with these knows it is not true.
While performance ceiling is likely higher in C/C++/Rust (but not Go), when we move beyond microbenchmarks Java provides competitive performance with much better ergonomics. Not to mention strictly superior tooling.
Am I the only one who thinks "pebkac" is a self-own? If you're blaming your users for the tool, it feels like maybe your tool is just hard to use properly. I have no doubt that there are big, fast, low latency Java systems out there, but why does it take so much extra effort to build these systems in Java compared with other languages? Maybe it would be better to have competent developers focus their attention beyond managing Java's shortcomings?
I'm not even hating on Java here--I actually like the JVM quite a bit, but blaming users feels like an implicit admission.
There's no such thing as a programming language that guarantees excellent performance regardless of programming competence.
> why does it take so much extra effort to build these systems in Java compared with other languages?
They didn't say that.
The entire premise of the conversation is that other languages allow us to write idiomatic code and get good performance, and the parent's position was that Java requires "competent developers". So yes, the parent did say that, however...
> There's no such thing as a programming language that guarantees excellent performance regardless of programming competence.
No one made any claim remotely like this. This is a very obvious straw man argument. What was argued was that you can get decent performance from other languages without additional competence beyond idiomatic code.
And then there is real pebkac. Java code tends to be the worst, even though there are more newb-friendly things out there like JS. Sure you can choose to be a good programmer and use Java, but if you're being offered to join a "Java team," that doesn't usually mean good things.