They lack in performance, stability (compatibility), observability (telemetry), productivity, or some combination thereof. They are chosen, of course (especially C#; Go and Rust are far behind), but not as much as Java.
25,377 karma · joined December 23, 2011
Working on OpenJDK at Oracle
Applications that don't upgrade their runtime are usually not in the market for new libraries (or even new features in old libraries).
Why would anyone think that having significantly less than 50% women requires no investigation if they didn't think women are intrinsically bad for a company? The problem isn't a skew in favour of women. If you expect 50-50 and get 30-70, obviously that's something to look into, but you're complaining there aren't people trying to increase the skew further. It's like complaining about the lack of organisations explicitly calling to raise the growth in average temperature as a counterweight to all those who want to lower it, or about the lack of charities promoting deforestation.
That throughput penalty is not very high with generational ZGC. It's not zero, but it's not very high, either. What ZGC mostly does is spread the memory management activity more evenly across the duration of the program (this does have a cost due to barriers being active more, but it's not huge). But we have some work planned to improve ZGC even further, which is why I didn't say I think it will become the default imminently, only eventually.
> ZGC performance is _catastrophically_ bad with <=1 cpus
That may well be true, but the JVM can automatically choose a different default algorithm for these circumstances. Indeed, until very recently, the default for low-CPU environments was different (Serial) than for bigger ones (G1).
In particular, the JDK's GCs are heavily optimised for short-lived objects with high allocation rates. What you want to avoid is temporary data finding itself in the old gen. The newer GCs may dynamically size the young generation to match your program's natural meaning of "short lived", but the longer an object lives, the higher the chances it ends up in the old gen. You want only objects that stick around for a very long time (and have a low allocation rate) to end up in old gen. If you just write naive code, chances are things will work out well. Once you start being clever, you're taking a risk.
So if you're willing to profile your program, with a workload that's representative of production workload (a microbenchmark is useless) on every runtime version and potentially change your "manual optimisation" every six months, you can try. But if not, the advice for the best performance over time is to rely on the platform and let it do its thing. The reason is that the JVM is continuously being optimised for "normal programs". If you're doing anything too clever, you may find that in a future release, your code is making things worse because the optimisations that target normal code don't help your code (or could even treat it as unusual and have it hit slow paths).
I once spoke to a company that were very proud in getting something like a 10% improvement over naive code in Java 8, thanks to some hand optimisation they worked a lot on, only to discover that it caused a 15% regression compared to doing nothing special on JDK 11.
1. Upgrade your JDK for the best performance (as the article says, the slowdown is gone in JDK 26).
2. Don't try to help the GC by pooling objects. Mutating old objects can be expensive, while allocating new ones is cheap (at least for objects that don't do some exceptionally expensive initialisation).
So what you're really getting is, typically, a smaller executable, a faster startup, and lower footprint (which is actually a much more complicated matter, but I won't get into it now) in exchange for significantly higher evolution and maintenance costs forever. This is a good and reasonable tradeoff for small programs, especially CLI tools, and not a very good tradeoff for larger and/or longer-lived programs.
I don't know how long you've been programming, but that's not true. In the late nineties and early aughts I was working on large, performance-critical, soft- and hard-realtime systems, and we only started moving away from C++ when Java started beating its performance.
> The reason was not because managed languages were faster or even had equal speed, but because of the costs associated with memory unsafety (not just security), exactly what Rust prevents (which was of course not available then).
That's a myth, and a fairly recent one. Sure, there were non-performance-sensitive programs written in slow languages for a long time. But the industry was mostly using C++ for anything that needed to be big and fast, and back then "memory safety" was mostly just another type of bug. It was nowhere near reason enough to use slow languages, which is why we didn't use them.
Lack of memory safety is a serious problem, but the claim that it's the biggest issue with C++, let alone the one that's always been considered the biggest issue, is just a myth. Back then it was certainly considered no bigger an issue than the language complexity, compilation time, and even performance issues in large, long-running programs.
> You can see empirical evidence of this, for example, by the post about Aurora DSQL rewrite in Rust
I talk to the people at AWS, and this is not the evidence you think it is. First, their problem was primarily with GC pauses, and it was before pauseless GCs. Second, the codebase isn't very big. Third, because Java and C++/Rust offer similar performance - sometimes one wins, sometimes another - you expect to see exactly that. I can tell you that we recently wrote a distributed cache in both Java and Rust simultaneously (using the pauseless GC). The Java version achieved twice the throughput of the Rust version, and significanly better latency across all percentiles. So sure, on the smaller end, there are programs where Rust would be 2x as fast as Java, there are programs where Java would be 2x as fast as Rust, and on average they're about the same. But over time, Java's advantage starts to show as it makes it easier to keep the good performance over years of evolution.
Yeah, this is not true, and there's no time limit. I mean, maybe some JIT compilers, like JavaScript's have a time limit, but their goal is to run JS at an acceptable speed. Java's JIT is intended to reduce the runtime overheads of AOT compilers, and the only way to do that is by optimising significantly more than AOT compilers, obviously not less (otherwise, we'd just always use an AOT compiler).
You can easily see why there's no time limit if you understood how Java's optimising JIT works. First, code is run in the interpreter and some profiles are collected, then a non-optimising JIT runs and continues to collect profile, and finally the optimising JIT runs. The vast majority of the time is spent waiting for profiles to collect, and so if compilation itself runs, say, even 3x slower, it won't even be perceptible. Also, because we have profiles, we don't have to compile much of the program at all, because we know what the hot spots are. Initialisation code that runs once is never compiled (remember, the focus is long-running programs, exactly those that low-level languages have trouble with).
Finally, the reason sophisticated JIT compilers can optimise more - which is why they're used in the first place - is thanks to speculative optimisation. AOT compilers need to spend a lot of time on optimisation, and even then they are limited, because they need to prove that the program transformation is valid (i.e. that there's no miscompilation). The power of JIT compilers is that they don't. They only need to speculate that a certain profile will continue to be in effect. So if so far some virtual call always hits a certain target, they can go ahead and inline it (not only to the cost of a regular call, but to no call at all, and then they optimise the whole inlined code). If they're wrong, a fault triggers and they decompile the relevant subroutine going back to the interpreter and non-optimising compiler.
> and that C++ and Rust are allocating much, much less than Java and even C# or Go, so a faster allocation scheme is much less needed there
This is true, but the causaility here is that the reason we avoid allocation in C++ is precisely because it's so slow.
> but even in those cases it's usually possible to alleviate the costs with wise organization of allocations
The problem is that this is true in principle. In practice this is certainly true in smaller programs. In larger programs, this work is not easy at all, and you find yourself doing harder and harder work just to keep up.
> including using arenas etc. in some places
One of the reasons I'm excited about Zig (I'm a low-level programmer) is that it makes arenas much more viable. Arenas in C++ and especially Rust are not really a pleasure to work with, and they're viral and a constant maintenance burden. BTW, the reason moving GCs are so fast is that they work quite similarly to arenas.
> and they're also offloaded from the better-optimizing compiler.
It's a worse-optimising compiler. In C++, I use templates to achieve similar optimisation to what Java does, and in Zig I can use comptime, and again, it's certainly possible but it's hard work. You can't let the templates explode all over the codebase, and, as it evolves, you have to go back and profile and take out the ones that no longer help, replacing them with new ones.
Just to tell you a bit about me, I was a C++ programmer for many years, and when Java showed up, like many, I was sceptical. When I saw that the JIT + moving collector hypothesis actually accomplishes its goal in reducing the overheads we were seeing in C++ in many situations, I went to work on the JVM. Back then there were still latency tradeoffs due to GC pauses, but GC pauses no longer exist as of three years ago.
Now, a lot of people, including some of the world's top compiler and memory management experts, believe that the vision of using JITs and moving collectors to address the performance problem of low-level languages is working exceedingly well. We can certainly argue about under which conditions Java wins and under which C++ (or Zig it Rust etc.) win and how common they are, but people who think low-level languages win across the board or almost across the board clearly don't know what's going on. Early on it was people who were sceptical about how effectively JITs and moving collectors could do their job in practice (even though the theory was clear), but these days I think it's mostly people who haven't struggled with performance issues in low-level languages long enough, and just see that for small or young programs they work fine. They always were. Writing a new program in C++ was never harder than writing a new program in Java, and the performance was great (and people weren't concerned about memory safety in particular). The problems came later - in the 5th year, the 10th year, etc.., when the cost of evolution and trying to keep performance good were piling up.
In general, low-level programming languages yield relatively fast small programs, but relatively slow large programs, and with Java/C# it's generally the opposite. The low-level control that helps performance when you're small, starts hurting it when you're big.
That's not the deal I'm getting on either side of this.
> In all benchmarks I saw
If you trust those benchmarks then you deserve whatever you pick. I was talking about experienced experts who understand performance. Low-level languages can help your performance when the program is small and they generally hurt it when it grows large, evolves through many people etc. This is something that people with a lot of experience in low-level languages know.
Obviously, but that doesn't help me if the complexity in the unsafe parts is made worse, while the safety helps the parts where little help is needed. It's not like the danger in a C program is spread evenly, either.
> but will benefit from the performance
Not so much. Safe Rust is faster than Python and Go for sure, but is, on average, about as fast as Java and C#; sometimes faster, sometimes slower.
So right now, when we want control, we need to give up some safety, but weaker things are still helpful.
Also, in low-level code, the problem of "I might forget to do something" sometimes clashes with the problem of "I need to see exactly what operations are done and where". Various kinds of implicitness help with the former at the expense of the latter.
I'm not saying this is universally better than other approaches, but many people who do serious low-level programming would prefer this.
Second, the reason why the compiler cannot infer on its own that a class does or does not need identity without you declaring it is that the use of identity can be in a different module.
It won't, because that is exactly the thing good moving GCs detect and size the young-gen accordingly.
> Using RAM to save CPU is exactly the wrong tradeoff when such a system becomes CPU bound.
Did you mean to write something else, because it's pretty obvious that it's the right tradeoff? If something is CPU-bound, you want to reduce the CPU usage.
> Unless each collection takes less work as the heap increases in size, backing off the GC rate to free CPU instantly drives the system into metastable failure, where the GC becomes more expensive because the GC is expensive.
The whole point of moving collectors is that each collection takes the same amount of work, but you need to do it less frequently as the heap rises. So yes, as they heap grows, moving GCs are supposed to work less. The heap grows as a function of the allocation rate while the CPU devoted to memory management remains the same. That's precisely the optimisation that moving collectors bring.
> Instead of reasoning about this all the time
The whole point is that the GC is what "reasons" about this for you.
> it’s much easier (for me, granted, I am not a typical java developer) to just jam the CPU intensive work on a low priority event queue so that it uses 100% CPU but never blocks low latency stuff, or things about to retire requests.
That's orthogonal. You can do that at least as easily in Java.
> This ends up being easier to deal with than java, since everything is thread safe, allocations are predictable, and there are CPU escape hatches I can use.
Thread safety is orthogonal, and now with ZGC, memory management in Java is more predictable than malloc/free allocators.
> C++ lets me use smart pointers that have exactly the semantics I want, and that are memory safe but racy in practice. Rust makes them actually memory and thread safe, but sometimes adds useless copies, initializations and thread synchronization (or requires unsafe).
Yes, and it's also less efficient and less predictable in the memory management work as programs grow larger.
> When I say tradeoffs", I mean exactly things like "inherent increased footprint", or my earlier "wasting RAM" point.
Well, that is a real tradeoff, but there's a reason why it's a very attractive one for a huge class of applications. There are two ways of looking at this, which amount to the same thing:
1. Because both RAM and CPU are needed for computation, what matters isn't each of their utilisation values separately but only the more constraining or impactful of the two.
2. Because CPU is needed to use RAM, every CPU cycle you spend effectively takes away some other program's ability to use RAM.
This means that for any amount of CPU utilisation, there is some amount of RAM that is effectively free (i.e. has no additional impact), and the more CPU a program consumes, the more RAM it can consume without it making additional impact. It's easiest to see in the extreme case of a program using 100% CPU: no other program can make progress, and so it doesn't matter how much of the available RAM your program is using - it effectively captures all of it whether it uses it or not. But this scales to any amount of CPU utilisation (not quite linearly). What wastes RAM is not using that "free RAM" to reduce the dominant resource, CPU. And what further determines the RAM/CPU "exchange rate" is the RAM/CPU ratio offered by the hardware, which is more RAM-heavy than some appreciate (it is very hard to find a metal or virtual deployment with less than 1 GB of RAM per core these days - taking into account partial cores in virtual machines - outside of embedded devices).
This means that if you have a memory management algorithm that uses more RAM to help reduce CPU as CPU utilisation rises, that's usually a good thing. And moving collectors work exactly like that. The heap overhead in a generational moving collector is only a function of the allocation rate, and a high allocation rate also means high CPU usage.
My colleague, the main developer of ZGC these days, gave a keynote about this very subject at ISMM: https://youtu.be/mLNFVNXbw7I
> I view them as just years of continued tuning of tradeoffs (heap size vs CPU cycles vs memory cycles), while you them as major breakthrough that makes garbage collection much more desirable. Is that fair?
I say that for many years, the main practical, most "felt" tradeoff of moving collectors has been their STW pauses. With pauses eliminated, there is a qualitative change in the attractiveness of moving collectors, making them more appropriate than other memory management techniques for an even broader class of applications than before. Previously, applications that were very sensitive to tail latencies didn't want moving collectors; now, the tail latency is no longer an issue (unless your application's tail latency tolerance is such that a realtime OS is needed). In other words, I'm saying that moving collectors' most impactful tradeoff is now gone.
"Easier" perhaps isn't the right word (in terms of reducibility, it certainly isn't), but FPT is just a way to demonstrate that NP-complete problems can be different from each other in ways that matter (with respect to "natural" instances).
> Now, take a cryptographic instance, translate it into a CNF formula and call a SAT solver, I doubt it will shine.
Exactly. And researchers are very much interested in these differences that arise in natural instances and are hidden by "crude" reduction.
That's simply untrue. You can write arbitrary proofs about programs in Turing-complete languages, too. In fact, most formal proofs are of programs written in Turing-complete languages.
> but languages like Rust or Idris make it simple because they force correctness early in the coding process.
Rust doesn't actually let you do that, though. In terms of the expressive power of proof, it is far closer to C than to Idris (in fact, from Idris's vantage point, Rust is almost indistinguishable from C).
There's a lot of art to using formal methods around how to specify the system at the right level of abstraction (to make verification tractable) and how to specify the correctness properties so they can be easily evaluated. Even with AI assistance as it currently exists, users need to know formal methods well enough to at least understand the specification of the system and the correctness properties, which requires ~90% of the effort of learning formal methods in the world before AI.
But the real hope is that one day AI will be able to use formal methods correctly on its own, benefitting those who don't know formal methods. AI can sometimes do that today, but sometimes isn't good enough for people who don't know formal methods. It is certainly possible that soon enough AI will be able to do this more reliably, but then we get into the hard problem of speculating the "AI future". It is very hard to predict what an AI that can take over the art of using formal methods cannot do. Predicting that AI will be able to do that yet not be able to collect requirements and build software autonomously, or even come up with the idea for what software to build in the first place, or even replace the software's users seems arbitrary to me. In other words, if people think AI will take care of the verification letting us focus on requirement validation, my question would be, why wouldn't an AI that knows how to verify also know how to validate the requirements? For that matter, why wouldn't it also know how to replace the users altogether?
But my favourite example (shown here in Java) demonstrates the difficulty of analysing simple, realistic programs without necessarily being undecidable:
long foo(long x) {
if (x <= 2 || (x & 1) != 0)
return 0;
for (var i = x; i > 0; i--)
if (bar(i) && bar(x - i))
return i;
throw new Error();
}
boolean bar(long x) {
for (var i = x - 1; i >= 2; i--)
for (var s = x; s >= 0; s -= i)
if (s == 0)
return false;
return true;
}
Generational moving collectors are a very powerful memory management optimisation, but because they necessarily require some "interesting" FFI layer between the ordinary heap and any passing of pointers between the program and the hardware/OS - the very thing low-level languages are designed not to have - this is a powerful, general optimisation (not perfect, but extremely useful in a wide class of programs) that is not available to low-level programming languages. And it's not the only one, BTW. JIT compilers are also designed for "global" average-case optimisations at the cost of precise low-level control over the worst case, which is another thing that low-level languages trade away.
In the most simplistic way, I would say that the precise control that low-level languages are all about helps their performance when programs are small (and can be manually optimised globally) and hurts their performance as programs get large. They have to give up on some optimisations that come at the cost of ceding low-level control, and that includes moving collectors.
I'm not sure what you mean by "complexity classes of first-order systems" and by "in the abstract".
But it seems like you're asking about the intersection of computational complexity and formal systems, and there's definitely work there. I already mentioned proof complexity, which analyses the number of deduction steps needed to prove something in various formalisms, and there are famous undergrad-level examples, such as TQBF (https://en.wikipedia.org/wiki/True_quantified_Boolean_formul...). But an intersection that is of more interest to me, as I'm interested in software correctness, is that of the model-checking problem.
Now, many people are confused whenever the model checking problem is discussed, because they confuse it with model checkers, which are a set of algorithms intended to solve the problem, but complexity theory is typically interested in the inherent difficulty of answering problems regardless of the algorithm used to do it. So the model checking problem is that of determining whether a formula in some formalism implies another formula, and its inherent complexity exists regardless of whether this question is answered via a formal proof or by some technique involving the logic's semantics. In the context of software verification, the model checking problem is that of determining - by whatever means - whether a program satisfies some non-trivial property.
Philippe Schnoebelen has some papers on the model checking problem in temporal logic (https://lsv.ens-paris-saclay.fr/Publis/PAPERS/PDF/Sch-aiml02..., https://lsv.ens-paris-saclay.fr/Publis/PAPERS/PDF/DLS-jcss-p...). One of his findings that I've found most interesting with regards to programming is that programming languages cannot, in general, make answering the question of whether a program satisfies some property any easier. This result is surprising. The reason is that without a programming language, we could describe a program as a huge state transition graph (this is called a Kripke structure). In that representation, it's been proven that verification is linear in the number of states, i.e. there is no general approach that is faster than brute-force. Now, the size relationship between a program written in a programming language and its Kripke structure is easily exponential or more, so if there were no algorithm that's better, in the worst case, than a brute force of the Kripke structure, then obviously verification is intractable in the size of the program. However, the number of Kripke structures of size N that have a succinct representation in some programming language is far smaller than the total number of Kripke structures of size N. So it could have been the case that analysing programs would have been easier than analysing their Kripke structure (while ignoring their representation in the language). But Schnoebelen proved that this is not the case.
He also proves that program decomposition (and verification of each component separately) cannot, in general, make verification any easier (i.e. the model checking problem isn't FPT in the number of program components).
These results are far more recent than the hopes expressed in the seventies and eighties that we'll be able to prove the correctness of all/most/many programs we write, and indeed, even though the results talk about the worst case, what we've seen in the last few decades is that the power of program verification indeed behaves more like the worst case than something far from it. The gap between the size of programs we can verify and the average size of programs we write has only widened (what saved the day has been the effectiveness of unsound methods, but that's a whole other discussion).
> but that the overall improvement likely has more to do with improvements in CPU speeds and RAM size, than the latest GC version (which tends to simply make a different set of engineering tradeoffs).
Well, the biggest improvement has been the creation of a new "pauseless" collector, ZGC, with a novel GC algorithm (at least for OpenJDK), which does _zero_ GC work in STW pauses, i.e. no scanning, no marking, no moving. In particular, even roots, including stacks, are processed entirely concurrently with the program. The main practical impact of that has been saying goodbye to GC pauses, and getting low latency, that is perhaps even more predictable than malloc/free (and obviously, still has higher throughputs in a large class of interesting programs). The tradeoff is the usual footprint tradeoff, which is the core of moving algorithms, as well as more CPU cycles compared to STW collectors (but again, still less than malloc/free in many programs). The additional CPU can, of course, be compensated for with an even larger heap.
The general idea is to use RAM chips as hardware program accelerators, but in the past latency was also something you had to sacrifice, and this is no longer the case today.
> we're still not studying the nature of computation, we're studying the cost
That is one way of studying the nature of things, sort of like the use of the Hamiltonian in physics, especially if you're interested in problems and classes of problems, and their broad similarities and differences via reductions, rather than in specific computations.
> The higher orders of the field are decidedly not-physical at all (and pedantically, hypercomputation isn't strictly computation).
Hypercomputation isn't really a big part of the main thrust of complexity theory, but (computable) oracles do very much play an important role in complexity theory, even in its lowest complexity classes, such as in the relativisation barrier, which shows that some proof techniques cannot separate P from NP.
> I did also mention Landauer's principle being plausibly challenged,
That's not complexity theory, at least not the standard theory, which treats time and space (or circuit size) more abstractly than concrete physics. There are, however, theoretical reversible models, just note that they don't yield different "classic" deterministic complexity classes (i.e. they do not yield exponential differences).