Shenandoah in OpenJDK 17: Sub-millisecond GC pauses | Hacker News Reader

Shenandoah in OpenJDK 17: Sub-millisecond GC pauses

developers.redhat.com

226 pointsBenoitP294 comments

It took me some time working with a non-GC language to fully appreciate how amazing Garbage Collection is. It really makes a lot of things easier, simpler and specially safer. Yes, of course it has some drawbacks. Yes, of course, not every single kind of program can afford to have a GC. But hell, life is so much easier with a GC! I'm very happy to see that there is active research and development happening to make the drawbacks of GC as minimal as possible!

I feel like the key is knowing that your problem domain is fine with there always being garbage collection in the loop. Otherwise there's the risk that down the road you'll have to start tracing the code, rewriting things to reuse objects, etc etc.

> reuse objects

Caching is often not such a bad idea regardless of GC.

Fair, but I think some GC'd languages (Java, JS) end up with a culture of boxed-everything where it's extremely easy to inadvertently throw off a ton of garbage from an inner loop, even doing operations that in other languages would only ever instantiate objects on the stack.

Newer Java runtimes are very conscious of tight garbage creation loops, and will generally either unbox or perform some internal object pooling to avoid GC pressure. Value types (targeting JDK 18? or maybe already in 17) will give developers a tool to create new stack-allocated types for performance-critical sections as well

Primitive types are not in java 17. Hoping to preview it on 18

Seems unlikely, we are getting Loom before that. Don’t forget that we have been teased since 2014

Also, because GC is “lazy,” your program may even end up running faster than one that uses an “eager” memory management, such as the one used in C++, where destructors are invoked, and objects on the heap are deallocated, early on, i.e. as soon as something goes out of scope, rather than when the process is close to running out of its memory quota (which may never happen).

Yeah totally! And this not only applies for deallocation, but also for allocation. A generational GC with bump allocation for the young generation can be much faster than a naive C/C++ application using malloc for every single allocation. Of course you could use an arena in C/C++ to improve this, but it requires thinking ahead and it add some extra challenges to the design. A GC gives you this for free, you don't have to even think about it.

Faster isn't always the primary concern, sometimes memory matters a lot, and keeping memory usage down important (for example embedded, or even if the system just isn't dedicated to this one application so so the "memory quota" is not clearly defined at all times). That a GC might let you control this to a degree and choose the right approach for the current job is good though.

That is a theoretical effect I frequently see mentioned, but never seen materialize in practice. Quite likely it is hidden by other things that affect Java performance negatively.

Cache effects might play a role here as well. Deallocating memory immediately after use allows you to allocate it again when it is still hot in the cache. So eg if you're iterating a structure and doing lot of interleaved allocations/deallocations they are likely to not cause cache misses (assuming the allocator is not dumb and uses some kind of LRU strategy).

With GC and deferred deallocation, you're moving a lot more data between the main memory and the caches, because memory for sure gets pushed out of cache by the time it is reclaimed. And additionally, GC has to touch quite a big number of objects when tracing (and move unneeded stuff to cache, and push needed stuff out of cache). Memory bandwidth is a scarce resource these days.

If you want to see these effects in extreme, try running a GC based program in an environment that is low on memory but has swap enabled. Hitting the first full GC is basically performance game-over, regardless concurrent or not.

On the other hand, manually managed apps can often deal with big chunks of their heap swapped out without terrible consequences.

Everyone else: pauses are bad, we can’t have garbage collection.

Java: pauses are bad, let’s make the garbage collector better.

Much credit to the Java community for ignoring the noise and building something great that lets most applications have the best of both worlds.

.NET is also regularly improving their GC, aren’t they?

Also notably Go, though really all GC languages which care about performance, of which there are quite a few.

Go isn't playing in the same league though since their GC is non-generational and non-compacting. A lot of the difficulties implementing concurrent GCs comes from moving things between generations and compacting.

I had this benchmark[0] in mind while writing the comment, though perhaps Go does well there by avoiding GC rather than having fast GC as-such? That would explain why its performance is close to the non-GC languages.

[0] https://awesomeopensource.com/project/ixy-languages/ixy-lang...

Without digging into that benchmark, Go leans on stack allocation a lot more than many GC languages, and that’s a big part of why Go is able to achieve otherwise impressive performance. The same mechanism (idiomatic value types) that allows Go to elide GC in many cases also permits better cache coherency—there are fewer pointers into random regions of the heap so cache lines need to be flushed less frequently.

Is that a good or bad thing? What’s the trade-off being made here?

https://blog.plan99.net/modern-garbage-collection-911ef4f8bd...

That article is beyond out of date when it comes to Java GC.

Compacting GC tends to have cheaper allocs and longer pauses (allocs need to be moved to different regions of memory and pointers to those allocs need to be rewritten to point to the new memory regions) while noncompacting has the opposite: slow allocations and small pause times.

Though many part of compaction can be moved to a parallel thread, can’t it? The actual stop-the-world time can be quite small that way (using read-barriers though)

I’m not so sure about barriers, but compacting requires rewriting pointers that are shared with the mutator threads, so it seems hard to parallelize.

As far as I know, this is the “secret” of ZGC at least, it replaces the usual write barriers with read ones. So a parallel thread is free to move an object to a new place, and any thread trying to access it will at that point lock said object. This will have a performance impact (the usual write barriers have better throughput because they only “lock” when writing an object), but more of the GC can be parallelized.

But I’m by no means anywhere close to an expert in the topic.

Interesting. What about an application thread (“mutator” in GC parlance) that tries to write to the pointer? Does that hit the read barrier as well? Does this imply worse application throughout to improve GC metrics? If so, I wonder how much worse? (Not asking you per se, but wondering aloud)

Shenandoah is also not generational, at least the implementation I looked at a year ago wasn't.

It's region-based, the regions serve a similar purpose to generations but in a more granular fashion.

The problem is that there is not benchmark for GC, but from my experience the Go GC is very good, aka sub 1ms.

Is this the case very large go heaps? ZGC claims sub ms pause for TB heap size.

Not only the GC, since version 7, most of the Midori stuff has been landing on C# (and to some extent F#), turning into a kind of "Managed C++".

Contrary to the WebAssembly speech, CLR was one of the very first VMs to have support for C like languages, but those capabilities were only fully surfaced to the original Managed C++ (.NET 1.0) and C++/CLI (2.0 onwards).

So now those capabilities are being exposed into C# as well.

For example stack allocation of arrays is now also valid in safe contexts.

You can apply using to any type that has Dispose() without inheriting from IDispose.

Memory slices are now also a thing and native function pointers for example.

> apply using to any type that has Dispose() without inheriting from IDisposable

Last I saw, this was still being debated by the LDT and weighed against similar proposals such as introducing a `defer` keyword like Swift/Go.

You can use it today on C# 9.

Can you link to any documentation/examples?

Yes, although I'd say that .NET is still a little tied up with how fractured their environment is. There is a lot of tech debt sunk into the .Net Framework, and it'll be awhile before it really gets flushed away.

> There is a lot of tech debt sunk into the .Net Framework

I’m working on a website that has been growing since 1996… The chances of us ever migrating to .NET 5 and onwards is basically zero.

Who is everyone else... Rust and Swift?

C and C++?

C++ and Rust.

I think it's mostly targeted at C or C++ users. Swift is garbage collected.

Incorrect, Swift is not garbage collected. It uses reference counting. Read more here: https://docs.swift.org/swift-book/LanguageGuide/AutomaticRef...

As for C and C++, yes, of course. I meant "everyone else" as in the trendy new languages.

Reference counting is technically a form of garbage collection, FYI.

Reference counting deals with operations that have constant time, which I think puts it in a different category than what people normally view as GC.

I'm not sure that's true. Python mainly uses RC, yet most people think of Python as GC'd. I would argue that most people think that GC == automatic memory management.

Yes, I think you are correct actually.

And has a different tradeoff, due to atomic operations being needed for objects shared between threads (and of course, circular references).

But no matter what, it is literally a GC algorithm, described in every book about GCs.

Swift uses ARC (atomic reference counting) which differs from tracing GC. C++ and Rust have similar features but can avoid the overhead of ARC in many cases.

I know ARC is different from tracing GC, I just don't like how some people try to sell Swift as not garbage collected when it is. Python mainly uses RC, and no one would image calling Python not garbage collected. Calling Swift not garbage collected is a marketing trick, saying they make different tradeoffs compared to tracing GCs is true.

I have big and complex server in C++. Not a single explicit memory related call in it. STL does wonders in this department.

It all depends on your application.

I worked on one which was required to respond to a packet from the network within 5 microseconds of the packet arriving.

Please, don't assume all applications are mobile uis, webapps and corporate backends.

I very deliberately wrote “most applications”. Of course there are exceptions.

Your operating system you are running is not just "an exception".

Neither is software running in all your electronics.

All these require software that has some different requirements that typically are incompatible with unpredictability of a garbage collected dynamic language.

Garbage collection is the least of your problems in that case. You can't use a standard OS for that kind of scenario either, Linux has far bigger jitter than that. And if you have a hard real-time system, you could have a hard real-time garbage collector too.

Garbage collection gets a bad rep from garbage collectors that are tuned for throughput rather than latency, but there is no lower limit to latency for garbage collection and there are GCs out there that prove it (like Azul). If you can afford dynamic memory allocation, you can afford garbage collection. And if you can't afford it, you can just preallocate and turn off the garbage collector.

You're incorrect, garbage collection would be the biggest problem for that use case. You have to be really careful even with your C/C++ code, warming up the branch predictor between packets etc, see my article for some pitfalls: https://rigtorp.se/virtual-memory/

> You can't use a standard OS for that kind of scenario either, Linux has far bigger jitter than that.

This is 100% false. It's easy to configure Linux itself to only occupy 1 or 2 cores (one real + the hyper-thread for that real core), and then pin your own application threads to the other cores where no other code will run. This setup has identical application performance (other than not having those two Linux cores) to having no OS at all. 100% predictable, ZERO jitter application code (including user-space networking).

Except…you can still SSH in to your "no OS" Linux box, you have a file system, can run cron jobs, git, gdb, WireGuard, etc. So it's much, much better in practice and what actual high-performance network developers actually do.

Essentially no one has run a literal "no OS" box for at least a decade; everyone who is running on normal hardware does what I just described with Linux because the cost is dirt cheap and it's way too convenient.

You are right not all of them are, some control weapons systems on battleship fleets.

https://www.ptc.com/en/blogs/plm/ptc-perc-virtual-machine-te...

Better deflect that incoming missile on time before the next GC pause.

Let's do the math on that shall we.

Aegis is built for defending against missiles going at most mach 4-5. Mach 5 is ~1700 m/s, so during a millisecond GC pause it will travel slightly under 2 meters. Ship-based SAMs will have proximity-fused fragmentation warheads and even back in the 70s those had a kill radius of >100m (see for example https://en.wikipedia.org/wiki/S-75_Dvina). So even a GC pause at the worst possible moment would not significantly impact the probability to hit the target. Even really fast things in the real world are really slow compared to computers.

(Of course, in practice the Aegis system will only provide midcourse corrections to the missile. The fuse and end phase guidance control software would not interface much with the rest of the combat management system anyway.)

I think you don't appreciate how rockets kill other rockets.

The rockets close head on (so the speeds add together) and then the rocket trying to kill the other one explodes at right distance to the side of the other rocket, exploding in a cone of debris.

That cone of debris must intersect the other rocket at a right point. It can't just hit anywhere, to reliably kill it must hit a specific part of the other rocket.

So, realistically, you have to time your explosion to something like tens of microseconds.

Also getting there is a control loop problem and it requires very precise control. The more jitter in execution of the control loop the worse steering it will be and less chance it will get to the right place at the right time.

I was a weapons engineering officer for the Dutch navy for 14 years; I'm well aware of the mechanisms involved.

First of all, a SM-2 will not use the flight path you describe as they tend to "dive down" on a sea skimming missile. But even the shorter range Sea Sparrow missiles that do behave as you describe use multiple guidance phases. In the early and mid-phase guidance phases, the radar reflections from the target missile will not be strong enough for the relatively small receiver in the missile, so the system must depend on midcourse guidance updates (little bit to the right, little bit up, etc) from the combat management system on board the launching vessel to navigate to a "close enough" location from where the sensors on board the missile can acquire the target. The end-phase guidance and fuse timings are quite critical, as you say, but they are also not really under command of the CMS but rather done by embedded systems on board the missile. Generation of the mid-course guidance updates is not nearly as time critical and won't suffer much from a 1 ms pause. The error in location estimation of the target due to sensor imperfections (limited angular resolution, clock jitter, athmospheric effects, etc etc) will be way larger than the 2 meter from the GC pause.

Impressive, but you need to factor in it is a real time JVM, with corresponding soft real time GC, and hard real time threads.

Might be a naive question, but can't you just disable GC while missiles are in the air? Defer it until there are no immediate threats, and then let it catch up, even if it takes a couple of seconds?

I can't imagine a scenario in which an enemy would be able to saturate the defense system with a continuous onslaught that would make the system run out of memory. You'll sooner run out of SAMs and ammo for that fancy gun that's used to shoot down missiles at close range. And if the enemy can mount such an onslaught, then the ultimate GC will soon run anyway - all memory will be released when the ship sinks.

(I think back to that famous story of a missile with a memory leak, whose designers figured the missile will hit its target or run out of fuel faster than it'll run out of memory.)

I think it was a satirical answer, that GC can indeed be used for even hard real time systems (though in this case it is a specific hard real time JVM implementation that has a maximum allowed time to spend on GC). Contrary to what many thinks, hard real time is not about speed, but predictability. And calculating even I don’t know how many missile paths is computationally not too hard, compared to eg. even a computer game.

But to answer your question, there are cases where the JVM is used without GC and it runs for a day, and gets restarted at night. Afaik some bank does it for some form of HFT.

Actually, if you think about it, the missile is already equipped with ultimate in garbage collection:)

> Actually, if you think about it, the missile is already equipped with ultimate in garbage collection:)

https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...

"[...] Since the missile will explode when it hits its target or at the end of its flight, the ultimate in garbage collection is performed without programmer intervention."

Reminds me of an argument I witnessed at a meetup:

Dev A "You should use X"

Dev B "We tried X, it was way to slow"

Dev A "You must be doing it wrong, I use X all the time and it's very fast"

This went back and forth like this for a good 15 minutes before they realized they where both getting the same performance, but just had a very different definition of "fast" and "slow".

>I worked on one which was required to respond to a packet from the network within 5 microseconds of the packet arriving.

Sounds like HFT? But then your competitors are probably using an FPGA to respond in 300 nanoseconds, so good luck with that 5 microseconds tick-to-trade.

No, that could be a lot of things. SCADA is what came to my mind.

Why do you think everything has to be either manually submitted order or HFT?

Sending a response within 5 microseconds of receiving a packet seemed like a very HFT thing to me.

I don't even know how you came to the conclusion that I think that "everything has to be either manually submitted order or HFT" but that's on you.

This is possible with a mostly Java application (assuming you're willing to venture into user-space network stacks with either a JNI bridge or shared memory with a C/C++/Rust). You only really people go out of their way for this when doing HFT or very high throughput networking apps.

I was testing ZGC the other day on my machine (24vcores, 64GB RAM) and it ate through about 4GB/s of garbage with about 10milisec of pauses over 220sec of runtime (24 threads doing nothing but allocations). G1 ate through about 2x as much garbage but with higher pause times. It's amazing how far we've come.

I hope this means we can get more games in Java since I'd like to code one or two in my favorite language :)

Wasn't Minecraft written in Java?

Yes, the original Java edition is still in Java. There's now a C++ edition called bedrock as well.

Yes, there are plenty of games written in Java, but afaik there are no big game engines written in it. I'd love to be proven wrong regarding this.

Yes, but as someone who ran a medium-sized server (200ish players), depending on what your map looked like, you occasionally found yourself spending a lot of time doing GC tuning, especially back in java 7 days when we didn't even have the first generation of G1GC.

Ah, good times. I currently work in Hadoop and it's honestly stupid how much tuning a Spark job is like tuning a Minecraft server (from the JVM level anyway). Life's path is strange sometimes.

It was, although it got bitten at one update explicitly by the GC: entity positions moved from a bunch of arrays, to a single Position() class for each entity, leading to memory allocations that murdered performance.

But then again, that's a problem you'd get without GC too, just maybe a bit less harsh/unpredictable.

> it ate through about 4GB/s of garbage

Perhaps a nitpick: 'moving' garbage collectors [0] like Shenandoah don't 'eat through' memory occupied by now-unreachable objects. They work by copying/moving the still-reachable objects. Their performance is a function of the number of still-reachable objects, and their size, rather than the unreachable objects.

[0] https://en.wikipedia.org/wiki/Tracing_garbage_collection#Mov...

Friendly reminder that Minecraft: Java Edition, is still pretty active and now runs and ships with Java 16 (MS Build of OpenJDK).

Gaming in Java has always been possible.

Thanks, I still prefer exactly-zero ms pauses. Maybe that's just me, but I don't think so. And, I don't think I am giving anything up by skipping them.

You don't see any benefits to garbage collection at all? Would you even prefer to do your own memory management if you had a magic GC which never paused? I'd be interested in your reasoning if so.

I'm perfectly happy to have all you lot depending on GC.

And, I don't "do my own memory management". I just don't rely on a GC to do it.

It may be constructive to say what you do do, instead of leaving us to guess and telling us we're wrong.

I mostly use "}".

You write programs so short and uncomplex that you rarely need to free memory?

Ever heard of RAII?

That's still memory management, because you need to worry about lifetimes.

You only need to worry that data that you need doesn't go out of scope.

It is automated, not manual, memory management. Just not GC.

I interpret it as meaning they don't use dynamic memory allocation.

False. Use of dynamic memory allocation without GC, without reference-counting (which is GC) and without manual memory management is the usual practice in C++ and Rust.

Reference-counting GC is available when it is not too costly, which is common, and more convenient, which is rare. Ordinary automatically-generated destructors handle almost everything, almost all the time. Once in a great while, performance demands a concession such as an arena allocator, which also is not "manual memory management", and also not GC, and is radically cheaper than either one.

Memory isn't the only limited resource which needs to be managed. File handles, network ports, etc. all all need proper design to avoid leaks. If you are used to only the allocation and never the cleanup, your design style won't hand effective management of these other types of resources for which GC isn't an option.

I'd use Java if it let me annotate when I want things to be deleted.

This strikes me a somewhat narrow view of the world. Using a GC is an engineering trade-off like any other.

Taking a dogmatic view on an engineering trade-off isn't good engineering, and closes you off from exploring other, potentially better, ways of solving the problems you actually care about. (Which probably have nothing to do with the details of memory management)

Something tells me you wouldn't say "I prefer exactly no borrow checker [in Rust], and I don't think I'm giving anything up by skipping them".

Wrong on both counts.

> Wrong on both counts. "Tradeoff" implies giving something up, and I just said I hadn't. And, I am happy to abjure the borrow checker; it doesn't solve any problem I have. Either.

(Comment was edited while I was writing a response, original comment I responded to above)

I'm glad to hear that you've taken the time to understand how GC work, and the trade-offs that languages make when use them.

I would be curious to learn more about what you're research as has shown with regards to using a GC in your problem space. It's always interesting to learn about areas of engineering that unique requirements.

> I don't think I am giving anything up by skipping them

My understanding is that there are some lock-free algorithms that we do not know how to write without garbage collection [Keir, 2004], so you strictly are giving things up, because you can't use those algorithms.

(But I'm not completely up to date on latest lock-free work.)

I optimize these days mostly by deleting lock-free apparatus.

If you don't use a GC then you're either using reference counting or manual memory management. Those have their own costs. With RAII popping stack frames means freeing resources recursively. If your free() implementation takes a lock then that can block other threads too, for an arbitrary time if the object graph being released is large.

A garbage collector can do that work concurrently.

Not using GC. Not using reference counting. Not using "manual memory management".

Yeah, right.

CppCon 2016: Herb Sutter “Leak-Freedom in C++... By Default.”

https://www.youtube.com/watch?v=JfmTagWcqoE

Better making use of the best practices advised by Herb, otherwise those destructors are going to surprise you.

It'll surprise him anyway, eventually. After all--humans aren't perfect programmers.

It has been more than thirty years since a destructor surprised me. Herb was unlikely to have been programming at that time.

So what are your credentials in the C++ community to top those from Herb Sutter?

Been on ISO C++ Standard committee for much longer than Herb? But I have no disagreements with Herb.

And authored impactful C++ books and ISO C++ papers such as?

I have no need to "top" Herb. He says nothing inconsistent with anything I have said.

Apparently yes, when you place in doubt his advices,

> It has been more than thirty years since a destructor surprised me. Herb was unlikely to have been programming at that time.

So apparently you have skills that top one of the major C++ community figures.

Being so, the C++ community at large would appreciate those valuable insights.

The only conclusion one can draw is that you are wholly unable to comprehend what you have just read, or aggressively don't care what it says. Either way, enough is enough.

Erm something is managing your memory somewhere, even "the computer explodes before it runs out of memory" methodology used in guided missiles is a form of manual memory management, just a very unusual one.

Same applies to the "program terminates before it runs out of memory" approach.

Yes, something is managing my memory. Just not GC, and not reference counting (which is also GC). As I said.

C++ has destructors, and owning pointers (called "unique_ptr"), and also arena allocators for when those are useful. Rust has its Drop trait and analogs of the other things. (Use of arenas with Rust standard library containers is approaching maturity.) They make memory management automatic and wholly painless. Both languages offer reference-counted GC for places where that is helpful, but such places are rare, and where used typically burn an unmeasurably small fraction of runtime, with no "pauses".

Using destructors just reference counting done the hard way.

> burn an unmeasurably small fraction of runtime

That’s also known as a very short pause.

> With RAII popping stack frames means freeing resources recursively.

Your "GC pause" simply happens on scope-exit.

Except, without the pause.

Oh, those cycles are free? That's not how I remember having to plan compute budgets and deadlines when I worked in real low-latency programming.

When I developed programs in Fortran 77, I did indeed have zero pauses, because all memory was allocated statically at the program start. With any form of dynamic memory, you have effort for managing that dynamic memory and thus pauses. The different technologies only result in those pauses happening at different times and other consequences. Modern GCs at least allow most of the necessary computation to happen in a parallel thread.

And not contributing to the discussion.

At least bring some alternatives to the table or explain what you are doing and why you mean it's so much better..

> With RAII popping stack frames means freeing resources recursively. If your free() implementation takes a lock then that can block other threads too, for an arbitrary time if the object graph being released is large.

For every RAII implementation I've seen, the recursive part is not within the free() implementation; the recursion happens outside the free() (or equivalent) calls, and free() (or equivalent) is called separately for each step, to release the memory for just that step. The time taken by each free() call is independent from the size of the object graph being released.

> Thanks, I still prefer exactly-zero ms pauses.

You're not using caches, then?

If your cache misses are measured in milliseconds, you have a bigger problem than GC pauses.

They just need to have not-exactly-zero ms pauses to violate the requirement above - which they have.

Cache misses are not GC pauses. But GC often causes more cache misses.

Better never call the OS memory manager then.

Also, avoiding GC pauses while still being subject to interrupts and context switches is a common kind of tunnel vision. Non-GC languages are great for systems programming, but if GC wasn't going to be your biggest source of jitter anyway then using one just means more work and/or more bugs.

...not using one...

This is why those of us who make low-latency systems turn off interrupts and context switches.

Not using a GC does not, in fact, make more work or more bugs, given a language that provides resource management facilities, such as C++ and Rust.

Programs using GC do typically leak, and most GCs make leaks much harder to find and fix. Most GCs even make it hard to determine if you have a leak.

I know you think you're a perfect programmer, but the fact is that most programmers aren't. Of the hundreds, possibly thousands, I've worked with, maybe one in twenty could be counted on to write non-trivial code without introducing new object-lifetime issues that every programmer on that project for ever after would have to keep in mind when interacting with that component. Yes, that includes quite a bit of C++ with RAII etc. but also with threads, callbacks, and ill-fated attempts to avoid copies by using references and move semantics. (C++ RAII is a hack BTW, less a resource management facility than a primitive you can use to build one, always with the sad side effect of forcing programmers to wrap their cleanup code in classes even when it's an ill fit conceptually.) So yes, in practice in real-life teams, manual memory management does lead to more work and more bugs. Only somebody who has never seriously tried to use more than a small fashionable subset of languages could claim otherwise, or that they're not using memory management at all when in fact that's only true for toy programs.

I am ready to believe that you, personally, introduce numerous bugs in code you write, and then blame them on averyone else and on everything but your own bad habits.

But that experience does not generalize. Am I spectacularly lucky in my generally competent colleagues? I doubt it.

Well, apparently you are indeed.

https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76D...

https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-s...

https://support.apple.com/en-us/HT212805

https://support.apple.com/en-us/HT212622

https://support.apple.com/en-us/HT212531

But maybe Apple, Google and Microsoft just aren't able to hire the right kind of highly skilled C++ devs that would write such perfect code, despite having a seat at ISO, and being clang/LLVM contributors.

Apple, Google, and Microsoft all employ very many competent programmers. But they share the problem that they need to hire many more programmers than are both competent and eager to work for them, and so must make do for the rest. It is a much easier task to select a place to work with competent colleagues. Thus, there is no inconsistency.

Perfection is not needed. Ordinary good code suffices. Good code using modern C++, or current Rust, is easier than in older languages (among which count older C++). When bad code is extra work, it becomes an unattractive alternative even to the lazy, leaving its production mainly to the masochistic and the aggressively incompetent, who are often the same.

So the ideal programmer matra that C folks keep stating that has unfortunely taited the C++ community, despite the facts of security exploits and no such people actually existing for the last 40 years.

Here is the thing, when the ISO C++ leaders, and major C++ contributors, push for a change, it is time to start doing some self reflection.

It would help discussion a great deal if you were to actually read what you are about to reply to, and reply to that, rather than just making up something entirely unrelated that you wish had been said instead, and then spouting foolish remarks about something no one has seen or said.

That personal shot was just childish. The moderators chose to ignore the flag, but it was deserved anyway and that's all it deserved.

Exactly zero pauses only exist if you never free memory, which for most purposes is not an option. In most languages, freeing all all the heap-allocated objects of a big array will take longer than in a language with GC. The reason pauses tend to be bigger in a GCed language is because all the freeing is being done at once, instead of every single time an object is determined to no longer be needed.

In any case, the tradeoff you're making is that you have deal with memory. Even in a language like Rust, you need to care about references, ownership, struct vs heap and possibly even reference counting. You might even be bitten by "running out of memory" due to memory defragmentation.

In most cases, you don't need to care about these things in a GC'ed language, at the expense of larger memory usage and possibly noticable pauses.

Of course, certain applications do indeed never allocate once they have started up.

Indeed, that is a very common practice, where latency matters. You might have a great deal of memory heap activity in the first second, and then hardly any for months after.

> In most cases, you don't need to care about these things in a GC'ed language, at the expense of larger memory usage and possibly noticable pauses.

GC has other drawbacks as well. The whole tracing workload (which still exists even in low-latency, concurrent GC's) messes up your locality of references and interacts badly with CPU- and OS-level caching and memory management. Plus the mutator part of your program is heavily constrained in how it can layout objects in memory, since the tracing GC must be enabled to select references to other objects unambiguously. It's a non-trivial drain on performance on memory-bandwidth limited workloads, which tends to be most of them these days.

Reference counting does away with most of these issues, and gives you deterministic finalization of all resources not just memory. Alternately, you can selectively use arenas to defer the freeing of some objects, while still being deterministic elsewhere.

zozbot has it right.

Reference counting is a form of GC to use when performance doesn't matter. Performance never matters except where it does. But automated memory management without reference counting is usual practice in a modern non-GC language.

Freeing an array of objects only involves a series of calls to a deallocator when the array elements have pointers in them. That is typically unavoidable in Java, but not in non-GC languages.

Arena allocation, where memory in a subsystem is allocated using a specific allocator object, deallocation there is an in-line no-op, and all of subsystem memory is reclaimed en bloc at a chosen event boundary, is a common alternative where more control is needed. It is still not "manual memory management"; memory for objects is managed invisibly, and still without reference counting.

The usual advertising for GC is that it means you don't need to think about managing memory. The actual experience is that, where it matters at all, you have to think about it a great deal more.

> In most languages, freeing all all the heap-allocated objects of a big array will take longer than in a language with GC.

Only if elements of the array are separately allocated (boxed) or have destructors.

And if the array is heap allocated.

If the array is stack-allocated and the elements are boxed (i.e. heap-allocated) or have destructors, you are still going to need to call the destructors or the free on the elements when the array goes out of scope.

Such a layout, and all it costs, is typically unavoidable in GC languages, and usually easily avoided in non-GC languages, particularly in cases where it would matter.

Thus, it is incorrect to talk about multiple calls to a deallocator. You get just one for the whole array.

My point was: it still takes longer time to de-allocate a large array even if the only thing that needs to be freed is the array itself, when compared to a GC. GC never needs to free anything, it instead overwrites garbage memory with live memory.

I can write (and am in fact developing right now) a music app in a non-GC language, with a real-time audio thread which (unless faced with unusual inputs like hundreds of simultaneous events) performs zero allocations or deallocations after initial startup. It acquires and releases all heap allocations from a wait-free queue between the audio thread and GUI thread.

If I were to port the app to C#, I have no way of telling the C# GC to never pause the audio thread, and to only pause the GUI thread. I'm not sure if it's possible for a tracing GC to coexist with a never-paused thread which owns manually memory-managed types, and for the two threads to exchange manually freed or GC'd objects without FFI boilerplate, serialization, or copying. If it exists (and there's a GUI framework written in the GC portion of the language), let me know!

Admittedly manual memory management does mean a lot more things to worry about (ranging from manual memory management to lock-free wait-free programming) Personally I find deterministic lifetimes and single ownership which can be moved/swapped to be elegant. However, Qt's QObject system combines the practical disadvantages of manual freeing (having to track complex semantics in your head, and check whether each method call transfers ownership or not, with leaks or use-after-free if you get it wrong) with the inelegance of GC (pervasive aliasing and mutability, unclear ownership, a magical runtime-like system).

Sure, and in your specific use case, the tradeoff of dealing with memory seems worth it.

My point was that there are tradeoffs, which you admit. The comment i responded to indicated that there wasn’t one.

I agree that your specific use-case is indeed good reason for manually managed memory, but theoretically, I don’t see anything inherent in specifying GC on a per-thread basis. Like, perhaps one could set a property on a new Thread object and set GC to false - that would probably allow your program to function even in a managed environment.

Not using GC does not, in fact, imply "manually managed memory". Manually managed memory is the only thing possible in C, but C is not the only non-GC language. The most commonly used non-GC languages provide other mechanisms, and usual practice in them is to rely on their non-GC, automated memory management.

Do you mean RAII and rust’s ownership model? It still has very similar tradeoffs to manual memory management, they are just much less error prone.

But the fundamental thing is that you still have to think about memory, memory layout will have to be considered during refactors, etc.

If you are not thinking about memory, or memory layout, you are not programming. Anyway, not programming well.

And "tradeoffs" is a misleading term to apply to a process that does not involve giving up anything in exchange for the benefit of pause-free, fully predictable operation.

There are different levels of “thinking about memory”. The distinction between low and high level languages is not arbitrary. In a CRUD app, you don’t want memory layout to be the center of attention, a major refactor in abstraction should not really require a re-thought on the life time of objects, etc.

And you do absolutely give up something with not using a GC: speed of development, much bigger teams can work on the project at the same time while not stepping as much on each other’s foot, etc. It’s not an accident that perhaps the majority of all software development is in managed languages, where it is not absolutely crucial to control the exact memory layout.

> "you do absolutely give up ... speed of development"

That is the advertising claim, not substantiated.

A more plausible explanation for "the majority ... in managed languages" is: it enables employing lower-skilled, thus more easily obtained, labor. It doesn't so much matter how fast development is. Such practices commonly split a simple job among five or more people (a "team") who, collectively, probably cost more than one more-skilled employee who could do it all in much less time, but who is hard to attract to do it at all, and anyway better used elsewhere.

If it were 50-50 or so, I would be open to this explanation. But it is an overwhelming majority, making the claim not only substantiated, but empirically proved.

And frankly, you are not getting ahead with your arbitrary gatekeeping on “who is a real programmer”.

I used to work in a C/C++ embedded shop, and I work now at a place where only GCed languages are used.

The statement about enabling lower-skilled employers to be hired is pretty much false. Embedded development payed significantly less than most places that uses Java today (at least in Norway), and there's really no difference in the skill of the people I work with.

There's a _significant_ difference in the problem being solved though. The project I'm working at now is, code-wise, bigger by an order of magnitude. As is the problem domain (ticket-selling services, vs firmware of payment terminal in old job).

When doing embedded, I never had to care about racing conditions in threads or across several microservices. The embedded device was single-core anyways, and only talked to one server.

Dealing with memory has been replaced by handling concurrency. Of those I'd argue that the latter is significantly harder than the first. But both add to development time, as it becomes a core point in most design discussions.

In our case, we truly do not need to care about memory. If we run out of it, we either upgrade the instance or add another instance. The monthly hardware cost is lower than the hourly cost of a developer, anyways. Having one less thing to worry about decreases developer time, as there's one less thing to include in our designs, and one less thing to worry about going wrong.

Erlang is a good example of a per “thread” GC (erlangs unit of concurrency is the process). When one GC triggers, it doesn’t pause the rest of the program.

Not sure if you can disable the GC entirely, but you can set the initial heap high enough that it won’t trigger for the lifetime of the process.

... and, generally, if you have to insist on falsehoods to make the case for depending on your GC, you haven't, in the end, made a case at all.

It is a fact that the majority of programming in modern non-GC languages, such as C++ and Rust, involves no manual memory management, no GC, and no reference counting (which is also GC). The memory-management automation provided by the core language and the standard library are equal to almost all challenges, all by themselves.

Thus, users of modern non-GC languages do not experience the pauses seen in GC languages, or the unavoidable pointer chasing, or the cache poisoning, or the "reachable leaks" that are GC languages' dirty secret. They are not, in fact, obliged to "trade off" anything at all for being free of those failings.

That is not to say that all design goals are easy to achieve, but overcoming a GC's failings is is not among the activities needed achieve them.

"Manual memory management" means more than just explicit free(3). If you have to create a destructor, and a class for it to sit in, just to make sure some bit of memory gets freed, that's manual memory management. The falsehoods here are yours.

Self-deception in the terminally confused is always sad to see.

It is extremely rare, nowadays, to "create a destructor" to manage memory. The destructors generated automatically from templates in the Standard Library suffice.

Creating classes is just programming.

So, no, that is not manual memory management.

You're confusing implicit with automatic. That's the best illustration of self-deception and "terminally confused" in this thread. If programmers have to think about when something truly goes out of the last scope it'll be in (sometimes not obvious when it's put into and taken out of containers and passed between callbacks) then it's not automatic. If they have to think about whether they can use/pass a reference and be sure it won't outlive the object itself, it's not automatic. These have been issues in every C++ codebase I've seen, so even if I were personally able to create a codebase without those issues it wouldn't change anything. It also doesn't make a difference if you're so accustomed to a certain set of ownership rules that the time you spend thinking about them is very small. It's still not zero, and other programmers who are less inured to the mess still exist.

It's OK that these issues exist - as you say it's just programming - but don't delude yourself into thinking that's automatic memory management. It's implicit, but it's not automatic and that means it's (at least partly) manual. When you say you don't use manual memory management but then say you let standard-library templates handle it, that's a contradiction and tantamount to a lie.

Repeat that enough, you may come to believe it.

> Thus, users of modern non-GC languages do not experience the pauses seen in GC languages

/doubt

Once you're facing, for example, the deallocation of an entire tree that you own and that's going out of scope, your pause is unbounded. Compared to that, a GC'd language could delay the deallocation, or even in case of stuff like C4 have a dedicated thread deal with it.

Sure, you can always write code in a way that avoids this, but non-consing is also an option for many of the GC'd languages (at least the more advanced ones, like Common Lisp).

GC languages, boxing everything, oblige constructing elaborate linked structures. Boxing is exceptional in non-GC languages, and trivially avoided where latency would matter.

So, doubt all you like, unavoidable random millisecond pauses are not a problem in non-GC languages, no matter how much you wish otherwise.

Everyone else: You can only have systems programming in languages with manual memory management.

Java(JikesRVM)/Oberon/Go/D/Nim: Let rewrite the whole toolchain in X, including the memory manager itself.

Everyone else? Most languages have GC and Go has had a low latency GC for almost a decade IIRC.

It is most definitely not low latency for heap sizes as big as what the OpenJDK can routinely handle (up to a terabyte).

Go’s GC has it easier due to the language having native support for stack-based allocation which can sometimes be used in otherwise garbage-heavy places.

> It is most definitely not low latency for heap sizes as big as what the OpenJDK can routinely handle (up to a terabyte).

A claim without evidence isn't worth very much.

> Go’s GC has it easier due to the language having native support for stack-based allocation which can sometimes be used in otherwise garbage-heavy places.

Java (i.e., "modern JVMs") also has "native" stack-based allocation (indeed, it has an escape analyzer whose sole purpose is putting things on the stack); however, it lacks value types.

Yes it has escape analysis, but even if it were perfect and correctly recognize every possible pattern, it is still nothing compared to having native support for it in the language and mandate it’s usage throughout the codebase.

One can easily add a single line and “break” the escape analysis, while it is compile time checked with value types.

Go doesn’t have semantics for stack allocation, it also has escape analysis. It’s pretty easy to intuit about the escape analysis, but you have to profile to be sure.

The issue with Go's GC on big heaps is not getting high latency - it is always low latency - but rather the murderous throughput and memory overhead impact.

Java's been making the garbage collector better for decades at this point and is still in the second performance tier compared to those other languages.

Of course, if one does GC tuning like a pro, uses a sufficiently smart GC and maybe a custom VM (and unicorns) one may reach the mythical native speeds (typically at the cost of using a lot more memory).

Tell me any runtime with a better performance than the JVM..

It's more like this:

C diehards: Pauses are bad, we need manual memory management.

Everyone else: Does not have manual memory management.

> Does not have manual memory management

That is a misunderstanding.

If you would like to avoid pauses in c, you must be careful about how you allocate, what you put where, &c; you cannot simply use malloc and free. Similarly, if you would like to avoid pauses in a gc language, you must be careful about how you allocate, what you put where, &c; you cannot simply allocate.

These techniques are actionable in any language. Java 'has' automatic memory management, but it also 'has' manual memory management (if you implement it).

Not without relying on implementation details.

? no

In almost any language with automatic memory management, I can portably make my own freelists, regions, temporary (reused) buffers, ...

Great article. The JVM is one of the most astonishing pieces of software ever created.

I was looking at gRPC benchmarks the other day (https://github.com/LesnyRumcajs/grpc_bench/wiki/2021-08-30-b...) and 6 of the top 7 performing gRPC implementations on 3 core CPUS were on the JVM.

Completely agree.

Say what you will about Java (the language), the JVM is a seriously good piece of kit.

Java is a fantastic language. It's simple and in the right environment can be elegant. It's incredibly easy to ramp up new engineers on it and the tooling thats available make discovery and navigation a breeze.

It may lack a lot of the fancier features of some newer languages but its still a very effective language.

> It may lack a lot of the fancier features

Coming from JavaScript, TypeScript, Python and Go back to Java I can tell you that’s actually a good thing.

Agree 100%. Let me build stuff using simple easy to understand tools.

Cleverness in code starts working against you as a code base and organizations scale.

Doesn’t Java tend to have a lot of cleverness/magic? Pretty crazy runtime codegen, reflection, etc? Spring and other DI frameworks seem overly clever to me, as do many ORMs. I guess my standard for a simple, minimal-magic language is Go.

Yes but idiomatic java doesn't require you to use those thing.

What does “cleverness” refer to then? Couldn’t I equally argue that idiomatic Python (or any other language) doesn’t require I use cleverness? And if so, what sense is there in praising Java for its lack of cleverness?

The aspects mentioned (Spring, DI containers, ...) are tool on top, though. If we just look at the language, I'd argue Java is boring - which is a good thing for certain types of applications. In languages like JavaScript, you have a number of ways to do things with all kinds of different syntaxes. In Java, there are fewer, but well understood options, that sometimes require more and sometimes require less actual code to achieve the same thing. This is highly subjective, if course.

Ok, I understand your meaning, and I agree that gratuitous syntax sugar is a bummer; however, I've never found it to be particularly problematic. On the other hand, I have found a culture of magic/cleverness and a magical ecosystem to be problematic--this is one of my bigger grievances with Python after a decade and a half of experience.

I am leading a transition from TypeScript to Java + Spring Boot and comparing those side-by-side, Java is significantly easier to follow. In TypeScript, we have a (from my point of view) too complex type system and features like destructuring and spreading, that sound smart but actually make it difficult to understand if you are not the author of the piece in question.

What do you find magical in terms of Python in particular?

I can't speak to TS. I'm not sure what "spreading" even refers to, although I've never had a problem with destructuring even though my daily driver languages don't support it.

> What do you find magical in terms of Python in particular?

Lots of libraries are magical. SQLAlchemy, most of the data science libraries, even the standard library returns different types based on the values of inputs (e.g., open() returns a binary file or a text file depending on the string you pass into it). Additionally, people often go crazy with various dynamic typing features (rather than refactoring into a common interface, people will do a bunch of `if isinstance(...)`es all over the place. Similarly people tend to go crazy with metaclasses and so on. I could go on and on.

> I'm not sure what "spreading" even refers to

We have code like (stupid example)

    something.map( {a, b, ...rest} => ({ ...rest, ...a, ...[b, c, d]}))

I know what it does and how it works, but I argue that's hard to understand if you haven't written it and know the context.

My Python experience is limited to only a handful of projects, but I looked into meta classes at the beginning of learning Python and that was some crazy stuff.

I honestly don't see how destructuring and spreading is harder to understand than things like DI and annotations in Spring. I do agree about the TypeScript type system being too complex, I wish simpler types like Rescript won outside of the "typing legacy JS" use case.

> sometimes require less actual code to achieve the same thing

What are some things thing you can do in Java with less code than in Javascript?

Is DI really being clever? I'd rather work with people who understand DI than a big ball of procedural mud.

"DI" and "DI frameworks" are different things. For example, in Go we inject our dependencies (we assemble our object graph in the main function rather than in thousands of constructors), but we're not using XML/etc and a magical translation layer.

How do you maintain a single main function for large projects? I personally don't think @Singleton and @Inject are that magical.

Pull the graph construction logic out into helper functions if necessary. Pretty straightforward.

That DI layer (speaking for Spring) is pretty handy for large projects, though, because I can switch out implementations based on configurations centrally, I can use factories without change of injection patterns, ...

I've heard that, but I don't understand it. Is the idea "I don't have to deal with static typing"? In vanilla Go code I can change out implementations super easily as well (via interfaces, which are less tedious than Java interfaces due to structural subtyping). We can also use factories, but this is precisely the kind of stuff I prefer to avoid except in certain very rare cases ("but with Spring, no one is forcing you to use factories!"--maybe not technically, but if it's idiomatic or otherwise culturally accepted then you can guarantee that your coworkers will reach for them).

The code base at work has 100s of thousands of lines in several services based on Spring Boot. There's not a single factory in our code.

We also do not use XML to configure DI. Just add a annotation to a class and you can inject it automatically in any constructor without any additional config/annotations.

If this is too much "magic" for someone's taste, I will not argue even if I have a different view. But please do not make absolute statements based on a bad experience in some companies.

Modern Java and frameworks have evolved a lot in recent years.

My take on this is that just injecting stuff in a constructor is an antipattern. Why not pass the data through methods instead of accessing it through a singleton in a constructor?

I understand the need for DI, sometimes the order of instantiation is necessary but for everything else I get lost in the DI constructor injection stuff.

There’s typically a separation between “services” and “data” in this style of app. Data is typically a request or some result obtained from another service such as the database. A service can have a lifetime distinct from the data, such as a database service that uses a connection pool, which is scoped to the lifetime of the application.

In a functional language you could use a reader or environment monad to abstract the dependencies, but you don’t have that ability in a language like Java (and it’s not worth torturing the language to do it because it’s not idiomatic). So DI in Java ends up constructor-injecting service dependencies and using the method parameters to declare data dependencies.

Edit: another benefit of DI is that it allows for multiple lifetime scopes beyond application and method call. Spring has request and transaction scopes, for instance. Managing all that in Java without DI is a nightmare.

What I am referring to with factories in particular is that I can have a bean injected in Spring automagically using an annotation, have an interface and an implementation behind it.

Now, I can go ahead and make the concrete implementation dependent on configuration, e.g. by providing multiple implementations annotating them with a condition that is evaluated using the application configuration. I can also switch to a factory method that creates and sets up the concrete implementation for the interface, I have not to change any place in the application that is using that piece, though.

The same should be possible in Go with reflect and an IoC container, but I am not sure if there's a solid implementation for that out there.

I'm a bit ignorant, but it sounds like you're re-inventing "functions" in any programming language. Maybe if you could give a simple, concrete example it would help.

Oh interesting! Would you have an example you could link, I’m curious what this looks like

I will choose one “magic” annotation over repeating the same buggy code thousands of times. Not having abstraction is just as bad as too much.

And yet in Go you don't have NPE that plagues Java code.

They haven't really been a plague for a while, not like they used to be. A lot of forces are behind it, my top three reasons would be 1) because of the Optional type and Optional mapping makes them easy to avoid in chained calls and is less ugly than a nest of null checks or the equivalent err != nil mess in Go, 2) because of better development practices becoming more common like SOLID and especially https://en.wikipedia.org/wiki/Law_of_Demeter and 3) because of functional programming styles becoming more widely adopted, with Clojure/Scala/Kotlin showing what's possible on the same JVM.

> less ugly than a nest of null checks or the equivalent err != nil mess in Go

"null checks are ugly and error handling is messy" are your opinions, not objective facts. I actually prefer Rust-like enums, but short of that, I think Go's error and nil handling seems quite a lot nicer in practice than Java's `Optional` facilities. Note that these are my opinions, and I'm not posing them as objective facts.

Using the word "ugly" marks the opinion as inherently subjective.

In whichever case, a subjective argument is very weak.

Ok.

This is transparently a false dichotomy. No Go programmers are "repeating buggy code thousands of times". Go has a bit more boilerplate than Java until generics land, but I maintain that it's a non-problem 99% of the time (mostly it's just the simplest of conditionals and for loops). And anyway, a generic type system is a much nicer solution than an annotation or other magic.

How about eg. transaction management, resource handling, db connection pooling? Could Go implement “generic” solutions for that? The declarative/meta-programming aspects can make common use cases extremely easy to implement (and web frameworks are full of already implemented 100x times functionalities)

Yeah, Go has generic db connection pooling in the standard library. I'm not sure what transaction management or resource handling you're referring to specifically, but for resource management we usually just do `defer foo.Close()` or similar, although I prefer to make `func withFile(fileName string, callback func(*os.File) error) error` helpers that open a file, pass it to the callback, and then close it. To your point, we have to write one of these helpers per resource type, but it's such a small, simple amount of boilerplate that I don't even bother putting that into a package to reuse. Not a problem in practice.

My experience with Go is that it's quite a bit more stripped back of a language than Java. Could you explain what you mean by this?

I love Java but happen to like TypeScript a lot, at least the way we write it.

The rest of them I've been enthusiastic about for a few weeks until reality catches up with me and I realize what I am missing out on.

I find it interesting that you'd lump Go with those languages. Java and Go are my favorite languages to code in for muchof the same reasons.

Yeah, I have my complaints about Java the language (and mostly use Scala myself) but the tooling for Java development is really compelling. Everything just works (IDE, debuggers, etc) and idiomatic Java code tends to be very high performance without a ton of special effort required for performance tuning. And if you do need to do performance tuning, the tooling for metrics and performance tuning is also amazing. One of the most underrated aspects of building on the JVM is that the runtime itself provides a lot of visibility into runtime performance "out of the box". If I do have to debug performance issues, I can just fire up JFR and get a ton of useful metrics about performance bottlenecks with no changes to the actual source code. I can even run JFR on my production workloads and get great visibility into the were the "real world" performance bottlenecks are happening.

All I want is named parameters so that I don't need to have N variations of the same method with different signatures, also particularly helpful for understanding boolean parameters.

Named parameters are great but RequestTypes are better. It lets you roll validation into the request type itself which helps clean up methods a ton

If Java had anonymous (structural) records, then named arguments could just be a function with a parameter that has a record structural type. It could even be given syntactic sugar, something like:

foo{ bar=true, baz=true }

My main complaint is the treatment of generics and lack of value types, which are actually two sides of the same coin.

Another thing I'd like to see is opt-in null safety. The Optional type is a very good mechanism, but when every reference is potentially null, its ability to enhance language safety is limited.

Other than that, it has everything I want. And of course, the ecosystem, tooling, etc. is simply the best in class.

My personal preference is for ML-style languages, so Java will never be my true love. But as far as languages go, you can do much worse.

I think there's work being done on value types in Java. Although this was the case 6+ years ago. I'm surprised they still didn't land into Java.

JEP 169 is for value objects and JEP 218 is for generics over primitives. As the grandparent comment said they go hand-in-hand. My sense is they're more concerned about getting it right then getting it shipped, as both have been in the works for years.

Yeah, it doesn't look like Project Valhalla will land before the next LTS. The original separation of primitive types and reference types was necessary when Java was first released, but it's a deal with the devil. It made everything else much harder than it could have been otherwise.

Even then we won't have fully reified generics, but some of the stuff they're doing with type projections is insane. I'm pretty sure it's going to be a game changer.

Use Kotlin to get null safety (and more), plus the ecosystem and tooling.

I haven't found switching to Kotlin to be worthwhile enough to justify moving away from Java, but YMMV.

If we're considering other JVM languages, Scala and Clojure are also on the table.

What I especially like about Java that it moves slowly, but when it does, it is a very thought out feature. For example, here’s one on the future of a seemingly trivial feature, string interpolation: https://github.com/openjdk/amber-docs/blob/master/site/desig...

Java is anything but simple, all the major framework are heavy duty things behind layers of magic abstraction and un-necessary complexity. Did you worked with Spring, Hibernate / random ORM and all the "enterprise" frameworks?

So, the libraries that some developers have to use are complex. Not the language itself.

Additionally, Spring Boot exists and is lightweight and super easy to understand, jooQ is an ORM/query builder that makes Hibernate look like a toy (and it super simple).

Just because Java gets tainted by overengineering doesn't mean the language itself isn't simple.

But in Java you have to use frameworks since the std lib does not offer anything for web / DB. Does not matter if the language is simple but you're forced to used a framework.

Servlet API and JDBC are as standardized as you can get. If you want more than that, in any language, you're reaching for a framework.

Java has JDBC and HttpServer for a rudimentary server.

There are plenty of alternatives to ORMs. We use JDBI (https://jdbi.org) which is dead simple and a happy medium between pure JDBC and a full blown ORM.

Even using the built-in JDBC support is an option with modern Java, PreparedStatement and ResultSet implement AutoClosable, which means you can write code like this:

    try (
        final var rs = conn.prepareStatement("""
            <SQL here>
        """).executeQuery();
    ) {
        while (rs.next()) {
            // do your thing
        }
    } catch (SQLException e) {
       // your code here
    }

Which is just the standard library and no more verbose than most other frameworks.

As for web frameworks, there's a million of them. Pick a non-magical one if you don't like magic.

Yeah, Java can be bad, but thats more the old school enterprise frameworks. A lot of newer things in Java are much simpler and easier to use. You can be very successful building out your own framework using things like Guice, Guava, and Dropwizard. It lets you pull in what you want and leave out the things you don't need.

Helidon, Quarkus, Javalin etc. There's plenty of options if you want to build your framework yourself.

I had to learn Spring and Hibernate.

Spring had good documentation that you can read on an ereader. It's also very mature, so there's lots of help on Stackoverflow.

Hibernate was a bit more difficult, but Vlad Mihalcea has improved the documentation side a lot.

Hibernate also allows you to drop down to native queries, which coupled with Spring Data's "projections" are useful.

Do you think that the alternative of not caring about transactions, using ad hoc ORMs, home-made DI-like contraptions would be beneficial for a typical CRUD app? And I know it is a logically fallacious argument on my side, but I think we tend to understate the huge amount of repetitive task that you will definitely need, but think it is not yet important.

Also, I think it is overblown how magical those annotations/code generation are. It is simply class path scanning and code generation under the hood, so that a proxy object is used instead of the interface. I would take this path any number of times instead of what abomination n developer hacked up over the years with no proper abstraction and documentation.

Yeah, to me that's the most amazing thing about it. That the JVM has become an actual platform on which many languages are able run and get the amazing performance of the runtime. In the gRPC benchmarks the top spot is actually akka-grpc which is mostly written in Scala!

And .NET as well, given the value types and low level improvemetns done since 7.0.

I was very much on the Java/JVM side of the Java v .NET wars of the early 2000s (mostly because of an instinctive distrust of MS and the imo better OSS ecosystem around the JVM) but still always though that C# was the better-designed language.

Same here, but as things turned out, I keep jumping between both ecosystems.

There are good things and bad ones across both of them.

Both also provide enough rant material. :)

C# is a nice language but the JVM is the better VM. And you have other languages on the JVM like Kotlin and Scala. Plus Java is getting nicer as time goes on.

F# is nothing to throw away as well, and the CLR has native support for C++.

You know that thing, which WebAssembly is trying to clone.

I really like .NET, but I feel like their GC implementation is really lacking in performance. Their codegen is great, that's why their focus was on trying to reduce the amount of allocations in the past few releases.

I've recently written a pretty compute-heavy app in .NET 5 that generated some garbage (mostly nursery stuff), and I tried to run it on a 16-core CPU.

The results were disappointing, even with the Server GC, mainly due to the fact that the GC still does a ton of stop-the-world pauses, and the amount of garbage basically scaled linearly with the number of threads.

My benchmarking results indicated that adding more than 8 threads were basically pointless, since the increased garbage meant that more time was spent in STW GC mode.

I 'fixed' the perf issue, by eliminating all GC allocations, which alone resulted in a 2x speedup per thread, and allowed basically perfect scaling across cores.

In the day and age where you can rent 64 core CPU VMs for $2/hour, I wish they would spend more engineering effort on fixing this.

I have been in both ecosystems since they exist, and I think .NET suffers a bit from .NET vs C++ wars at Microsoft.

You can see this in practice when following up which tech gets out and how it evolves across its lifetime.

I bet many teams before the whole C# 7 effort, would just write a couple of COM stuff in ATL, call them, from .NET and be done with it.

This is also why WinRT was designed the way it was, given that Windows Dev was driving it.

There is also that background or concurrent GC option in .NET, if you have a lot of threads.

You shouldn't read too much from the benchmarks.

The gRPC libraries are primarily developed by Google. Google pours most of its development resources on its core languages: Java, Javascript, Go, C++, and Python. Google then allocates a developer to port the implementation to other languages and platforms. C++, unlike Java, is not typically used for cloud development, so I don't think a lot of resources were allocated for the language.

You can see how the difference in implementation effort can impact performance by comparing dotnet_grpc (Microsoft's fork) against csharp_grpc (Google's original implementation). There's more than a sevenfold improvement in req/s in Microsoft's implementation in the 1 CPU server case (35070 vs 5337), outperforming nearly all the Java benchmarks.

Also, many of those top performing JVM implementations are the same code running under a different garbage collector. .NET has two GCs each with a parallel option, but we only see one benchmark using likely the slower GC (Workstation GC instead of ServerGC).

Yeah, they are microbenchmarks so should definitely be taken with a grain of salt but the top performing one (on 3 CPUs) was akka-grpc which is definitely not built by Google and uses akka's native HTTP/2 stack. It's not even written in Java for the most part but mostly Scala.

> It's not even written in Java for the most part but mostly Scala.

Scala still runs on the JVM.

> C++, unlike Java, is not typically used for cloud development, so I don't think a lot of resources were allocated for the language.

I though most of Google's infrastructure (such as Borg or things like that) was in C++.

> C++, unlike Java, is not typically used for cloud development, so I don't think a lot of resources were allocated for the language.

I think actually its the opposite, and the C++ implementation had seen most activity. Some other implementations (e.g. the node.js one, the initial version of .NET support, and also the rust_grpcio flavor) had been built on top of Googles C++ gRPC core library. I am not sure however if some of those now have moved to more native implementations recently, since I haven't followed the development closely.

There's also a lot of questionable decisions in the gRPC benchmarks. Thread polling plays a key role, and in the Java examples, the framework allows for a wide range of types.

Disclaimer: contributor to the gRPC Java benchs.

We wrote some experimental network drivers in high-level languages a few years ago and Java performed better than expected. Yeah, that's a very special and somewhat odd use case, but it was fun :)

We've got some graphs [0] comparing throughput and long-tail latency for the various Java GCs available in OpenJDK 12. Shenandoah's worst case pause time was ~45µs which is the same as disabling the GC (Epsilon GC) which is pretty impressive. Overall performance did suffer under Shenandoah a little bit back then, though. However, I've heard that this improved recently.

[0] https://github.com/ixy-languages/ixy-languages/blob/master/J...

Java can be amazingly performant. One particular example is a video compressor handling many concurrent streams, recompressing them on the fly and shipping them out again saturating a gigabit link.

One neat feature of Shenandoah is once the application shuts down, it prints into the GC log a table depicting how much time each phase of the GC cycle took. This data can be crucial to optimize the GC pause time further:

- Removal of weak references (finalizer, phantom, soft) can help reduce the GC pause time further as these warrant stop the world phase

- Shenandoah likes ReentrantLocks better than synchronised classes the latter bloats up its monitors and increases the size of the rootset

- The reflection problem may initial Shenandoah to run GC cycles even when there is no memory pressure and thus may reduce performance over time. It is better to limit it using reflection inflation flag

- On Amazon ECS, provide the CPU share count explicitly; Shenandoah likes more threads

> One neat feature of Shenandoah is once the application shuts down, it prints into the GC log a table depicting how much time each phase of the GC cycle took. This data can be crucial to optimize the GC pause time further

Other GCs will do the same for you (G1, ZGC) which you can then put into a tool like gceasy to pretty graphs.

> - Removal of weak references (finalizer, phantom, soft) can help reduce the GC pause time further as these warrant stop the world phase

I'm in way disputing this, but why would a weak reference be different than any other reference, beyond the added flexibility it gives the GC (you may or may not GC this reference if you please)?

I admit I don't give GC much thought beyond every coupe of years when I switch jobs :)

I haven't looked at Shenandoah and my GC experience is a bit stale, but to determine if something is weakly referenced, you need to determine whether or not something is universally weakly referenced. A strong reference anywhere means it isn't weakly referenced at all, and as I recall it's been thought (for now) that while this may not require stopping the world to sweep the full set of references, it's a safer and more correct option.

Consider if during collection an object that was only reachable becomes strongly reachable as a result of promotion. This needs to be reconciled at some point, and there's also atomicity expectations. If there's 2 weak references for the same object, you can't have one return non-null after the other already returned null (part of the API requirements: https://docs.oracle.com/javase/8/docs/api/java/lang/ref/Weak... )

The easiest way to handle this conflict is to just pause the world when doing the processing of weakrefs after the concurrent GC finishes.

> - On Amazon ECS, provide the CPU share count explicitly; Shenandoah likes more threads

Can you explain this further? I'm missing some connection here between ECS configuration and how the JVM sees the container.

Not OP, but many systems will scale the amount of threads based on the core count in your system. So on a 32-core system you might have 32 GC threads. On ECS (and many other container orchestrators) when you set some CPU allocation like perhaps 0.2 CPU, your process will still "see" all the CPUs present in the system but the process will just not be scheduled if the allocation for this time period has been used up. This could cause Shenandoah to over-allocate GC threads and eat into your CPU allocation.

Based on the number of CPU cores on the machine, Shenandoah sets up its threads. JVM will see only one CPU core if the number of CPU shares for the container is not set explicitly[0]. In that case, Shenandoah will use only single parallel thread to do the collections, and the entire application may crash.

[0]https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...

Ah, right. I wish ECS had an option to expose all of an EC2 instance's underlying cores to a container with no constraints. In that case, the scheduler could use the memoryReservation setting to figure out which containers can fit on which instances.

Notice that with Shenandoah GC, weak references are handled concurrently since JDK 16, native roots like locks, classes, etc are handled concurrently since JDK 14. Therfore the first 3 problems that you mentioned are solved. Follow the links the OP article to find explanations for each of these improvements.

BTW, you can specify the number of GC threads that you like to use -XX:ConcGCThreads=X -XX:ParallelGCThreads=Y (X for concurrent GC, Y for STW pauses - which should no longer be very relevant).

Cheers!

Awesome work! In our shop, we managed to solve significant application bottlenecks simply by switching to Shenandoah GC. The application in question is very complex, so any kind of rewrite in another language is out of the question.

This is the power of JVM we do not get from other languages/VMs. Kudos to JVM (previous and current) architects, developers, and everyone involved with it.

> This is the power of JVM we do not get from other languages/VMs.

sub-millisecond gc pause has been available to Golang users for ages.

I'm guessing the parent is referring to the ability to transparently change runtime parameters and benefit from technical improvements without having to switch languages or even needing recompilation. That's not something golang can do given its statically linked afaik

Exactly!

Only by not allocating garbage in the first place by using “value types”. No way its GC would even get close to the JVM’s (and certain problems do require garbage collection, where the JVM will have better throughput)

> The application in question is very complex, so any kind of rewrite in another language is out of the question.

I think I know what you mean. If a couple of guys quit, the company is in major trouble? :-)

Forgot this: /s

Not the OP but you can have a large, complex system that's widely understood and supported by a decent sized team, where a rewrite is still not a commercially wise choice

True!

This is probably the norm.

Yeah, rewriting everything is almost never a good choice from a business value perspective. Usually, the most sensible thing to do, is to break the system up in components, and rewrite them slowly while keeping interop with the old system. Downtime is no money time. And new features sometimes give more value, even if the programmer is grumpy.

Shenandoah is keeping pace nicely with ZGC, especially important to make this into the jdk 17 release since that will be what most users target for the next few years.

Given the similar feature sets, does anyone with experience running both ZGC and shenandoah have any differentiators?

The main difference from user perspective are 1. ZGC doesn't support compressed references (-XX:+UseCompressedOops) which could affect workloads <32GB heap and 2. Shenandoah GC is not available in Oracle builds

I’m slightly confused as to what the relation is between OpenJDK and Oracle JDK. Are they based on the same codebase? I was under the (apparently mistaken) impression that OpenJDK and the ”official” (Oracle) JDK were totally different implementations, but seeing how Oracle’s JDK just hit 17, and apparently OpenJDK did at the same time I guess they are derived from the same source. Could anyone enlighten me?

They are built from the same code base. Any differences are just down to the license, packaging, etc.

Doesn’t the Oracle one include some relatively obscure but non-free libraries?

It certainly has done, but currently the differences are pretty tiny:

https://www.oracle.com/java/technologies/javase/17all-relnot...

It's really just 'usage logging', which is for Oracle's management tools. Their stated aim is to eliminate the functional differences, so presumably they are awaiting a non-proprietary replacement for that feature in JDK.

Thanks! So, given the recent license changes to Oracle’s distribution, why would anyone use it? ”Support”?

It would have been valuable to see the percentile distribution of latency, 99th, 99.999... max etc

I am a layman with regards to JVM internals.

Are there any differences in the targeted workloads or capabilities between ZGC and Shenandoah GC?

Both seem to be marketed for large heaps and low pause times.

I saw this start during my internship at Red Hat in 2012 :) cool to see it kicking around

So given that the JDK is open source and dotnet is open source, would it be possible to port Shenandoah?

It is my understanding that D's garbage collector isn't that good, could they pick up an OpenJDK collector?

I am unfortunately not an OpenJDK dev (but would really like to become one down the road), but afaik a GC is way too dependent on the internals of the runtime. The JVM has a very different object structure.

Also, I think D allows for pointers which is a no-go with compacting GCs (as those move objects around). So all in all, the algorithm used itself can be ported, but it has many runtime specific logic, which is just as important for great performance.

The quality of the JVM GC is impressive to me as a .NET developer. I would love to have reliable sub-ms pauses in my apps. I am working on one right now where even small pauses >1ms are potentially detectable by the user.

As a Java developer I'd love to have .NET's value types and runtime generics. Something something grass is always greener on the side :)

Hi. Can the Shenandoah's concurrent stack scanning in RedHat builds of the JDK 17 be considered production ready?

If it's not behind XX:+UnlockExperimentalVMOptions (or a preview/incubator module), it's considered stable and prod-ready, yes. Will there be fixes and improvements? Yes. Should you be afraid to use it for business? No, but obviously test it first.

You can buy support from Redhat, which would presumably include support for their JDK build with Shenandoah.

Not sure what that means in practice, but if someone is putting money on the line, it must be worth _something_.

Reply on news.ycombinator.com