Hacker News Reader

Comment by azhenley | Hacker News Reader

The most interesting new feature I think is the Shenandoah GC. The summary from [1]:

"Add a new garbage collection (GC) algorithm named Shenandoah which reduces GC pause times by doing evacuation work concurrently with the running Java threads. Pause times with Shenandoah are independent of heap size, meaning you will have the same consistent pause times whether your heap is 200 MB or 200 GB."

The original algorithm was published in 2016 [2]. It consists of 4 phases: initial marking, concurrent marking, final marking, and concurrent compaction.

[1] http://openjdk.java.net/jeps/189

[2] https://dl.acm.org/citation.cfm?id=2972210

View on HN

GC pauses have been one of the major barriers to using garbage collected (read: higher-level) languages for game development. This could open up the JVM for games, which could have some exciting implications.

There are a lot of applications where predictable performance is just as important, if not moreso, as good performance.

Their lower end target is 10 ms. In a game, 16 ms is your entire time budget.

In VR your budget is 11 ms and you can't miss it or the entire frame is thrown away and you introduce a frame of latency to compensate. So for VR these kind of effects of GC are even worse.

Also, while most games are fine with losing a frame or two every few minutes, in VR dropping frames will cause motion sickness really fast. FPS is, from what I've observed, the most important part of a VR game. If you're dropping frames or stuttering, the experience will suffer greatly

When I was developing a VR game, my team noticed that some ultra-sensitive people started to get sick when we would drop around five frames (the vive displays 90, so this is only a ~5% drop). There was more to it (dropping a few frames every few minutes was usually fine, but consistently dropping frames every second or two is an issue).

10 ms isn't all that much. When games load in assets they'll do various things like malloc huge chunks and you lose a few frames.

If GC only causes occasional (maybe every 2 minutes) loss of a frame or two it should be no problem. If you watch benchmark FPS traces that count every frame delay you'll see that occasional stutters happen in basically every game.

Loss of a single frame just isn't noticeable

Frame drops that happen more than once every several seconds are definitely noticeable (and, as a sibling comment pointed out, especially so in VR). Plus, throwing away more than half your frame budget just because of GC seems unreasonable. And this isn't even considering monitors with high refresh rates, which are slowly getting more and more popular.

> When games load in assets they'll do various things like malloc huge chunks and you lose a few frames.

Streaming/Loading assets in the main render thread is a very 2005 thing to do.

Yeah, the whole reason you have pools and arena allocators is so that you don't do small/large malloc mid-frame. Perf 101 is don't malloc during regular gameplay.

Quake, back in the late 90s, would preload assets for all entities in a level at level load time explicitly to avoid stutter when the entity finally shows up and needs to make a noise or whatever.

And some games much later didn't. I recall needing to fire off a shot on each new gun I picked up in Far Cry to be sure I wouldn't get a stutter later at a more critical moment.

ZGC, included in JDK 11, has a 1ms average pause time and 4ms max on some demanding benchmarks, and they're now targeting 1ms max pauses.

https://www.opsian.com/blog/javas-new-zgc-is-very-exciting/

Wow great link. I live pretty deep in Javaland and had no idea it was that good!

If it lives up to claims surely it’s a paid for add on?

Originally it was intended as a commercial feature, but since JDK 11 Oracle open sourced all previously-commercial features in the JDK, and now it's part of OpenJDK. It's already available in JDK 11 and 12.

I never got to the level where I needed anything more than the stock default GC. But this looks compelling. Does Kotlin and other JVM based languages benefit from it, too?

Sure. All languages that run on the Java platform[1] can benefit from it, and from most other Java platform enhancements.

[1]: I prefer saying Java platform rather than JVM, because a program written in Kotlin shares over 90% of its infrastructure code with a program written in Java (not including the OS), and the JVM constitutes only about 20% of that.

It works really great with heaps >100GB and is included in the openjdk since 11. Note, that only linux is supported, I think

As my world is currently only serverside this isn’t a limitation but a feature ;)

16ms is only if you're just targeting 60fps. We now have 144hz monitors, leaving less than half of that if you want to take full advantage of the hardware.

If you have some control over when a GC triggers you could sneak one in during hitstun.

A few of us built a webgl game backend in Java. It was fun but optimizing to make everything always work within 16ms was hard. In our case it was especially complicated because we shared state over websockets back to the clients.

Once upon a time the barrier was the existance of VMT tables and method dispatch.

GC use can be optimized in high level languages with support for value types (which are still missing in Java, though) and it is not like every game needs to be the next Fortnight.

Only tangentially related to your comment, Fortnite is based on UE4 so it does use garbage collection. And it has caused framerate/hitching issues as recently as a year ago according to an Epic developer.

https://www.reddit.com/r/FortNiteBR/comments/7gu8aq/hitching...

My point being that everyone evaluates programming languages for writing games as if they would be doing a AAA hit, when they will actually be doing a minesweeper clone.

Thanks for pointing it out though.

I didn't realize UE4 adds a garbage collector on to C++. Interesting.

Yes. UE4 also extends C++ with its own reflection system as well which is used all over the place, and critical to the garbage collector.

https://wiki.unrealengine.com/Garbage_Collection_Overview

Honestly, if you need real performance when writing a game, there's no reason not use the language that's most popular for your engine of choice. Mostly that's gong to be C++ with python scripting.

The GC doesn't just affect high-performance games. The problem is it doesn't just make things broadly slower; it introduces what can easily become a second-long pause a couple times a minute. Even if your game runs buttery smooth the rest of the time, that's pretty much unacceptable. There are measures to minimize it - object pooling, for example - but they remove most of the benefits of having a GC in the first place.

And Lua scripting

LuaJIT, presumably?

This has already happened in a way. Unity uses a ton of C# for scripting. While C# is somewhat nicer to work with, this low pause GC makes java a compelling option for moving much more logic into the VM language

Yeah, I do wonder how Unity deals with the GC. Maybe there's an algorithm like this one, maybe the engine's core systems (physics and graphics) are written in native code.

C# as a language does tend to be nicer than Java, but its VM doesn't have as rich a family of languages.

My limited understanding is that high end Unity devs code their stuff not to make garbage. Unity has a very nice system for profiling that sort of thing.

Also, CLR has F#, which I like much better than Scala (and, no type erasure in CLR), Clojure which is ... just like Clojure on JVM, and C#, which as a Java dev since 1.1 I have come to prefer as a language even as I remain JVM ecosystem preferring on the server side. JVM has ABCL, a neat Common Lisp, and now Graal/Truffle which is super interesting. If not for Clojure though (and maybe Kotlin) I would probably give up on the JVM. Oracle does not help. Just my opinion.

Totally agree. C# and kin are far better designed. It's the lively Java ecosystem that keeps me around as well

Less “lively Java ecosystem” - more “slow-moving Enterprise Java environments”.

Is Clojure on CLR actively maintained?

Yes, it's one of the officially supported platforms (albeit the least used one).

I did just remember that C# has value-type data structures (which last I checked, Java didn't) which probably helps a lot. Most objects can probably be kept on the stack, and objects on the stack don't need to be garbage collected.

Keep in mind that C# has support for non-primitive value types (structs) which means you can avoid garbage collection entirely if you're careful (not all that difficult in my experience). Additionally, Unity has just introduced a specialized low-latency garbage collector in their beta build. Overall, from where I'm standing, Java doesn't look very appealing for game development.

Lack of value types sucks so yes I agree. On the other hand this new GC might be fast enough that it just doesn't matter if you create garbage for most purposes

The first thing a game programmer will do in a garbage collected environment is either try to disable it or control it.

Or learn that maybe for the game that they are making it doesn't matter.

And if it does, making themselved acquainted with how value types, slices and gc-free pointers work would probably be a better idea.

>GC pauses have been one of the major barriers to using garbage collected (read: higher-level) languages

There is such a thing as reference counting :)

Trivial reference-counting can lead to memory leaks when you have an object graph that’s disjoint from an RC root object.

Non-trivial reference counting starts to look like mark-and-sweep GC.

And even then, when a ref count drops to zero, the runtime cost of destruction and deallocation of particular objects can be expensive. Stop-the-world GC is bad, but “make a blocking call because you can’t use async APIs in destructors” is worse - especially when the destructor runs in a program thread instead of a GC thread.

I feel that languages which provide support for defined object ownership and lifetime (Rust, any others?) will become the future because in many scenarios object-lifetime can be nailed-down by the compiler and thus eliminate the need for GC or ARC.

From my understanding of Rust, you will still pay the runtime cost of destruction and deallocation of objects with the object lifetime system ? It's like reference counting, but precomputed by the compiler. Please correct me if I'm wrong.

Right. It’s just easier to reason about the process with Rust’s ownership system compared to a reference-counting or GC system.

It’s like manual allocate/deallocate. Only that the compiler will insert free (or drop, in case of Rust) for your variables when it reaches end of scope.

The way the "it's like reference counting" mental model goes wrong is that things don't live arbitrarily wrong. Reference counting is used to extend the life of things, Rust will not do that.

But yes, it's malloc and free.

What makes you think reference counting leads to shorter pauses than tracing? While it is absolutely true that sophisticated implementations of garbage collection via ref counting can exhibit similar performance to sophisticated tracing algorithms[1], most implementations of ref counting aren't nearly as sophisticated as modern implementations of tracing, and tend to be worse.

[1]: https://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf

Not necessarily shorter, but predictable pauses.

Very predicatable indeed.

Go and C# running circles around Swift.

"35C3 - Safe and Secure Drivers in High-Level Languages"

https://www.youtube.com/watch?v=aSuRyLBrXgI

Possible stack overflows and unexpected pauses due to heavily nested data structures.

"CppCon 2016: Herb Sutter “Leak-Freedom in C++... By Default."

https://www.youtube.com/watch?v=JfmTagWcqoE

If your memory usage is regular and predictable, then both ref counting and tracing would behave regularly and predictably, and if it isn't, neither is.

How is memory efficiency? Usually faster and/or consistent GC performance comes at the cost of extra memory usage.

(This is why I'm excited for ARC / Swift on the server...)

From the tests I've seen on beta versions, no difference.

The performance increase is from a different algorithm + full multithreading. Not from collecting garbage less often as is a common tactic to reduce time spent in GC

Go and C# running circles around Swift.

"35C3 - Safe and Secure Drivers in High-Level Languages"

https://www.youtube.com/watch?v=aSuRyLBrXgI

Well, not exactly C#. At 39:00 they show significant latency problems with C#’s garbage collector. Go and Rust did well though.

Will be interesting to see if ARC gets optimized for use cases like this (drivers). I wouldn’t say it’s super representative of general server programming though. And unless I missed it I don’t think they compared memory footprint.

This thread reads eerily like threads about Go's low latency GC from 2015 and how 10ms isn't good enough and throughput will be impacted and on and on. Three years later Go treats any 500 microsecond pause as a bug as Go continues to focus on throughput. Shenandoah is being put together by some very very smart people and I'm optimistic that the only thing that stands in the way of Java reaching the "500 microseconds is a bug" level is engineering hours and resources. More kudos for this achievement are in order.

Is there a guide to the various GC algorithms? Suppose I'm running an app where pause times don't matter, and I want throughout or reduced memory usage. How much more of this will I get by switching to another GC algorithm? 2%? 20%? 40%?

Mostly profiling.

Each JVM implementation (Azul, IBM, OpenJDK, PTC, Aicas,...) has their own collection of GC algorithms.

And their behaviour depends pretty much on the application as well.

Thanks. I understand that the numbers require profiling to determine, but I wish there was at least a high-level guide like:

Do you want to optimise pause times? Use X. Optimise throughput without regard to pause times? Use Y. Optimise memory use? Use Z.

It may not be accurate in every case (no guideline is), but it would be a good starting point for the vast majority of us who don't know the pros and cons of various collectors.

Here are some guides for different JVMs,

https://docs.oracle.com/javase/10/gctuning/introduction-garb...

https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/gen...

https://www.infoq.com/articles/G1-One-Garbage-Collector-To-R...

https://www.infoq.com/articles/tuning-tips-G1-GC

https://developer.ibm.com/articles/garbage-collection-tradeo...

https://www-01.ibm.com/support/docview.wss?uid=swg27013824&a...

> The most interesting new feature I think is the Shenandoah GC.

Well, one could build and run Shenandoah GC with JDK8 and JDK11. Ref: https://wiki.openjdk.java.net/display/shenandoah/Main#Main-B...

Here's a presentation on Shenandoah by one of the principal developers (focuses on the how): https://youtube.com/watch?v=4d9-FQZZoVA

Slides: https://christineflood.files.wordpress.com/2014/10/shenandoa...

Another presentation (focuses on the how, why, and when) at devoxx: https://www.youtube.com/watch?v=VCeHkcwfF9Q

Slides: https://shipilev.net/talks/devoxx-Nov2017-shenandoah.pdf

---

Tangent: Go GC (generational, non-copying, concurrent mark and concurrent sweep) and the improvements it saw from 1.5 -> 1.8 https://blog.golang.org/ismmkeynote

The mutator/pacer scheme, the tri-color mark scheme, using rw-barriers during the sweep phase are interesting to contrast between the two GCs.

Will this help spark performance at all? I remember having to make a lot of GC tweaks in order to get some of the larger jobs to run.

For most ordinary applications I've seen around about 5-10% of the time is spent on GC. Maybe some badly tuned applications would see benefits.

If you want an order of magnitude jump in performance need to see if the IBM work on Scala Native version of Spark manifests.

It should help performance on almost any JVM app, but much more on large heaps.

The GC time is lower and multi-threaded but most importantly the "pause time" is low and nearly constant (based on root set size)

The old GC's all scaled pause time roughly linearly with heap size. Apps that created a lot of garbage would have all their threads yielded for large amounts of time.

Now, essentially, it doesn't matter how much garbage you create. This is awesome because you used to carefully watch how many allocations you made to avoid bad GC times. Now it doesn't matter. Make as much trash as you want and the GC will deal with it

Interesting that they explicitly exclude time to safe point from that improvement. In my experience that is usually most expensive part of a stw pause.

> Pause times with Shenandoah are independent of heap size

I expect this needs to be taken with a chunk of salt, since a smaller heap can only require so much GC...

> I expect this needs to be taken with a chunk of salt, since a smaller heap can only require so much GC...

Pause times. Not GC times. Shenandoah pauses to scan the root set only. The size of the root set doesn't grow with the size of the heap.

You can have a large root set and a tiny heap, or a tiny root set and a massive heap. They're entirely independent.

What happens if a huge object becomes unreferenced and a new one of the same size is then allocated before the older one is GC'd (and you don't have enough memory to hold both simultaneously)? Does it just crash or does it pause to collect the older objects first?

It won't crash, but what it does is somewhat complicated. There's a section in the OpenJDK wiki addressing this question:

https://wiki.openjdk.java.net/display/shenandoah/Main#Main-F...

> Full GC. If nothing helped, for example, when Degenerated GC had not freed up enough memory, Full GC cycle would happen, and compact the heap to the max. Certain scenarios, like the unusually fragmented heap coupled with implementation performance bugs and overlooks, would be fixed only by Full GC. This last-ditch GC guarantees that application would not fail with OOM, if there is at least some memory is available.

> Usual latency induced: >100 ms, but can be more, especially on a very occupied heap

Seems like it confirms precisely what I wrote? You can get longer pauses the less memory you have left, i.e. the more of the heap you've used...

I mean that's effectively a deployment error. You're not supposed to run your application without enough head-room. You can degrade almost any GC by restricting the head-room. You're reading from the 'failure modes' section there.

Reply on news.ycombinator.com