Hacker News Reader

Comment by nelsondev | Hacker News Reader

The p50 speed up from delaying memory management comes with a trade off, namely you get Garbage Collection pauses, bad p99, and spend your effort tuning the Garbage Collector instead of your code.

View on HN

Does it really? With Java’s G1GC the only knob you may end up having to tune is the max target pause time - which pretty much chooses between better p50 and worse p99 va worse p50 and better p90 - aka throughput vs latency, that are almost inherently opposite ends of the same spectrum. It is not GC-specific that any improvement will fail to increase one of those.

So the actual tradeoff is more whether you want better throughput or tail-latency. To improve the latter, you have a singular command line option of using ZGC as well.

You mean the integer parameter "-XX:MaxGCPauseMillis"? The name says it all. You can do over a dozen sync disk writes in a millisecond these days.

Have you seen the output of JVM flight recorder or some other tool used to observe memory allocation/reclamation patterns? The JVM is very lazy when it comes to GC - its algorithms are crazy good and can catch up with really high allocation rates as well. So the maximum pause time’s unit doesn’t really say much, and when it comes to dynamic memory allocation, the JVM is the best choice according to the biggest web services operating on terabyte sized heaps.

With modern concurrent GCs such as ZGC [1] this is really not an issue for most people anymore.

[1] https://inside.java/2022/05/30/sip053/#:~:text=ZGC%20was%20d....

With Java 17 I found that using ZGC eliminated [1] the GC pauses that yield bad P99 latencies without any additional tuning. However, it has lower throughput than G1GC so your P50 advantages will diminish.

[1] Reduced them to always well below 1 millisecond.

Which is always the tradeoff with GC. You get to pick latency or throughput and you trade one for the other.

In my experience, this is not related closely to GC and are apparent in many systems. A great book on optimizations explicitly mentions that “improving one aspect of performance may degrade another”.

Low pause collectors exist explicitly for this purpose. ZGC and Shenandoah both trade throughput (more concurrent collections) for extremely low pause times: https://malloc.se/blog/zgc-jdk16

Does that mean they also trade _memory_ for low pause times? Because, in our application, the memory required for avoiding significant GC pauses is at least 4x the amount of memory we actually need at any given time.

This can be significant too.

Have you measured that with ZGC? Because due to using virtual pointers specifically, ”third-party” tools may report higher virtual memory usage than what it actually uses.

It's all about compromises. Only Rust is uncompromised.

That sounds a lot like the complaints about Java from a decade ago. Have you used it lately?

Java's GCs are incredible and you have a menu of algorithmic options that let you avoid whatever problem you're worried about.

The JVM GC is fundamentally the same as 10 years ago.

Shenandoah did not exist 10 years ago, and G1 was not deployed.

Saying this just shows you don't know much about modern JVM.

Do you use a proprietary JVM at your place?

Nope. Only OpenJDK and Azul Prime.

10 years ago the algorithms you could pick were serial, parallel, and CMS.

Today we've removed CMS, added G1GC, and added ZGC.

G1GC is similar to the parallel in the way it operates but is divided up enough to control for latency.

ZGC is fundamentally different from the parallel collector or G1GC. Suggesting it is the same is to suggest you know nothing of the changes that have happened.

> Java's GCs are incredible and you have a menu of algorithmic options

can these options be used in separate parts of a single app ? e.g. the app I'm developing in C++ has parts that do realtime audio, others that do GPU rendering, others that do classic Qt Widgets GUI, others that do offline computations on datasets - and they all have different performance characteristics and need different memory management schemes to get the best out of each, all while being in a single process; there's reference counting, tree-based allocation, pooled, linear, a GC-ish thing which ensure that memory is freed in specific non-realtime threads... Can that be done with Java or is one tied to a single GC implementation for a given execution of a process?

If Java had isolated heaps then C++ would become redundant in every aspect except memory efficiency.

What are the advantages of isolates vs running multiple JVM processes other than avoiding slightly higher memory usage and some small context switch overhead on IPC - which should be negligent as presumably communication between isolated components would be coarse grained?

I can give you an example: in the Bitwig digital audio workstation, you can choose whether external third-party plug-ins (small audio processors) are in the host process or in external processes communicating through shared memory. I just tried with a very simple plug-in (3 band EQ) to see how much I can stack in either case:

- In the "in-process" case I can stack ~1400 plug-ins on a single channel before I hear a crack in the sound.

- In the "shared memory" case I can go up to ~200 at most. And I'm confident that they really did the very best things possible for the implementation to be performant.

So for me the "things isolated in their own process" means literally getting seven times less out of my system than in the host process (and that's frankly unuseable, definitely not "negligent").

Actually it does if you use the GraalVM Native Image AOT compiler. It's a sort of pseudo-JVM that gets linked into your app and provides the following features:

1. Instant startup due to AOT compilation and a cached heap. Can start faster than C!

2. No warmup.

3. Can create native code shared libraries.

4. Offers isolates, which are segregated heaps that do GC separately but run in-process and which can communicate with each other.

The tradeoffs are that unless you buy the more advanced edition, peak performance is lower due to lack of JIT profiling, you may need to write configs and do other fiddling to ensure the AOT compilation doesn't miss any code that's accessed via reflection, it takes a long time to compile, and you can't dynamically load bytecode (which some libraries do behind the scenes transparently).

> Garbage Collection pauses

True concurrent GC have been available since late 2000s. Azul had a read-barrier GC as well - effectively a pauseless GC (or pauses under 1ms)

> tuning the Garbage Collector instead of your code.

This is a feature, not a bug. Which is riskier?

Now-a-days, if you are tuning your GC you are almost certainly doing the wrong thing.

While the JVM offers a plethora of levers to pull in case you are hyper concerned about different things, the heuristics are VERY good. Mucking with the fine grained details can disable heuristics and ultimately give you worse results than if you just left stuff alone.

The levers to pull are algorithm, max memory, and max pause time. All other levers should be left alone unless you've got GC logs to back up what you think needs changing. (And even then... Do you really?)

Typically, the better route is flight recorder and eliminating wasted allocations.

An uncommon but not unheard method around that (in services that have diurnal patterns) is to throw memory at it and collect once a day during the down time. You can take the server out of service, restart it, and put it back in service.

I read this was semi-standard memory management in the early lisp machine days, because that gc was so slow.

Early Lisp (even pre-lispm) had no GC at all -- it just allocated until it ran out. Memory reclamation was done by saving the heap to tape and then reading it back -- only live objects were saved. So it was kind of GC with extra steps; the first GC algorithms removed the steps.

To this day the Common Lisp standard doesn’t require GC.

Nah, Lisp had GC in 1960 already. McCarthy invented the mark & sweep GC.

Lisp I Programmers Manual, 1960

https://www.softwarepreservation.org/projects/LISP/book/LISP...

Chapter 6.3, Page 95, The Free-Storage List and the Garbage Collector

This is kind of hard to do in practice unless you’re really watching where you’re allocating memory. It’s very easy in Java to end up accidentally throwing a bunch of allocated memory on the heap without realizing it.

Fwiw, the JVM now has a noop garbage collector so this is easy enough to benchmark.

Can you provide an examples of accidentally allocating memory? The only example I'm aware of is autoboxing which is huge misfeature IMO.

Any method call could allocate objects in their call trees, and it may not even behave consistently with new versions.

But it is easy to monitor with Java’s killer observability.

Concatenating lots of Strings instead of using a StringBuffer will do the trick. Sure, people who are familiar with Java know not to do that. But I’ve seen plenty of production code that concatenates.

Reply on news.ycombinator.com