Hacker News Reader

Comment by cesarb | Hacker News Reader

I wonder how much the perceived advantages of generational GC are only due to Java's tendency to generate a lot of short-lived garbage.

View on HN

“A lot” is relative—I seem to remember a discussion long ago mentioning that Haskell/Scala-style programs were a problem for the JVM because the JVM is not designed for so many objects dying so quickly.

I remember long time ago JBullet, a physics simulation library, had to maintain an error-prone pool of vectors and matrices because all the calculations involving the creation of temporary vectors and matrices put significant stress on the GC. Today, Minecraft allocates around 600 MB/s for a similar reason (Position3D objects & friends); however, on my PC, I don't notice any pauses.

Is that really error-prone for a physics simulation engine? That's pretty common to only refer to data by indices in other places - sure, there are languages that can deal with it a bit better (e.g. operator overloading, manual memory management and/or value types), but as far as I know this is even often done in Rust, which has the previous properties (it's done to avoid the borrow checker there).

Also, Minecraft was famously written famously inefficiently.

By "error-prone," I mean that if you accidentally stored a reference to a pooled vector and used it after it was returned to the pool, all sorts of unexpected things could happen since everything is by reference. You had to be very careful.

I had a friend/former mentor who proved that object pools are slower in Java once GenGC was introduced because they create pointers from the old generation to the new. He also authored the fasted XML parser of that era and previously helped me land a project everyone but us and our boss said was impossible because Java Is Too Slow, so I take him at his word.

So unless those pools are full of primitive data types, you’re going to cause more frequent full GC pauses, and longer due to all the back pointers.

If you could add some sort of hint to the object pool instance that it should always be in the new generation, then you wouldn’t have this problem. This could probably be provided generically as a class named something like NurseryCollection.

However, I thought the people that really were serious about performance would allocate the pool in memory the GC doesn’t control (back in the day using internal APIs, but I think there are new official APIs for this) such that the object pool and every object in it would just be ignored by the GC.

Looks like an arena allocator is available in JDK 20. But I’m fairly sure it’s been in the RT spec for a lot longer.

> So unless those pools are full of primitive data types, you’re going to cause more frequent full GC pauses, and longer due to all the back pointers.

If you have enough objects pooled to get low allocation rate then you never trigger full GC. That's what "low latency Java" aims for.

I think that the main downside of object pools in Java is that it's very easy to accidentally forget to return an object to a pool or return it twice. It turns out that it's hard to keep track of what "owns" the object and should return it if the code is not a simple try{}finally{}. This was a significant source of bugs at my previous job (algo trading).

You only get low full GC as long as the entire app makes no allocations, not just the hot path. If you allocate garbage at all eventually you will trigger one.

The JVM is more than happy with short-lived objects, they are barely more expensive than a normal stack allocation (mostly by the header only).

The JVM has a so-called Thread-Local Allocation Buffer, which is basically a simple buffer with a pointer pointing to the next free slot. It can be simply bumped up on a new object's allocation and later on the still alive objects will be evacuated to another generation and the whole thing can be cleared. This is faster than any malloc implementation CPP/rust/whatever might use, besides they also doing arena allocations.

Haskell has a few tricks up its sleeve that can help with its kind of allocation patterns, but speaking specifically about Haskell, it's mostly the laziness that makes it quite different.

But it’s not as fast as just allocating a structure on the stack instead of checks graph 20 objects on the heap, all pointing to the next with 64bit pointers. I’ve got Rust struct that are 80 bytes and the same structure in Java is 800 bytes.

What's the difference between the stack and a thread-local, constantly hot/in-cache other region of the same memory? Especially that in many cases the objects are allocated right next to each other, so fetching happens from cache again. Not saying that expertly written Rust code can't beat naive Java one, and especially with the object headers Java will have a performance disadvantage in many cases - but that's a tiny disadvantage in most real-world use cases, multiple times offset by its advantages.

(Besides, Java has on-stack replacement which can do the exact same thing as rust does, but the JIT-compiler Gods have to be in the correct alignment for that)

What’s the difference…?

1. The size.

2. I don’t see that happening and I look at the output of the C2 compiler when I’m dealing with hotspots.

In aware of the cool shit the JVM can do but done right, areas of code can be as fast as C or Rust. It in general, My experience is that Rust is much faster. Sometimes 10x faster where it matters to me (sustained server load).

>Sometimes 10x faster where it matters to me (sustained server load).

I don't think that is a reasonable stat. What do you mean by faster? I'd be extremely surprised if this is representative of any real-world examples where someone has taken moderate care to write good Java code.

Well just the fact that a structure can fit in one cache line vs ten is a start. A lot of the systems I deal with aren’t “ephemeral” meaning the data gets copied out of Eden into survivor and then tenured just in time to die. What else. Interfaces uses that don’t always point to a single type so they don’t get hard coded. Interfaces are quite a bit worse than even abstract base classes in Java if they actually have to be decided upon. Most of the time you’ll be doing that same behavior in Rust using enums rather than dyn. Lets see. “Clever bit tricks”. So like fitting a Set implementation into a cache line if it only has 8 element and using a regular hash set if it’s bigger. So much opportunity.

Deallocating stack-allocated objects is completely free. Allocating in a thread-local buffer maybe can be fast as allocating on the stack, but the allocations still increase the frequency with which the gc has to collect garbage.

Not quite - the still reachable objects can be evacuated concurrently, and then the whole buffer is reset with literally zero work (just setting the pointer to the buffer's start).

Even if collection is run concurrently it is still the same amount of work.

A lot of Lisps don’t even have a stack. It’s effectively a second heap with only call frames on it. That’s one way to implement tail call elimination for instance.

When Java got thread local allocation pools, stdlib’s malloc was still a blocking operation that didn’t scale with thread count. It was out a couple of years before I heard solid stories of someone pushing for a concurrent malloc implementation to be added to stdlob.

So however wide the gap was between stack and heap, its even wider once you had concurrency.

I have long suspected that there is some sort of deeper truth here. It’s not really about Java. If you squint hard enough, a C program has the stack as its nursery, and the heap as a singular old generation. The stack is even arena allocated, of sorts!

This is of course a bit hand-wavy, I haven’t had the time to truly try and investigate this in a more rigorous way.

In general, memory management needs to be understood as a continuum and not a binary "uses GC or uses manual memory management". Per the very discussion we're having, GC is not an atomic thing, and on the other end, what is commonly called "manual memory management" is itself not. malloc still makes non-trivial decisions, as evidenced by the fact that some programs can be greatly benefited or harmed by switching out the malloc it uses. Beyond what most people call "manual memory management" lies "arena allocation", and in the really extreme case, fully static memory layout with no allocation at all. The latter is how NES games were generally written and sufficiently small embedded projects still get written that way today.

Further pushing this into a "continuum" rather than a binary flag is that programs can and do freely mix and match. Nothing stops a program from using two mallocs, some arenas, GC for certain values, and static allocations for yet others. The Memory Management Police do not come along and arrest you for having Impure Designs or anything. There are languages like (I believe) Zig where the ability to cleanly do this is more-or-less a first-class language feature.

Generational GC was perceived as useful even before Java existed, for example in Lisp.

Funnily enough, my impression is that Clojure, the most popular Lisp on the JVM, creates a lot of garbage—probably more than Java—but I believe that’s due to it using immutable collections and not due to it being a Lisp.

Yeah, immutable collections have to use structure-sharing to be implemented efficiently and this frequently results in common operations allocating lots of small objects rather than updating some object in-place.

You can use a borrow checker to replace immutable objects with update-in-place ("ephemeral") ones where feasible.

And, escape analysis and "dynamic extent" hints can have similar effects by stack-allocating intermediate values. Also, you can design your transformations to compose before handing off to a result builder, which is how modern Clojure reduces this problem.

You've clearly never observed the allocation behaviour of a continuation-based compiler (every function call causes a short-lived heap allocation - the heap is the "stack")...

And the stack the heap with Cheney on the MTA.

Reply on news.ycombinator.com