Hacker News Reader

Comment by mcguire | Hacker News Reader

A quick note: Shenandoah is not generational, according to the article. Most bog-standard web apps (including REST thingys; not sure why the author calls out those) do strongly obey the generational hypothesis. For most web apps, in my experience, if you can tune your GC to serve the vast majority of your requests from a young generation, your latencies will be good, your performance will be good, your pauses will be infrequent and short, and plump unicorns and bunny rabbits will gather in your cubicle to share their rainbows.

View on HN

Hi, author here. You are saying exactly what I was thinking before. But turns out, generational GCs have nasty failure modes when things don't go as expected. E.g., if an upstream experiences its own difficulties and returns responses slower, our service has to keep all the requests in memory longer, so the heap runs out, and G1 performs a few fruitless YoungGCs (without freeing much) and then tenures all those requests to OldGC, and now you have a big OldGC pause bomb waiting for you.

Non-generational GCs don't have this problem, and it's one of the reasons why Shenandoah suited us well there.

Typo

< tenures all those requests to OldGC

> tenures all those requests to OldGen

If practically everything is collected in the young gen GCs like most request/response applications, do you even gain anything from GC being generational?

Exactly right. The application pattern for which the generational hypothesis is most true is stateless servers.

Keep your caches out of GC memory for extra goodness.

> Keep your caches out of GC memory for extra goodness.

Could you expand on this please?

DirectByteBuffers allow java programs to use unmanaged memory without needing to drop to JNI or similar. There are open source and commercial libraries that wrap that API with caching code. Using one of those solutions keeps your cache out of GC memory.

Caches violate the generational hypothesis. Entries die in middle age: long enough to have survived multiple young generation collections, so that they are promoted to older generations. The problem is that older generations are (a) not collected as frequently, (b) are often larger than newer generations, and (c) have a lower proportion of dead space to live objects, so the effort of tracing has lower value.

Caches that are in scalar data forms (e.g. byte arrays) or off-heap aren't too bad - bytes and off-heap memory doesn't need to be traced. If you're caching an object graph dense with pointers, then not so great.

So does this mean Shenandoah is not suitable for things like caching? Should this even mean I should avoid trying to re-use objects if possible?

Completely vice-versa, Shenandoah is much better for caching because it is NOT generational. [LRU] Caches go against generational hypothesis because the oldest elements are evicted first.

I understand what you mean, but wouldn't the majority of allocations still hapen during a request? For example, generational GC works really well with Elixir and Erlang caches.

> wouldn't the majority of allocations still happen during a request?

Could you please clarify this question? Do you mean that if cached objects are a small part of the total allocation rate, then generation GCs work well with that?

Exactly. Go explored a request oriented collector with explicit generational boundaries.

Well, if caching takes a small part of the overall workload, then you can't really say it's a "cache workload" or "cache-heavy workload", right?

My answer meant that Shenandoah would work well in a program where cache occupies like 70-80% of the heap, and generational GCs might not. But surely, neither are going to break from a 1%-heap cache.

Reply on news.ycombinator.com