Hacker News Reader

Comment by gwbas1c | Hacker News Reader

> The cost of object creation can be far higher than cleaning them up if they are very short-lived.

I thought this problem was solved decades ago? Doesn't Java have a well-tuned generational garbage collector that can handle this?

.Net was hyper-optimized for this over a decade ago with its 3-generation system, where the 0th generation allowed for extremely rapid allocation and collections. I thought Java had similar capabilities?

(Summary, which BTW is probably outdated)

0 generation: Pre-allocate a continuous chunk of RAM. Every allocation goes to the next bytes of the chunk. When there's no more free RAM left, find all reachable objects in the 0 generation and copy them to the 1 generation.

1 generation: Works the same as the 0 generation, except it's larger and objects are copied to the 2 generation.

2 generation: Garbage collections here are traditional "mark and sweep" collections.

View on HN

I suggest that you pay attention to the numbers in the article. Even super-fast allocation can become an avoidable cost at certain scale.

I get that; but the issue isn't language / runtime specific. You'll hit similar issues in pretty much any programming language / runtime.

Reusing memory used to be pretty standard for C/C++ programmers; and is also standard practice for "real time Java."

Exactly, it is not language-specific. The article could phrase its conclusion better.

The problem is that even if the allocation cost (slab allocator) is zero, and even if the GC cost is zero, a high allocation rate on modern hardware effectively flushes the cache lines at the rate of allocation, effectively reducing your L1 + L2 + L3 cache to 0MB total if your allocation rate is high enough. A slab allocator will be almost guaranteed to be allocating from a non-cached line, and thus the object init will evict a hot cache line every time.

But on top of that, neither the slab allocator nor the GC are free. They're fast, and they're very good, but they also make heavy use of the memory bus, thus competing heavily with the cache-thrashing already being caused by the objects being allocated.

This is why in Java, on a heavily threaded and heavily loaded process, you can see that the CPU cores are far from 100% utilization, but yet there's no blocked threads (and more threads than cores). In other words, once the memory bus saturates, the effective throughput of the CPU drops off and the system cannot make full use of the processing power available.

The relative cost of memory access has gone up 2 orders of magnitude in the past 25 years, and that's before the bus becomes saturated. When the bus is saturated, it can go up dramatically from there (see: queue theory).

If a Java thread is on the CPU, then the CPU is utilized, no matter if the CPU is just waiting for memory.

That is the problem with utilization; you have no clue how busy the CPU actually is. IPC is more useful in these situations.

Errr. If you read the article, his testing was on the allocation side. The collection side was explicitly called out as a non-issue.

That said, otherwise you're right that the JVM generally makes use of the generational hypothesis depending on the collector. The Gen 0 is referred to as Eden in Java-land and uses thread-local allocation buffers that just bump pointers to allocate. They're basically just like arena allocation.

.NET has added multiple C# language versions, a rewrite of the dominant language surface (Span instead of arrays/IEnumerable) and years of investment of a low level stack (http server, postgres driver and socket layers) to overcome heap allocations in favor of performance. It executed in scale what the article suggests.

The multi generation heap is irrelevant at that scale mentioned in the article.

Well, it is solved, but it is still not free.

Java uses something called TLABs (thread local allocation buffer) which are basically arena allocators where a new object creation is a pointer bump (no need to even synchronize!). Later the GC will move longer-lived objects to a more permanent location, reusing the whole buffer for basically free.

The GC isn't involved with allocation, the expensive bit here. The GC handles the clean up. The allocations create memory pressure that can easily saturate the bus to DDR if you are not careful.

The GC can pace the allocation when it can't keep up.

the article seems a bit fuzzy on what they are really calling out as responsible, but the hint seems to be that even with super fast allocation and garbage collection you still trash your L1/L2/L3 caches which leads to lower performance over artificially re-using the same object pinned in memory.

Reply on news.ycombinator.com