Hacker News Reader

Comment by pkulak | Hacker News Reader

With generational GC, does this even matter? Generation 0 is pretty much a stack as far as performance and operation go.

View on HN

Every object requires a header and a 8 byte pointer. This means even an empty object will cost you 16 or even more bytes. If you have perfect data locality and zero cache misses then you will still suffer from exhausting your bandwidth on useless data.

Then there is the fact that you still cannot actually control the allocation. The allocation pattern ABBAABABBA has worse performance on read accesses of all As or all Bs than AAAAABBBBB but with two arena allocators you can write your code to allocate memory according to the irregular pattern but still have continguous data as the end result.

Problem is memory locality, or lack thereof. Having to chase pointers around and not having things laid out in sequence doesn't play well with the CPU cache.

I'm not arguing against value types as I definitely want those too but it's not that simple nowadays - a moving garbage collector (like all modern server jvm's have) can do things for locality that non-moving systems (gc'd or not) can't - https://shipilev.net/jvm-anatomy-park/11-moving-gc-locality/

That only applies in a few specific cases. Basically, a linked list or an array of objects containing no further references. Otherwise, those internal references get immediately followed and significantly dilute the locality.

There have been garbage collectors designed to improve locality of predictable accesses, but they are not the norm.

When you embed a composite object into another composite object, in Java there always needs to be a pointer indirection between them. Even if GC and allocation is free, this is a massive performance loss. (For example, if you have something that would fit into 4 bytes, now you have to fit a 8-byte pointer, and the minimum created object is probably 16 bytes.)

Alternatively you can just flatten everything into a single object, but this loses out on abstraction.

Can't the compiler flatten objects into one another?

For example, if one class includes a reference to another, merge the fields and methods of both classes into one big mega-class, then see if there is any redundant code which can be removed for this new megaclass.

Not in general, because of object mutation, subtyping, and the fact that object fields are semantically references. First, what happens if you update the value to point to a subclass with more fields? They won't fit in the data structure (there are some other problems too, but that's the biggest one). If the classes have to be final and can't be modified through reflection, you'd probably be closer to a solution, but now any time you take a reference to a value in the inner class, you will be pointing into an object of the outer class, which means that GC has to be able to figure out that the internal pointer is keeping the object containing it alive. That's a pretty tricky thing to do, at least in a precise GC. Ultimately, this is not the sort of optimization that a compiler can easily prove safe in general; you could prove it was safe in specific instances (e.g. a local variable that never escapes), but in such cases Java can often allocate the whole thing on the stack anyway (or even in registers). If you want the optimization to work in any other cases, creating something semantically different (that doesn't allow interior references and/or mutability) is a better option.

The GC still has to move objects that survive, and this isn't deterministic. It's fine for a lot of code, but for inner loops when latency matters, you want better guarantees than that.

It doesn't matter nearly as much as people believe. Value objects at this point are a kind of red herring. The JVM does aggressive escape analysis and is also very good at capturing short-lived objects. In the big picture, pointer chasing and memory latency is kind of the last frontier of performance grumps. It really becomes an issue is with very large object arrays that must be iterated over quickly. In that case it's straightforwards to implement your own value-types using code-gen or use excellent libraries like Chronicle-Values [1].

[1] https://github.com/OpenHFT/Chronicle-Values

I'm not sure why you argue against giving the compiler the information it needs to do its job, and instead recommend these crazy codegen hacks, that even describe themselves as "poor-mans" solutions.

> I'm not sure why you argue against giving the compiler the information it needs to do its job

There's several different factors to consider. Value objects will be a valuable addition to the core JVM if only to standardize the many different approaches in place today. But even the standard value types will have strict limitations -- they will likely be immutable and it's not clear that you'll be able to easily obtain their memory address. That means many projects will still need things like code-gen. Code gen, and buffer-backed objects in general, isn't a hack at all it's a proven, battle-tested solution for many domains (finance, scientific sims, big data) that are manipulating several several gigabytes of objects in a performant manner.

Reply on news.ycombinator.com