Maybe tuning for latency is the appropriate trade-off given Go's high level of control over allocation and data layout?
To rephrase the question more precisely: can you explain why OpenJDK, which has a multitude of garbage collector implementations tuned for both throughput and latency, performs worse on both throughput and latency than Go on this benchmark which has a garbage collector that, according to the original statement, is "tuned for latency above all else"?
Perhaps OpenJDK is problematic in other ways (the author of the benchmarks suspects it's JIT induced, even though that's attempted to be controlled for), or perhaps this test depends less on the GC for some reason. Or maybe typical Go programs don't require as many allocations making allocation time much less important making tuning for latency the correct engineering decision. Would HotSpot do significantly better as is claimed? That's the sort of interesting technical discussion I'd like to have, instead of pedantry.
I guess being a long time part of Java eco-system, kind of gets tiring of having outsiders (in general, not referring to you) always mixing up Java with what comes with their PC, as if C would be defined by GCC.
As for the actual question, naturally having value types helps reduce GC pressure. Which on Java's case could be helped by trying out Graal or other JVMs that do better job at escape analysis than Hotspot. Alternatively, although it kind of is cheating, using the language extensions from either Azul or IBM for value types.
In any case, when inline classes (aka value types) arrive, Java can easily do the same as Go here.
JIT and de-optimizations play a role certainly, and are to blame for some performance impact, which can be further improved if a JVM like J9 gets used, given that it allows for PGO across runs.
Finally, while Hotspot has good defaults, tuning all the knobs is a science, even with help of J/Rockit and VisualVM, which opens the door for performance consulting.
The JVMs I mentioned, are targeted at soft real time deployment scenarios, as such they have APIs for low level control of memory management, while also supporting AOT with PGO compilation, thus allowing for low level fine tuning out of reach for the regular Java developers (pure Java SE implementations).
Since most of the explanations that you gave involve things like controlling value types and data layout, it sounds to me like you might agree with the statement that in languages with better control over allocation and data layout, tuning a garbage collector for latency over throughput can be a good idea because your time spent allocating is less important. Is that fair?
Some previous discussion https://news.ycombinator.com/item?id=17551012
It should be straightforward (not easy, because compilers aren't easy) for somebody to write another implementation, tuned for another use case - I can't imagine it wouldn't be easier than doing it for Java.
Because those graphs measure overall throughput and latency in an I/O setting, not GC throughput and latency specifically. There are many other confounding factors. In particular, the application in question has been tuned to perform as few allocations as possible, so GC throughput will naturally not show up as much as in other apps!
Thanks to its generational GC and TLABs, allocation in HotSpot is like 5 instructions.
> Maybe tuning for latency is the appropriate trade-off given Go's high level of control over allocation and data layout?
Go doesn't give you control over allocation in a meaningful way.
Allocation being fast isn't very important if you don't spend a significant portion of your time allocating. Isn't it possible that some languages (maybe even Go) by the nature of their semantics spend significantly less time allocating than other languages, and so tuning a garbage collector for latency is the appropriate decision?
Why do you seem to believe that the Go application has been tuned more to avoid allocations than the Java application? If it hadn't been, then you would have to agree that tuning the garbage collector for latency is better because you can invest some effort into your application to have it perform better on both throughput and latency.
Or is the argument that because "[throughput] is typically what applications want", this application is not a good example of most applications (according to what measure of "most")?
The most salient difference between Go's GC and HotSpot's GC is that the latter is generational. There is no convincing reason I've seen for Go not to have a generational GC, which would dramatically improve throughput by enabling bump allocation in the TLAB. The tiny amount of latency that this could add is by no means worth the cost of making allocations an order of magnitude slower. Allocation in HotSpot is five instructions.
This whole "you can't control your allocations in Go" thing is a strawman. You don't have absolute control because of escape analysis, that's true. But there's a pretty wide gap between "oh I'll just move this allocation point out of a loop" and "oh I wrote a bunch of unnatural code". It's pretty idiomatic to manage allocations in Go, just look at all the posts about using pprof.
Besides, the spectrum of "clarity" (or whatever) to performance is present no matter what language you're using. Ex: ripgrep takes on more complexity so it can search a big buffer of text instead of just a line [1]. I wouldn't call that "unnatural" at all, just systems programming.
Here's just one example: The Go compiler judges that parameters to indirect calls such as interface method calls as escaping. So in Go if you don't want to allocate you have no choice other than to avoid interfaces. But Go heavily encourages the use of interfaces, especially in the standard library.
In C, C++, and Rust, on the other hand, you can use indirect calls without allocating, because the language guarantees the escaping behavior. This is a significant difference.
1. optimizing allocations away in serialization
2. optimizing allocations away in api or biz logic in critical paths around that
3. algorithmic improvements on certain naive code paths
4. blocking
... distant gap
5. everything else
Serialization is its own can of worms in Go, and many places we tried to optimize came down to trying to avoid a) stupid (don't use pointers to naive objects) and b) obvious (use value types) and then hit a wall where there weren't many cheap wins.
I probably can count on one hand the number of hot paths which were interface related, so while I've occasionally been annoyed at the forced allocation moving into an interface, it's rarely actual something I've gotten a win from.
That's just one particular experience, but the AMAZING integration with pprof from very early days has saved me far more time in improving perf than other Go annoyances has cost (relative to my experience in Java 2005-2011).
Who defines when a construct is “unnatural”? Can you show examples of “unnatural” patterns in the provided benchmark programs supporting your hypothesis that you have to write unnaturally to control allocations? If not, why are you making those claims?
Why is throughput of allocations important for “most” applications? How much time is typically spent on allocations in those programs? What is a typical application? What about typical applications in Go specifically since that’s the only language that Go’s garbage collector matters for?
Also, why did you bring up the point about generational collectors? It seems like a red herring. This discussion has been about tuning for latency, and how in this benchmark, Go’s implementation did better on both throughput and latency than any OpenJDK collector under any tuning, and that tuning for latency is possibly a sound engineering decision.
I’ve also noticed that you’ve made no explicit attempt to answer my more difficult questions, and if you continue to do so I will no longer assume you are acting in good faith.
There's plenty of empirical evidence—a wealth of papers dating back to the 80s—that generational GC provides better throughput than non-generational GC on most applications. I'd be extremely surprised if a properly-implemented generational GC with bump allocation in the nursery wouldn't improve the performance of Go's GC by trading off a small amount of latency for increased throughput. The reason why you won't see a benchmark like that for Go is that nobody has implemented such a collector for Go.
There’s no way I can believe you are acting in good faith. It’s obvious to me now that you’ve just been trolling for years every time you discuss the topic of garbage collectors. I’m no longer going to engage.
Back in the 90s, it was a running joke that if you were going backpacking, you should take along a 3' length of fiber optic cable. "If you get lost, just bury the fiber optic cable and ask the backhoe operator for a ride back to town."
In at least three programming-related communities I'm in, I have made a quip about "I think I'm gonna replace the 3' length of fiber optic cable in my backpacking gear with a short post about Go's garbage collector", and people have filled in "so if I get lost, I can just take the post out, and then ask pcwalton for a ride back to town".
I can't say whether or not the argument is made in good faith, but man, there sure is a lot of it.
I've been a part of systems that process million of events a second and never allocated. These will beat most C++ systems I've seen, and Go doesn't even start a chance. Java and HotSpot can do some amazing things (esp around inlining), but you do have to do them a little differently (and carefully), but at least you can. My experience with Go is that I never had the same level of control, and I don't think it is possible.
Well, they explored generational GC (after admitting that their uber-ultimate GC that they marketed as future-proof until 2025 could be better): https://blog.golang.org/ismmkeynote
> It isn't that the generational hypothesis isn't true for Go, it's just that the young objects live and die young on the stack. The result is that generational collection is much less effective than you might find in other managed runtime languages.