Hacker News Reader

Comment by pcwalton | Hacker News Reader

"The execution time is now dominated by the allocation and the garbage collection of small objects (e.g. the Message struct), which make sense because memory management operations are known to be relatively slow."

A perfect example of why Go should switch to a generational garbage collector with bump allocation in the nursery, like Java HotSpot, .NET, V8, etc. etc. use. There are consequences to Go's decision to make extreme throughput sacrifices in order to optimize for latency (low pause times) above all else.

View on HN

Ian Lance Taylor replied to this question once: (https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBU...)

> Now let's consider a generational GC. The point of a generational GC relies on the generational hypothesis: that most values allocated in a program are quickly unused, so there is an advantage for the GC to spend more time looking at recently allocated objects. Here Go differs from many garbage collected languages in that many objects are allocated directly on the program stack. The Go compiler uses escape analysis to find objects whose lifetime is known at compile time, and allocates them on the stack rather than in garbage collected memory. So in general, in Go, compared to other languages, a larger percentage of the quickly-unused values that a generational GC looks for are never allocated in GC memory in the first place. So a generational GC would likely bring less advantage to Go than it does for other languages.

> More subtly, the implicit point of most generational GC implementations is to reduce the amount of time that a program pauses for garbage collection. By looking at only the youngest generation during a pause, the pause is kept short. However, Go uses a concurrent garbage collector, and in Go the pause time is independent of the size of the youngest generation, or of any generation. Go is basically assuming that in a multi-threaded program it is better overall to spend slightly more total CPU time on GC, by running GC in parallel on a different core, rather than to minimize GC time but to pause overall program execution for longer.

> All that said, generational GC could perhaps still bring significant value to Go, by reducing the amount of work the GC has to do even in parallel. It's a hypothesis that needs to be tested. Current GC work in Go is actually looking closely at a related but different hypothesis: that Go programs may tend to allocate memory on a per-request basis. This is described at https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ft... . This is work in progress and it remains to be seen whether it will be advantageous in reality.

I'm aware of Ian's arguments. He's (a) too dismissive of the huge throughput benefits of generational GC; (b) ignoring the fact that both HotSpot and C# allocate objects on the stack too, with effectively superior escape analysis to that of Go because of speculative devirtualization; (c) neglecting the fact that Go's situation is almost the same as that of .NET, where the generational hypothesis certainly holds; (d) treating "request-oriented" GC as something other than generational GC, when it's really just a somewhat limited form of generational GC.

It's just a pointless thing for the Go team to dig in their heels over. They should implement generational GC, like most other languages with optimizing compilers do.

He's (a) too dismissive of the huge throughput benefits of generational GC;

The ObjectStudio VM engineers used to lament over the advantage the VisualWorks GC had: VisualWorks only had to garbage collect one thread. ObjectStudio had gone multi-threaded, and to stop the world, the VM had to signal all of the threads, and wait for all of them to come to an inactive state. The more parallelism in the form of more machine threads, the higher this penalty is likely to be.

treating "request-oriented" GC as something other than generational GC, when it's really just a somewhat limited form of generational GC.

Isn't it more accurate to say that it's a simpler way of getting to use low cost bump allocation and very low cost deallocation? Even generational GC needs some knowledge and skill from programmers to achieve the highest level of efficiency. (Or even some modicum of knowledge to keep it from failing.) The golang request-oriented hypothesis just moves the cost-benefit tradeoff knobs. To get beyond middling performance, you still have to profile and optimize either way. Profiling allocation is usually straightforward.

> The ObjectStudio VM engineers used to lament over the advantage the VisualWorks GC had: VisualWorks only had to garbage collect one thread.

I'm not sure what this is trying to imply. Modern generational GCs use TLABs to solve this exact problem.

> TLAB

Thread Local Allocation Buffer. Cool. Doesn't the stack in golang serve somewhat the same purpose? I've sometimes wondered about golang going the direction of Erlang, with multiple independent GCs. (Not per goroutine. It would have to be per "Actor" which I find myself implementing again and again in golang.)

Modern generational GCs use TLABs to solve this exact problem.

So that solves the problem on allocation, while the generational hypothesis reduces the frequency of stop the world. Generalizing in this fashion, I don't see why generational GC is inevitably necessary. With golang, the stack provides cheap allocation to local threads, even with concurrency, and throughput isn't as big a deal, with a bit more work needed by the programmer for optimization. (But perhaps my view would change with access to more data or with more experience.)

> I don't see why generational GC is inevitably necessary

But remind that almost everybody having to implement a GC (but the Go team) thinks otherwise. Do you believe you know more than langage runtime engineers ? Because that's Go folks seams to believe, and that's anoyingly arrogant…

The fact that Go has acceptable GC performance surely shows that it's not inevitably necessary.

I've seen you rebut Ian's remarks a number of times. I'm not sufficiently knowledgeable about GCs, but I would really like to hear some of his responses to your criticisms. I think I (and probably lots of others) would learn a lot from the debate.

There is most likely none.

Go team at Google would have enough real usage feedback from running Go systems in their massive data centers. They most likely find it reasonable with all things considered.

This debate about Generational GC is theoretical in nature, claiming Java GC design is universally applicable irrespective of language and its use cases.

There may indirect approval to Go approach by Java when developing value types:

http://openjdk.java.net/jeps/169

key Quote:

"In modern JVMs, object allocation is inexpensive, with a cost comparable to out-of-line procedure calling. But even this cost is often a painful overhead when compared to individual operations on primitive values. Thus, Java programmers face a binary choice between existing primitive types (which avoid allocation) and other types (which allow data abstraction and other benefits of classes). When they need to define small composite values such as complex numbers, pixels, or pairs of return values, neither approach serves. This dilemma often has no good solution, and the workarounds distort Java programs and APIs. "

If we go by claims that Java and its GC supporters always made:

1) RAM is Cheap 2) GC is incredibly fast so as not to worry about allocation.

There should be no need to waste tremendous multi year engineering effort to add value types to Java.

That's a straw man. Just because GC is fast doesn't mean it can't be made faster.

> Go team at Google would have enough real usage feedback from running Go systems in their massive data centers. They most likely find it reasonable with all things considered.

That's not just an appeal to authority, that's a big assumption. As far as I know, no Go maintainer has claimed that they have gathered large amounts of performance data and concluded that a generational GC wouldn't improve performance.

Also, calling generational GC "Java GC design" is disingenuous. Generational GC long precedes the creation of Java, and is used almost universally. Common Lisp (various implementations), .Net, the Lua VM (and the LuaJIT VM), BEAM, SELF, and Haskell (GHC at least) all use generational garbage collection. In fact, the current Go implementation is the only GC implementation I'm aware of that isn't generational.

I think it is other way round. No one has yet shown in production applications that Go's GC throughput is big problem for them.

The question to ask is: Have people switched away from Go for certain applications in order to get better CPU-bound performance? Yes, they have.

GC throughput is one part of the reason why Go code is often suboptimal for CPU-bound workloads. Adding a generational GC won't solve this, but it's bound to help.

I don’t think “has anyone ever switched from Go...” is particularly meaningful. Is the switch common? I’ve only heard of a handful (like 3) of cases where people tried optimizing Go and found it to be lacking, and those cases were at the extreme end of the performance spectrum. Also, Go will never be optimally performant because it chooses to balance performance with competing concerns (safety, simplicity, etc), so its unclear in those cases whether the suboptimalism was due to Go’s GC scheme or to other trade offs.

Neither Lua nor LuaJIT use a generational design, they're both tri-color incremental mark and sweep collectors without a separate nursery.

There was a plan for a generational GC for LuaJIT [0] (not to mention a lot of other cool features and a very clever design) but it was never implemented (and at this point it probably never will be).

[0] http://wiki.luajit.org/New-Garbage-Collector

Also, most of the listed languages don’t have value types or added them expressly because their GC scheme was insufficient.

It does not follow that, because value types allow heap allocation to be avoided in some cases, it's not worth investing in making heap allocation faster. That's why I made the observation upthread to begin with.

So for some context here, the work on the Go garbage collector was done by experts, for example: Rick Hudson (https://www.linkedin.com/in/richard-hudson-33339185/). These guys knew the state of the art, and made the decision they thought was best for Go.

This is an incredibly technical area of discourse, and sadly, hacker news probably isn't going to be the place to see it happen.

But the whole idea that Go is made of a bunch of strongly-opinionated, insular developers who have no idea what they're doing is hogwash, especially in this area. Go had a sub-par garbage collector and Google put together serious resources to fix it.

Note that I've never said that Go developers don't have any idea what they're doing. Very smart, competent people can be wrong. I know I'm wrong a lot!

For what it's worth, I've also heard that Go is considering moving to a generational GC, in spite of Ian's initial arguments against it. Hopefully they do.

Presumably after being in production for years, the very smart, competent developers would admit their error (if indeed they should have used a generational GC), right? Hubris? I'm trying to square these remarks with your cut-and-dry portrayal of the issue.

Current GC work in Go is actually looking closely at a related but different hypothesis: that Go programs may tend to allocate memory on a per-request basis.

Are there any go servers architected according to this? There are libraries for HTTP/HTTPS and JSON which strive to have zero allocations, so we're probably close to this as it is. If all code is allocated on the stack and all requests are served by goroutines, wouldn't this fit the hypothesis perfectly?

It's worth noting that Go, unlike Java, can pass objects by value. That makes it less crucial for the GC to be able to deal efficiently with small allocations, since a typical Go program will have considerably fewer of these than a typical Java program.

Same with .NET (C#), where the generational hypothesis certainly holds and therefore .NET has a generational garbage collector.

It's also pretty bad from an ergonomics (not to mention performance) perspective if the programmer has to manually switch to by-value copying of structs in order to keep escape analysis happy and avoid allocation.

Nonetheless, having copy-by-value semantics available for small objects reduces the need for a generational collector.

I'm not sure what you're referring to w.r.t ergonomics. Passing small structs by value should probably be the default choice in Go, and it is not difficult to switch from passing via a pointer to passing by value.

It reduces the need for a collector, any collector, as there will be less garbage. You have not made any arguments that indicate or even hint that the actual share of young objects that will be generated will be reduced.

I believe there is generally thought to be a rough correlation between object size and object lifetime in most programs. So reducing the number of small objects that are allocated in the GCed heap should, on average, reduce the proportion of young objects. Indeed, if an object can be passed by value and/or allocated on the stack following escape analysis, that suggests that it is an object that would have had a short lifetime in the equivalent Java program.

But I don't disagree with what you are saying. It may be that Go would perform better with a generational GC. My claim is not that Go is better off with a non-generational GC (I don't know if it is or it isn't), but simply that it is less in need of one than e.g. Java.

I don't believe that's true, because the Java VM will promote heap objects to stack objects via escape analysis. It even can do this post-speculative-devirtualization.

I'm not seeing the connection between that and anything that I said. Could you expand a bit?

edit: Do you mean that adding pass-object-by-value semantics to Java wouldn't decrease the number of short-lived object allocations because Java already does escape analysis? That may well be true. My claim was that the availability of pass-object-by-value in Go probably reduces the number of short-lived object allocations in Go, compared to what it would be if objects could not be passed by value and everything else remained the same.

> It's also pretty bad from an ergonomics (not to mention performance) perspective if the programmer has to manually switch to by-value copying of structs in order to keep escape analysis happy and avoid allocation.

I kind of like that Go's escape analysis is simple enough that I can reason about it reliably without profiling. I also tend to pass and return values unless I have a compelling reason to pass pointers (like mutation or rarely performance in the case of large structs), so I don't think I experience the sort of ergonomic issues you cite (and incidentally, I think this habit of mine makes writing Rust feel quite unergonomic to me since it seems to prefer egregious use of pointers). And I haven't noticed a significant speedup by switching from value copying to pointer dereferencing (but I've only tested this a handful of times).

You shouldn't have to manually reason about escape analysis. As compiler optimizations of 6g/8g evolve, your ability to reason about it will probably dramatically decrease, due to the effects of inlining, SROA, GVN, LICM, etc. etc. That's why ultimately there's no escaping the need for generational GC.

The beauty of bump allocation in the nursery is that heap allocation is as cheap as stack allocation, at least in the common case.

Even if heap allocation itself is not more expensive, there must clearly be performance benefits to stack allocation, or the JVM wouldn't bother doing escape analysis in the first place.

Java passes objects by value, not by reference. Try implementing swap(Object a, Object b) in Java, and you will see it is by Val not by ref

Java passes references to objects by value. That is, in Java, a variable for a heap-allocated object is a reference to that object. When you pass that variable to a method, the method gets a copy of that reference. So if the method makes changes to the object itself, that is visible outside of the method, but if it makes changes to its own copy of the reference (say, makes it point to a different object), that's not visible outside of the method.

I still consider this behavior "passing by reference" since it passes around references/pointers to objects, and does not incur an object copy.

Expanding on the sibling, Java is a pass by value language which is commonly mistakenly thought to be pass by reference. (That's probably the mistake you think I'm making.) It is not, however, possible to pass objects by value in Java.

Reply on news.ycombinator.com