Hacker News Reader

> lied to Go and told it there was only one CPU but then you loose all the cool features of channels etc..

channels work just fine in a single-cpu context. concurrency is not parallelism.

That struck me as odd. I questioned the entire poster's results based on that statement.

I took it to mean he launched several copies of the process, each copy pinned to one (and only one) core.

Correct it was the only way to get Go to scale to a reasonable response rate for this application and it thus meant my CPU loading was not even since it was based on the randomness of socket connections that persisted on one cpu or another.

He's not talking about merely being able to use channels in one core setup. This misunderstanding made me question your whole comment.

It sounds like he wanted parallelism for performance.

channels work fine in a single-cpu context, but in the setup OP described you won't be able to use them to have the various processes talk to each other.

At least not without using one of the various netchan-alikes or rolling your own channel<->ipc solution, neither of which is likely to result in an elegant solution (note how the original netchan was abandoned by the Go team).

I believe what your parent was looking for is being able to pin a specific goroutine to a single CPU. If you are doing network traffic analysis, if a goroutine will jump between CPUs, you get the cache-miss overhead. This is the reason snort[0] (all in C) doesn't even use threads and sticks to a single process. To be able to use multiple threads when network processing you need hardware support (or something like PF_RING[1]) to distribute your load between cores. You want to be able t keep a data stream (TCP stream) pinned to a given CPU core, and doing that in Go is near impossible with how go-routines are scheduled.

[0] https://www.snort.org/

[1] http://www.ntop.org/products/pf_ring/

> You want to be able t keep a data stream (TCP stream) pinned to a given CPU core, and doing that in Go is near impossible with how go-routines are scheduled.

Doesn't http://godoc.org/runtime#LockOSThread let you do that?

Here's the paper on the Zing JVM's C4 GC: http://www.azulsystems.com/sites/default/files/images/c4_pap...

I'm not sure how much of it applies to Go, but I'm guessing it's quite a bit more complicated than the current collector. It also may only work on 64 bit X86 hardware (which is fine by me, but looks like a subset of what go currently supports).

There's a simpler overview in http://www.azulsystems.com/sites/default/files//images/wp_pg...

One of the C4 authors posted some advice on golang-dev a while back:

https://groups.google.com/d/msg/golang-dev/GvA0DaCI2BU/1EpYa...

The good news is Go has been chipping away at that stuff.

That was a great thread thanks for linking it! If Go wants to be a serious solution in many applications pauseless GC needs to happen and until it does it will be just like all the other solutions that can't play at scale and low latency. If it stays in that zone it'll have to compete with a very large set of perfectly fine solutions with massive libraries and lots of people who know how to code in those languages.

The problem with C4 is that it requires a kernel module, so it's obviously not suitable as the primary GC.

That's a good point and often passed over when talking about C4. It's a real shame, because that is a fairly large sticking point. There was an attempt to get the kernel changes needed by C4 pushed into the standard kernel, but I remember it got a lot of push-back by the kernel devs.

> STW has to die.

Depending on your usage profile, there is very little more efficient from a throughput standpoint than STW collection, except maybe manual memory management with clever use of pool allocation. Even malloc()/free() will be slower in situations where you're creating a lot of short-lived garbage.

Concurrent Gen1 seems fine. You still have to worry about STW Gen2, but it's much better.

FWIW, no one serious about allocations in native code uses naive malloc()/free(). My favorite trick is in game programming where you have a current frame pool that you just reset every frame.

The thing I like about pools is that they seem to actually reduce the cognitive load required to consistently get memory management "right" in a large application.

With manual memory management you have to worry about the ownership convention of each chunk of code, with GC you have to worry about architecting strong/weak references so as not to inadvertently retain everything (not to mention latency issues), and with reference counting you have to worry about ownership cycles. In practice I haven't come up with a better strategy than enforcing some sort of top-down hierarchy which effectively smashes most of the differences in cognitive load between manual/refcounting/GC. GC has slightly less upfront busywork but in practice the tooling tends to be poor so it's a wash (if that). In GC and refcounting it's easy for one inexperienced/tired/sloppy individual to create a massive leak completely out of proportion to the footprint of their immediate code.

Pools, in contrast, allow the same top-down approach with the very significant benefits that I don't have to think about the hierarchy at a finer level than the pool itself (which I have to do for the 3 other approaches) and that memory management mistakes don't typically lead to the globally-connected-component of the dependency graph sticking around indefinitely.

One big problem with pools is that you have to deallocate the pool sometime (or else your program's working set will continually grow over time), and when you do, all the pointers into that pool become dangling.

Another problem with pools is that you can't deallocate individual objects inside a pool. This is bad for long-lived, highly mutable data stores (think constantly mutating DOMs, in which objects appear and disappear all the time, or what have you).

Another big downside to pools is that a lot of GC implementations will scan only 'live'objects. Large object pools unnaturally increase the number of objects that need to be scanned during GC, negating a (sometimes) useful GC efficiency tick.

Precisely. This is an especially important point in response to the oft-repeated "just use pools if you want to avoid the GC in a GC'd language" fallacy.

If your GC is triggered after the allocated memory increases by X% (which is fairly common), then this technique is effective, since it lowers allocation rate.

Also, Go doesn't scan arrays that are marked as containing no pointers, so representing an index as a massive array of values has proven quite effective for me.

I was talking about pools as a standalone memory management technique, not a supplement to GC.

Fair enough. I suppose there are a significant number of applications where there isn't an obvious way to perform coarse-grained partitioning of object lifetimes. If you are a language designer looking to force a memory management scheme on all of your users pools would be a bad choice.

> That helped some but a Nginx/LuaJIT implementation of the same solution still crushed it on the same box, identical workload.

That's very interesting because LuaJIT has GC as well. Could you please reveal a bit more about the type of application, number of lines, and if the code is public?

Not sure if it's related, but LuaJIT can remove allocations all-together in a lot of cases. This[0] talks about how it works some.

[0]: http://wiki.luajit.org/Allocation-Sinking-Optimization

Most JITs do that.

Static compilation can do it to some degree as well, but in most mutation-happy languages (Go included) it cannot prove the allocation does not escape frequently enough to matter.

Not really. Most do escape analysis/SROA.

Allocation sinking is much more general. It works even if the allocation escapes (the allocation is sunk to the point where it escapes).

Additionally, all allocations can be sunk, not just stuff like the point class used in the example on that page. Growing dynamic buffers, string concatenation, etc, are all sinkable.

My understanding is that it's fairly tricky to implement (requires special consideration in runtime design) and so is not very widely used. Last I checked, the Dart compiler is the only other place I've heard of it being used.

FWIW its not really a thing Go would need anyway, given that you actually have control over your allocations.

Sadly it's not public but yes I was very happy to see how well LuaJIT with Nginx peformed for this problem set. I developed several solutions in C, C++, Java, Java+Zing, LuaJIT and Go. The Nginx/LuaJIT (openresty to be exact) solution had a great ratio of "approachable code" to "performance" factor. I was able to get it built and transfer its use to several other teams without them having to know C and still be able to make changes effectively.

It seems that concurrent (non-STW) GC for non-functional languages (randomly accessed mutable data) is a myth.

Yes, Java claims to have an efficient GC (on paper). In reality, everyone know how it works (it bloats, and GC pauses are increasing dramatically).

On the contrary, Haskell has less sophisticated GC but much smaller and more predictable GC timings.

btw, some really smart people are suggesting that it is the code organization, not GC algorithm is a crucial factor. http://home.pipeline.com/~hbaker1/LinearLisp.html

>I cheated in one application and pinned the go process to each cpu and lied to Go and told it there was only one CPU but then you loose all the cool features of channels etc.. That helped some but a Nginx/LuaJIT implementation of the same solution still crushed it on the same box, identical workload.

If your primary motivation for using Go is lightweight concurrency, it has been developed for Lua, though in fairness not as extensively as goroutines:

http://github.com/askyrme/luaproc

http://www.jucs.org/jucs_14_21/exploring_lua_for_concurrent/...

Yeah if you want to hit 60 fps, 2/3rds of your processing time just evaporated.

60fps won't matter one bit if it's 120fps for one second followed by 0fps for another second.

That's why people are asking for pauseless.

Fully agree. I thought the GC only paused for 10 ms, which would be bad enough, if it's a full second... that's insane.

And it's worse, because... "only 60fps?"

60fps was fine pre-VR, but now you want to do 90fps times 2 eyes.

That's a lot of rendering.

If only someone would invent a co-processor that we could offload our rendering to. A "graphics processing unit," so to speak. Oh well. One day!

If you want to update some physics model at 60 FPS, you will be updating every 16 ms. An occassional 10 ms pause is not going to prevent that. Anyway, any reasonable game design needs to deal with pauses that are caused by network congestion intelligently, which can often be longer than 16 ms anyway.

Dude you have NO idea what you are talking about. An occasional 10ms pause absolutely will "prevent that". Those of us who work on software that does a lot of rendering using these fabled GPUs you mention know that feeding the GPU properly is a big problem and requires a lot of code to be running on the, err, CPU.

I don't even know what you are talking about wrt network congestion. What are you talking about??

If you're playing a multi-player game, you could easily lose touch with the servers or other players for more than 10 ms. So the game engine needs to be able to handle these pauses and extrapolate what happened with physics during the time you lost touch. It's the same thing here.

I developed games for Android at one point. GC pauses were common there. In the 1.0 version of Android, the pauses could be up to 200 ms. Now, it's more like 5 or 6 ms on average, as they improved the Dalvik VM. The GC hasn't stopped a lot of good games from being written for Android.

People need to be aware of the realities of scheduling on commodity computer hardware. A 10 ms guarantee is actually very good. You can't get much better than 10 ms timing on commodity PC hardware anyway. Even a single hard disk seek or SSD garbage collection event will probably block your thread for more than 10 ms.

Whether this guarantee is good enough for you depends on what you're doing. But I guarantee you that 99% of the crowd here doesn't know what you need to do to get better guarantees than 10 ms anyway (hint: it involves a custom kernel, not doing any disk I/O, and realtime scheduling).

Extrapolation is not at all the same thing. When you aren't getting network packets, yes, you run locally to do some kind of approximation to what is happening (in fact for a good game, your frame rate is never contingent upon what is coming in over the network, you just always do your best to display whatever info you have right now).

But that is totally different than the GC situation. With a STW GC you cannot even run, how do you expect to be able to display anything on the screen? Even with a non-STW GC, the reason the GC has to collect is because you are running out of memory (unless you are massively over-provisioned), and if you are out of memory for the moment how are you going to compute things in order to put stuff on the screen?

Accessing disk/network/etc induces latency, yes, but that is why you write your program to be asynchronous to those operations! But this is a totally different case than with GC. To be totally asynchronous to GC, you would need to be asynchronous to your own memory accesses, which is a logical impossibility. I do not see how you even remotely think you can get away with drawing an analogy between these two situations.

We both know that to acheive 60 FPS, you need to update every 16 ms. A 10 ms pause should not prevent you from doing this, provided that your code is appropriately fast to run in the remaining 6 ms. And if your code is not appropriately fast, you're going to have a lot of trouble dealing with older and slower machines anyway.

Having the latency be capped at 10 ms is an extremely powerful guarantee. It means that even if your machine is not fast enough to play game X one year, next year's machine will be. Because the GC latency doesn't change, and the rest of the program logic will run 2x as fast because of more memory and cores.

If you want to act shocked, shocked that GC has an overhead, then go ahead. If you want to pretend that nobody can ever build a game in a GC langauge (despite the fact that hundreds of thousands have, on Android and other platforms), then go ahead. Hell, even if you want to continue to use C or C++ to squeeze out every last drop of performance, then go ahead. But I find these comments really disingenuous.

A pause due to network congestion isn't nearly on the same level as a pause in updating your head orientation in VR.

Not sure if this makes it any better, but a bit later on they talk more about how the STW scheme will work. They call each thread which is not the concurrent GC a mutator, and part of the GC scheme will be ensuring that a mutator thread be stopped for less than 1ms, and that the 1ms pause should not stop any other mutator threads. If they can manage it, this scheme sounds promising for most soft real time programs.

That's not how I understood it. It says they're doing a hybrid STW and concurrent GC.

(a) A mutator may be stopped for up to 10 ms of every 50 ms by the STW collector.

(b) If more collection is needed, it will transition to a concurrent collector for the remaining 40 ms. Concurrent may pause stop a mutator for up to an additional 1 ms.

Therefore the worst case is 10 + 1 = 11 ms, not 1 ms.

I read them to mean that the one GC thread would stop the world for no more than 10 out every 50 ms, and in addition the concurrent garbage collector could ask any mutator thread to pause and do no more than 1 ms worth of GC work during any 40 ms non-STW period.

So in terms of real time, your deadlines could be no less than 11 ms.

Couldn't you avoid doing allocations in Go?

You could, but once you're not allocating things, why not write it in a language without a GC?

You could (and many people do) avoid heap allocations by putting things on the stack.