Hacker News Reader

Comment by eloff | Hacker News Reader

People use the JVM in soft realtime / financial applications, and the trick is to reduce allocations, especially of objects that are long-lived enough to make it to gen 2.

Go is better suited than Java for those kinds of applications, because it's easier to avoid allocations (Java doesn't have stack allocations, but Go does.) Also hard upper limits on GC time are very helpful for those cases where allocations can't be reduced any further. The standard library additionally has a Pool type that allows for reducing pressure on the GC through object reuse.

View on HN

We use Go for a soft realtime application (RTB), and it works well enough. We essentially have two performance issues, one being the particular database we use (which is not directly related to Go itself), the other one being GC pressure. Before Go 1.3, too many allocations were _very_ noticeable with GC pauses sometimes being close to 100ms (which is a problem when your maximum RTT is 100ms), but Go 1.3 improved things enormously. I'm confident things will get even better with Go 1.4.

When you're 90ms into your bid calc and Go pauses for 10ms STW you will loose all your connections and timeout your bids.

It's fine for basic stuff but if you want to really listen to full RTB bid stream and have real time responses you need pauseless GC to win. Or you have to budget for considerably less time to make a decision.

Also there are places in RTB where you have 10ms RTT and Go fails ought right in that situation at scale. :-)

I'm talking about online advertising. RTTs for common ad exchanges and SSPs are usually 100 to 120ms.

In the financial space the Zing JVM from Azul is very popular, at least it is among the London developers.

http://www.azulsystems.com/

It has a pauseless generational collector called C4 (which is detailed in the Garbage Collection Handbook). Gil Tene, the CTO and founder of Azul, appeared on the Go mailing list a while back and outlined the steps towards implementing the C4 collector in Go.

https://groups.google.com/forum/#!topic/golang-dev/GvA0DaCI2...

This looks like progress towards that. But I don't know if C4 is any kind of concrete (or even speculative) goal. I would love it if standard Go was running a C4 GC implementation.

The HotSpot JVM stack allocates (actually, even better: it SROAs) in very similar conditions to when Go does: when escape analysis decides that a value does not escape. Even if it doesn't stack allocate, bump allocation in the nursery ends up being about as fast as stack allocation (which is, as far as I know, why HotSpot chose only to lock elide and not to stack allocate when they implemented escape analysis).

I haven't looked into the JVM's stack allocation logic in several years, so it may have improved to the point where that is now true. (It wasn't so in 2010.) However, Go does have features that make it easier to control allocations than Java. One example,

  type s struct {
    x int32
    y int32
  }
  v := make([]s, 1e5)

The object v uses 1e5 * 8 bytes of storage. Equivalent Java code would require boxing of each s value. That is, instead of []s, you end up with []*s and allocating each s.

Sure, I wasn't disputing that Go has more interior allocation features than Java does.

I wrote a blog post that showed how powerful JVM compilers like the new Graal can be with removing allocations. I can show a test case in Ruby running on Graal where there are many complicated objects that cross many method boundaries, and they're all entirely removed.

http://www.chrisseaton.com/rubytruffle/pushing-pixels/

Graal can do normal escape analysis, and so scalar replacements, but it can also do partial escape analysis - that is doing scalar replacement even if an object will later escape.

I don't believe that. Yes, the bump allocation step is just as fast as stack allocation because you just increment a pointer. But the extra allocation triggers more frequent collections of the nursery which is much slower than decrementing the stack pointer which is how you "free" objects on the stack. Intuitively nursery allocation just can't be as stack allocation but I haven't seen any benchmarks.

Yes, it's true in general that stack allocation can be faster than even bump allocation in the nursery. But I wouldn't be surprised if the reason HotSpot doesn't bother is that doing SROA, post-inlining, covers enough cases in practice that doing interprocedural escape analysis in order to do stack allocation for a few more objects doesn't help much.

> Also hard upper limits on GC time are very helpful for those cases where allocations can't be reduced any further

Yes, but 10ms is too long to be useful for games. I'd rather take 1ms every frame than 10ms sometimes.

10ms is an upper bound. Games typically don't do a lot of allocations during gameplay, so it's unlikely that Go's GC would be an issue for most gaming applications.

And, as I mention whenever this comes up, Minecraft is written in Java and suffers from huge GC pauses, yet it's one of the most successful games of the last decade.

And yet the GC pauses are noticable and annoying. It's not game-breaking because Minecraft is just that good, but I doubt it'd work well for other games.

I brought up Minecraft as proof that gamers are willing to tolerate the occasional bit of lag. And, honestly, I haven't played an AAA game that doesn't lag out and drop frames from time to time.

To return to the topic at hand: 60fps is 16ms per frame. The proposed GC would achieve sub-10ms pauses. So, worst case you might drop the odd frame. To assume that Go is unsuitable for games because of garbage collection is to ignore reality.

I don't buy that argument. Lua has a much worse STW garbage collector and it's commonly used in games.

Furthermore with games you can do tricks like allocating out of an arena with bump pointers and then resetting the pointer at the end of the frame.

Lots of games are developed with JVM, .NET, or Lua. GC doesn't seem to be a show stopper for them, you just have to be smart about allocations.

Lots of low-end, non-premium-experience games are made with those things. And e.g. games that use Lua usually only use it for high-level gameplay logic, i.e. most of the running code is in C++ or something.

And yes, it's a problem.

See the other comments here about VR. With VR you want to render at 90 frames per second, in other words, you get 11 milliseconds to draw the scene twice (once for each eye). That is 5.5 milliseconds to draw the scene. If you pause and miss the frame deadline, it induces nausea in the user.

But this comment drives me up the wall:

"GC doesn't seem to be a show stopper for them, you just have to be smart about allocations..."

The whole point of GC is to create a situation where you don't have to think about allocations! If you have to think about allocations, GC has failed to do its job. This is obvious, yet there are all these people walking around with some kind of GC Stockholm Syndrome.

So now you are trapped in a situation where not only do you have to think about allocations, and optimize them down, etc, etc, but you have also lost the low-level control you get in a non-GC'd language, and have given up the ability to deliver a solid experience.

Bad trade.

> The whole point of GC is to create a situation where you don't have to think about allocations!

Nope. The point of GC is memory safety.

GC also means you don't have to think about freeing memory, which is important in concurrent systems.

But even if GC was about "not thinking about allocations", what's bad about only having to think about allocations when it's important? Code clarity trumps performance, except at bottlenecks.

Well, you're being a bit revisionist.

You can get memory safety without GC, and a number of GC'd systems do not provide memory safety.

If you think that, for concurrent systems, it is a good idea to let deallocations pile up until some future time at which a lot of work has to be done to rediscover them, and during this discovery process ALL THREADS ARE FROZEN, then you have a different definition of concurrency than I do. Or something.

If you want to know about code clarity, then understanding what your program does with memory, and expressing that clearly in code rather than trying to sweep it under the rug, is really good for clarity. Try it sometime.

> The whole point of GC is to create a situation where you don't have to think about allocations!

I would say for most uses, that's true. For high performance code, it's not. You have to be aware of your allocations on the fast path whether you are using malloc, GC, or something else. Schemes used in C++ to reduce allocations with malloc (which is slow as dirt) also work in Go.

Let's get real here for a second. A maximum pause of 10ms is one-third of your frame budget at 33fps. If you reduce the garbage you're generating it won't even be that much. I don't see how that makes it a show stopper for games.

At the same time, I don't see how it's a great language for games either. Go is slow at calling into C, like 2000 cycles slow. And to do any kind of work with with the cpu vector units, you need to call into C. That seems to be a bigger issue than the GC in my opinion. Rust seems much better suited for games, and I would expect to see some triple A titles using the language once it's more mature.

>Lua has a much worse STW garbage collector and it's commonly used in games.

No, Lua's current GC is better for games. It is an incremental GC. For example World of Warcraft originally had performance issues caused by the standard STW GC in Lua 5.0 but those largely went away after they switched to Lua 5.1 which was the first release featuring the new incremental GC.

I don't know why you're getting downvoted. Variance is terrible for games. We're at the cusp of a VR renaissance where frames are ideally rendered every 11ms (90fps), and VR is way broader than video games! STW is basically a non-starter in this environment.

> (Java doesn't have stack allocations, but Go does.)

Go also lets you have fields in place in structs, like C and C++. In Java, all fields (except primitives) are references.

C# has had structs since V1 I think.

Value Types are coming in the next version of Java. Yes, it is great to be able to allocate on the stack, but then Java has better garbage collectors, including a completely pauseless one available commercially. Even in this announcement the new GC mentioned for Go is non generational. I have a feeling that even Java's good old CMS is better. Java also does escape analisys. All in all, once value classes are in, it is going to be awesome.

Reply on news.ycombinator.com