Hacker News Reader

Comment by modeless | Hacker News Reader

1 ms pause times are pretty good. That's finally getting close to the point where it may no longer be the biggest factor preventing adoption in applications like core game engine code. Although at 144 Hz it's still 14% of your frame time, so it's hardly negligible.

Even if the GC is running on an otherwise idle core there are still other costs like power consumption and memory bandwidth. So you still want to minimize allocation to keep the GC workload down.

For too long GC people were touting 10 ms pause times as "low" and not bothering to go further, but truly low pause times are possible. I'd love to see a new systems language that starts by designing for extremely low-pause GC, not manual allocation or a borrow checker. I think it would be possible to make something that you could use for real time work without having to compromise on memory safety and without having to pay the complexity tax Rust takes on for the borrow checker.

View on HN

> at 144 Hz it's still 14% of your frame time, so it's hardly negligible

A single skipped frame is not a big deal and will probably not be noticed. It will probably happen anyway due to scheduling quirks, resource contention with other processes, existing variation in frametime...

True realtime work requires no dynamic allocations whatsoever (which, notably, is not covariant with gc!), so I think ‘low’ pause times are an acceptable compromise. Where performance is a concern, you need to manually manage a lot of factors, among them GC/dynamic memory use. There's no runtime that can obviate that.

Granted, 1ms pause times are probably still not low enough for realtime audio, and there may be room for some carelessness there (audio being soft realtime, not hard realtime). But I think just being careful to avoid dynamic allocation on the audio thread is probably a worthwhile tradeoff.

> A single skipped frame is not a big deal and will probably not be noticed.

Attitudes like this are why my phone sucks to use and I get nauseous in VR and GC devs spent so long in denial saying 10 ms pause times should be good enough. Yes, single dropped frames matter. If you don't think so then I don't want to use your software.

A single skipped frame usually means that we are talking about soft real time. And there it is absolutely acceptable, not in the average case, but eg. on a heavily used computer a slight drop in audio is “appropriate”, it’s not an anti-missile device.

It won’t make the normal case jittery, nauseus or anything like that. Also, in regards to your GC devs comment, I would say that attitudes like this is the problem.. The great majority of programs can do with much more than 10 ms pause times.

A slight drop in audio would be perfectly unacceptable for eg computers running concerts.

You're bringing up a specific professional use case in a general-purpose discussion. If you run concerts on anything other than dedicated hardware and software, you're going to have a bad time. If you record at home without low-latency devices and soft-rt software, you're going to have a bad time.

It's in the same category as "the performance of stock Camry is unacceptable for racing" - yes, plan accordingly when entering a race :-)

Dynamic allocations don't cause any issues with hard realtime, as long as you don't run out of memory.

Most allocators are not constant time, and are fairly slow anyway. (Actually GC tend to have faster allocators, but obviously unpredictable pauses.)

(Though there was an allocator I saw recently that promised O(1) allocations. Pretty neat idea.)

Core game code commonly uses custom allocators that do provide those semantics though.

A bump allocator that you reset every frame is O(1) and a dozen or so cycles per allocation for example.

Sure, yes. I was referring more to ‘general-purpose dynamic allocator’ (malloc or so). I agree custom memory management/reclamation techniques can be fine for RT; just semantics.

> A single skipped frame is not a big deal and will probably not be noticed.

Some folks definitely notice this phenomenon, called a "microstutter" by that group. You can see it here:

https://testufo.com/stutter#demo=microstuttering&foreground=...

Noone mentioned how frequent frame-skips are we talking about.

Is a single frameskip in an hour a problem?

A constant frame interval is better than occasional skipped frames. You don't need a super high frame rate for perceived smooth motion, but dropped frames look like stutter.

Stutter is becoming much less of a factor with variable refresh rate display technology. Modern consoles, TVs, and many monitors are being built with VRR these days, and in a few years it will probably be ubiquitous.

Unless you have a highly optimized game, you are probably not able to consistently run at a 144 Hz monitor’s native refresh rate anyway, so even without skipping frames you will see stuttering. VRR solves this problem as well.

I'm not sure if you and GP share the same notion of stutter or not. I never saw stutters when limiting the game at 24 or 30 fps while playing on a 60 Hz LCD monitor in the past. It stutters only when the fps is not constant.

VRR makes that kind of stutter essentially irrelevant, as the penalty for missing the refresh interval is basically zero. They don’t really drop frames, it’ll be displayed as soon as it’s done, so the frame will only delay for however late it actually is, instead of waiting a whole 1/60th of a second for the next refresh

Of course, we’re generally talking about much higher refresh rates, at least 120 if not more.

My display runs at 165hx and you’d be surprised how many games could hit that a lot of the time... even on my old 1080 (and certainly on the 3080 I have now).

The most frequent issue I run in to is actually games having hard FPS caps (120 typically, sometimes 150) and refusing to actually max it out.

Is that measured somewhere / have you got a reference? In my experience 120hz dropping frames (or more precisely rendering at 100 +/- 10fps) looks much smoother and nicer than constant 60 or 30fps.

Although at 144 Hz it's still 14% of your frame time well if we believe their numbers, the worst case is 0.5ms so 6% of frame time for 144hz. Assuming their stated average pause time of 0.05ms then average pauses (and the GC isn't constantly pausing) take 0.6% of frametime, which is negligible. Though your concerns on throughput and resource usages stands. Well newer programming languages could leverage ZGC (and improve upon it) by targeting graalVM + it enable cross-language interop.

In my experience GC developers wildly underestimate their worst case, so I don't really believe that 0.5ms number. But more importantly, you should not use average pause time at all. At 144 Hz the 99th percentile frame time occurs more than once per second. If you want to avoid dropping frames you need to design for the worst case.

ZGC used to target 10ms as worst-case latency, and they target 3-4ms now I believe.

The article above says that the target is 1ms.

Thanks for the correction, I remembered the 3-4 ms from the inside java podcast on the ZGC.

No worries.

There is a worst case that is much worse then the now mentioned 1ms. Namely really big change in allocation rate in which case the gc cycle does not finish before running out of memory (ZGC uses allocation rate to determine when to start next gc cycle). In that case ZGC stops allocations ("Allocation Stall" in gc logs). But this is basically a failure mode and should not happen during normal operations at all.

Though you can configure for this in a couple ways if you run into this issue

1. By telling it to treat some amount of heap as the max that is not actual max when it comes to it calculating when it should start next gc cycle. (-XX:SoftMaxHeapSize) https://malloc.se/blog/zgc-softmaxheapsize

2. Increasing the amount of concurrent gc threads so they will finish their work faster (-XX:ConcGCThreads)

3. Just run really large heap so the "run a gc cycle even if we don't need to based on allocation rate" gc cycle keeps the heap in check.

Though after JDK 15 we have not had to mess with any of these. Prior to that we had to adjust soft max heap size a bit. With JDK 16 it should be even better I guess (should be upgrading sometime next week)

I think games are just a bad example. Many games have both simple ownership semantics (making manual memory management easier) and simple object lifetimes.

In a game an object is likely to live for exactly one frame, exactly n frames where n is like 2 or 3 for some kind of deferred rendering, or an entire level. Some games mightn’t have levels as such but objects still tend to be grouped together in when they load/unload.

I think it is important to note this because games that need to care about performance are not using malloc. They will use some kind of arena allocation because allocating and freeing memory with malloc can be expensive. So what games want here is control over how allocation happens rather than when free is called—that is, it wouldn’t be sufficient to have a language where you called free but couldn’t use a custom allocator.

But there are standard ways to deal with these sorts of allocation patterns in a language like Java. For buffers (of eg vertices) these may just be reused. For smaller objects, pool allocation may be used which sucks but doesn’t suck much more than manual memory management.

I think it is silly to focus on games (or, for that matter, short-running command-line tools that needn’t call free at all) because the allocation pattern is so unusual and weird. I can’t think of great examples of systems that care about latency and don’t have game-like allocation patterns but maybe a complex gui application like a web browser or PowerPoint would be a good example.

Games also care heavily about data layout, which you can't really do in a GC'd language very well. Well, game engines care heavily about that, anyway. Pooling objects doesn't get you there.

There's no tradeoff between using a GC and controlling data layout. CLR already allows value types and JVM is getting support for advanced value types and specialised generics (templates) over them.

At this point, I wonder whether it's possible to architect an efficient GC that predictably runs only when alloc() is called and thus helps to avoid pauses in main loop. Of course it works best with a language having value types.

This is the standard way traditional incremental GC works. Typically allocation is done with a protocol like

1. Try to bump a pointer to allocate the memory

2. If there wasn’t enough memory, call a special function (which runs the GC) and try again

This special function may do a big gc pause if necessary or it may just do a small amount of gc work and grant the program some more memory (this way most gc work is shared more evenly with program time). In modern VMs like Java’s, concurrent gc (where gc and mutator run in separate threads possibly needing a pause for some steps) is more popular.

GC will most likely never be used in demanding games. You want total control over memory. 1ms sounds ok but still you don't know when and for how long the GC is going to kicks in.

> for how long the GC is going to kicks in.

1ms (max, average at 50us)

And for the 'when' I'll add that the very concept of having a concurrent GC means you don't need to do a (potentially pausing) malloc right in the middle of what you're trying to do.

The kind of people that care about GC pause times have their own allocators that are as cheap as jvm allocations and cheaper deallocation. They aren't poopooing GC's and then just calling regular malloc and free.

Engines rely on smart allocator and memory pool, they usually allocate everything beforehand. You're not running malloc between two frames. Imagine a game like Battlefield if you were to allocate memory for each fired bullet.

The biggest engine in gaming, Unity, uses c#, which is GCed.

It's def not the biggest, it's almost not used in "AAA" games also.

Unreal also has a GC to deal with. I've spent more time on AAA games than I'd like to admit trying to hide/mitigate/optimize the hitch.

I would say Unreal is the biggest game engine in terms of pervasiveness. Also, isn't C# just used as a scripting language in Unity? All of the heavy lifting is dome by the C/C++ backend.

Unity is way way way way more common than Unreal.

Unreal is more common than it was 5 years ago, but it's still a very distant 2nd. Probably 9/10 games released on Steam in 2021 are Unity.

You may not realize it, since the devs are slightly less bad at hiding the fact thay they're using Unity. (How did anyone at Unity ever think that disaster of a "launcher" was a good idea?)

Unreal uses a GC for Blueprints and C++ entities.

Yeah. These "you can't use GC in games" comments always crack me up.

https://docs.unrealengine.com/en-US/ProgrammingAndScripting/...

"Garbage collection in Unreal Engine 4 is fast and efficient, and has a number of built-in features designed to minimize overhead, such as multithreaded reachability analysis to identify orphaned Objects, and unhashing code optimized to remove Actors from containers as quickly as possible."

I always feel like there are a LOT of posers in every HN/reddit discussion of GC who have tricked out gaming rigs and are desperate to max it out in every game they play, but aren't actually game developers themselves. Whilst out in the real world there is Unreal, Unity, heck even Minecraft all making mad coin using garbage collectors at their core.

HN is mostly full of FOSS and Web development culture.

Coming from the demoscene and having had a foot on game development, I am fully aware that stuff like monetizing IP and getting a game published no matter what and how, are much more relevant in the game development communities than whatever gets discussed over here.

Just look at e.g. the general opinion about Flash around here, and how it was embraced by the game development comunities.

On Reddit there are gamedevelopment forums where you will have more luck finding more real life experience.

Then there is IGDA, Gamedev, Gamasutra, Making Games, PAX, IGF and similar forums.

Fact is, not every game engine needs to be used for a Far Cry clone, and there are many ways to make money.

And the amount of man-hours collectively spent on working around this terrible, terrible GC is immense.

It was the worst GC implementation I've seen in my life, could cause 0.5s GC spikes every 10 seconds on Xbox One even though we were allocating none or very little memory during gameplay. The amount of pre-allocated and pooled objects was bringing it down to its knees, because Unity's GC is non-generational and checks every single object every time. In the end we moved a lot of the data into native plugins written in C++. Nothing super hard, but you choose high-level engine to avoid such issues.

I've read that in 2019 they finally added incremental mode GC, that solves some of the issues but is still far cry from modern GC's.

Reply on news.ycombinator.com