Swapchains and Frame Pacing | Hacker News Reader

Swapchains and Frame Pacing

raphlinus.github.io

89 pointsraphlinus17 comments

The previous related thread:

The compositor is evil - https://news.ycombinator.com/item?id=24466929 - Sept 2020 (175 comments)

IIRC Overwatch has a setting that attempts to delay work on a frame until the last possible moment to reduce latency.

Edit: I noticed that the thread about the previous article also mentions Overwatch, although only its audio. Curiously, it doesn't mention the huge audio latency (around 150ms) Overwatch has. It even got some comments from top Quake (I think) players at the time. They might have fixed it since then, I haven't played much in the last few years.

Somewhat related, but more to do with motion blur: If one has more control over the hardware, it might be interesting to take a look at emulating arcade style beam racing. Blurbusters has some articles and threads about this. ZisWorks used to sell a prototype display that had a strobing backlight with progressive refresh -- the display would refresh with the backlight off, then, once a quarter of the frame was displayed, the backlight in that quarter of the display would turn on briefly.

Black frame insertion is another interesting technique, used with console emulators and high refresh rate displays. This is also used in Sharp's flagship phones -- the ones that supposedly have 240Hz refresh rates, it's actually just 120Hz with black frame insertion.

I talked about beam racing a bit in the "compositor is evil" post, but this time around wanted to focus on techniques that are practical to ship and can work reliably rather than depending on fickle hardware dependencies. I do think VR is going to motivate more work in this direction though.

The example chart of a "highly optimized game" shows games systems being multithreaded (in addition to physics). As far as I am aware, this is still pretty rare in games, unless I am wrong. I know Unity is trying to parallelize gameplay code (rather than just networking, I/O etc) via DOTs, to mixed results. Are there other games[0] or engines that are attempting to do this? I know their are GDC talks from Naughty Dog and Bungie about how they parallelized their engines, but the Bungie one about Destiny still made it seem like the gameplay logic was still largely single threaded.

[0] I believe Noita (a falling sand type game) parallelizes the falling sand update loop by breaking the screen up into different regions with "dirty" rects and splitting that work out amongst multiple threads.

Apple had a way to add a second thread to OpenGL to do some pipelining work. Prepping some of the data to send to the graphics card happened on a worker thread, allowing some speed up, but if memory serves it was less than 30%.

I think this document describes it: https://developer.apple.com/library/archive/documentation/Gr...

These sorts of improvements were interesting in an era of casual gaming on 4 core laptops, but when laptops start having 16 cores this is not so interesting.

NVIDIA and AMD's drivers currently offload a bunch of stuff to driver-owned worker threads for you, they've been doing it for a long time.

The chart is based loosely on the Call of Duty presentation (linked in the article), which does employ parallelism to reduce latency.

There's also a lot of interest in ECS engines that can parallelize the work, for example Bevy tries to make parallelism easy.

Reading through it, it seems that use parallelism in rendering to reduce latency but I'm not sure if they also use it for game logic (e.g. updating NPC positions, calculating projectile trajectories, etc.)

I don’t understand why you put PS2 as part of an era of synchronous rendering. It’s the epitome of asynchronicity: it exposes multiple DMA channels to send data and commands to the vector units and the GPU and good use of that parallelism was advocated to devs a big deal

What would you suggest instead, as a well-documented system? I considered a few, but went with PS2 because of the excellent analysis. My understanding is that it's asynchronous from a programming model, but close to synchronous when it comes to timing, because the queues are pretty small. I could have this wrong though.

Also the PS2 is closer to 20 years old, not 30. Unless I'm missing something (20 year old console using 30 year old rendering)?

Gah thanks, I will fix this. Originally I had an SGI Indigo, but I switched away from it because there isn't anywhere nearly as good a description of its architecture as copetti's on the classic consoles.

Windows 11 has a new composition swapchain API (https://docs.microsoft.com/en-us/windows/win32/comp_swapchai...) that's supposed to make it easier for apps to control frame timings. (I don't know enough and haven't dug in enough to evaluate it myself.)

That looks really interesting, thanks! I hadn't seen it before. It certainly looks like it has most of the goodies we want, including present identifiers, the ability to target a specific time, etc.

One question I have is whether this new API supports smooth resizes, or if that's still broken by design.

Now, what's your opinion on adaptive sync (Freesync / G-Sync)? Isn't it superior to V-Sync in all regards? I guess, in the near feature we will see even phones supporting this.

I think it doesn't help that much - if your system is well engineered, you should be able to hit the frame deadline reliably - and it also makes the situation with compositing more complex. My personal guess is that it will gradually be subsumed by high refresh rate displays, often running the game at some fraction of the full rate. You still need frame pacing to take full advantage of it.

> While triple buffering is at least moderately popular for games, I know of no UI toolkit that employs it, and the power concerns basically take it off the table for mobile.

Unity, which is the most popular game engine for mobile, is using triple buffering by default.

Reply on news.ycombinator.com