Comment by Waterluvian | Hacker News Reader

Comment by Waterluvian | Hacker News Reader

Kind of a weird New Years fever dream tangent. Bear with me:

Say we have a scene to render and a frame can be rendered in 10ms given the chosen game engine or graphics API or whatnot. There is a hypothetical optimum code for rendering that scene that can do it in less time. And the delta between those times is the price you pay for API abstractions, human abstractions (eg. maintainable code vs clever code), and other things.

Is this a concept with known terminology?

I was thinking tonight, “my hardware is running this game at 45FPS and I know that it could hypothetically be WAY faster if we put a ton of money and brilliant engineering behind it.”

And that got me thinking about how you would go about measuring efficiency loss or waste. We’ve all experienced it: two comparable games running at vastly different performances given the difference various technical and business choices made.

Furthermore, I’ve been very curious about why there isn’t a tool that can provide a concrete answer to, “what part of my PC is bottlenecking performance?” I’ve been playing a game that’s running slow and noticed GPU is never above 80% and CPU is pegged at 100% so I shuffled my computer hardware to essentially upgrade my CPU by a lot. Didn’t have any effect.

NVIDIA's internal term for this is "Speed of Light", or "SoL", though that's more focused on theoretical GPU throughput and efficiency.

Most of the time that you see you're locked on CPU, well, rendering is a huge cost, but often times it's gameplay code, it's the sound engine, it's the file I/O system, it's the Lua scripting. Rendering might very well be a very small amount of the CPU frame time, though it's still there too. As a game developer, when I profile a game, I usually don't see the render thread popping up as much. Things I remember:

* Some audio engine code that worked by running an expensive hashmap lookup many, many times a frame (fixed by changing how the hashmap lookup worked, which accidentally broke an older game, then had to add a configure switch to configure between fast and slow)

* AI Pathfinding code that recursively subdivided a bezier spline. Recursion was unfortunately slow on a specific console platform due to a compiler bug (and this was in 2019!). Fixed by changing it to an explicit while loop.

* A wind simulation system that simulated an entire world's worth of wind physics. Game engine was originally built with much smaller levels in mind, but carried forward for 15 years and now the levels were much bigger. Fixed by only simulating what was needed in the shot.

These things are a huge balance, and we fix it by profiling and getting things to run on our minspec hardware. And sometimes we miss.

It's hard or impossible to know that delta. The amount of complexity that goes into rendering a modern scene is beyond what could be formally specified without putting nation-state level resources and decades of time into it. Even then, you'd have to go through a massive amount of analysis and development to find the most optimal implementation for a particular problem, and it would only be a local optimum, not guaranteeing maximum performance for the entire scene. Lastly, we don't necessarily know the absolutely fastest algorithms for any particular tasks that might be executed in rendering a scene. At best you could use statistical methods to guess at what the delta is compared to a likely optimum based on similarly complex scenes that are recognized as maximally optimized by known techniques (e.g. the demo scene)

Appreciate the response. That’s in line with my amateur musings.

It’s funny how it might be impossible to quantify but it’s often trivial to qualitatively know, “this ought to be way faster…” (but also the biases that make us think that given we don’t appreciate what some games have to do to render that state…)

It's easy to "trivially know" because it's really expensive to disprove the statement, so you continue believing in your original one.

I am not in game development, but was recently looking in some deep learning networks. A lot of compute, bazillion layers of abstractions (erm python), but at the end of the day, 98% of the time was really spent calculating the matmul. I had luck that it was easy to check it and estimate the abstraction cost (not into great detail), and disprove my original thought.

Had i not checked it, I'd still trivially know that far more perf loss happened because of abstractions.

The term you're looking for might be "abstraction penalty".

For tools that can identify where the bottlenecks are, I recommend watching Emery Berger's talk "Performance Matters" from CppCon 2020 (either of the two links below should work). He introduces some innovative new profiling tools for this sort of analysis.

https://www.youtube.com/watch?v=koTf7u0v41o

https://www.youtube.com/watch?v=VzyhpbrC2Bs

The guy is unconvincing.

I don't believe RAM layout is that important. Of course it can affect perf, but slightly, not by 40%. For instance, Windows has ASLR (address space layout randomization) for security reasons, enabled by default. Microsoft would not do that if it would cost 40% of performance at random.

However, there're quite a few random factors. Random choices made by C++ optimizers can contribute about 20% random. Computer thermals when running the test can contribute 40%.

Too much statistics to my taste. For practical purposes, a simple "best of 5 runs" is usually adequate. If that benchmark is too unstable (typical for microbenchmarks with runtime measured in nanoseconds), "best of 20".

The statistics is probably wrong because these distributions are not Gaussian. Execution time of programs is bound from below for several reasons (no time machine, CPU throughput) but not from above (if you really unlucky, the computer may stall and the program will never complete), this factor creates large asymmetry in the distribution.

It's hard to selectively slow down code to emulate performance profile. You'd want same power consumption, and same load of external devices like disks and GPU.

> I don't believe RAM layout is that important. [...]

ASLR has nothing to do with the type of memory layout discussed here. ASLR only impacts compiled code/data, and only entire shared objects/executables at a time.

A bad memory layout can have a huge impact on perf. A simple example is iteration order of a 2d array, where not doing sequential access can result in a ~5x slow down.

> Computer thermals when running the test can contribute 40%.

Only if you forgot to apply thermal paste.

> A bad memory layout can have a huge impact on perf. A simple example is iteration order of a 2d array, where not doing sequential access can result in a ~5x slow down.

I know all that stuff, but the presenter doesn’t talk about RAM layout of data structures. They talk about a few [kilo]bytes offset caused by differently sized environment variables, and layout differences caused by linking order.

> Only if you forgot to apply thermal paste.

Try benchmarking a single-threaded code in 2 cases, in cold state, and when the rest of the CPU cores are running something like CPU stress test (but not accessing IO or L3 cache, i.e. not directly consuming any shared resources). You will easily get above 40% difference, despite the thermal paste.

ASLR only randomizes a handful base addresses of code and data sections, heap, stack etc..., but it doesn't change how data items or functions are located relative to each other.

Memory layout is most definitely important just because memory accesses have such a high latency. This cost may be hidden by prefetching and the cache hierarchy, but keeping the caches well fed is exactly why memory layout matters.

> Memory layout is most definitely important just because memory accesses have such a high latency

Indeed, but that’s not what the video is about. They don’t discuss how to implement cache friendly data structures.

They tell how small random differences introduced by the size of environment variables, and linking order, affect performance. The statement seems to base on that article: https://users.cs.northwestern.edu/~robby/courses/322-2013-sp... The problem with that article, it’s entirely based on one synthetic test. And that test is rather unnatural IMO, that’s not how people are usually writing performance-critical code.

I see such effects somewhat regularly, albeit not at an overall impact of 40%. With precise code layout having the biggest impact, leading to different L1i and iTLB hit ratios. Of course that requires execution costs to be well spread around, rather than allow in a small amount of code.

In my case, working on postgres, this is partially caused by the old school recursive row-by-row query executor model...

I know it can happen, but I would expect the result to be a couple percepts.

OTOH, I did observe up to 20% randomness based on other choices made by optimizer, compiler and linker. In my experience, the main source was different decisions what to inline, especially for release builds with LTO/LTCG. For me, a good workaround was compiler-specific forceinline/noinline function attributes.

However, my C++ code is probably very different from what's in postgres. I usually write and optimize manually vectorized and OpenMP parallelized numeric stuff, FP64 or FP32, doing little to no I/O.

You may be interested by FGASLR, which can do per-function random offsets

Gonna take a wild guess, Cities Skylines? I upgraded from a 1600x/GTX1060 system to a 10th-gen-i7/RTX 2080 system and I still can't get higher than 35fps. A city with 3k citizens and a city with 100k citizens may run at 35fps and 32fps respectively. Though I can run it with supersampling at 250% now and my GPU still doesn't bottleneck it (300% supersampling is what finally cracks my fps, jumping from 30 -ish to 5).

It's so weird to me, the game is definitely CPU bound, I would imagine it's almost all because of AI. So I would expect performance to decline linearly as my city grows. But it doesn't. It's frustrating when I'm playing a small town, but also pretty amazing when I'm playing a big metropolis.

Pillars of Eternity 2.

The ignorant part of me wants to scream, “it’s a mostly 2D game with almost no calculations being made!” But of course I don’t know what’s actually going on behind the scenes.

Oh wow. Never played it myself, but some friends do so I've seen a bit of the game. Given the genre and graphics I would definitely not expect it to have such poor performance!

dwarven fortress

the graphics are... spartan is a generous way to put it.

but it will tax your system! its doing some really crazy stuff behind the scenes

What you seem to be describing is the maximum theoretical efficiency. It's a common thing in lots of fields, and it's valuable because it gives you a goal you can aim for, and know that you have reached the theoretical limit of performance. As a director of mine once put it, it's important to know when to stop [trying to improve something]. For example, the Carnot cycle gives us a theoretical maximum efficiency of a thermodynamic engine, and the Cramér–Rao bound gives us an upper bound on how much accuracy we can extract from noisy measurements.

I don't know what this would be, or how we could determine this, in the graphics world, but it's still a very valuable and interesting idea.

Thanks for the term. That’s helps with searches. :)

You might be interested in the Factorio (also 2D, also where graphics aren't usually the bottleneck) blog posts that talk about optimization, like this one :

https://factorio.com/blog/post/fff-204

There's a concept in the research literature called "superoptimization" -- finding the absolute smallest program that meets a given specification. It's worlds away from being used for anything larger than a small handful of operations, but it's very close to what you're asking about.

Appreciate the term. Another lead for me to google and learn about.

Reply on news.ycombinator.com