For tools that can identify where the bottlenecks are, I recommend watching Emery Berger's talk "Performance Matters" from CppCon 2020 (either of the two links below should work). He introduces some innovative new profiling tools for this sort of analysis.
I don't believe RAM layout is that important. Of course it can affect perf, but slightly, not by 40%. For instance, Windows has ASLR (address space layout randomization) for security reasons, enabled by default. Microsoft would not do that if it would cost 40% of performance at random.
However, there're quite a few random factors. Random choices made by C++ optimizers can contribute about 20% random. Computer thermals when running the test can contribute 40%.
Too much statistics to my taste. For practical purposes, a simple "best of 5 runs" is usually adequate. If that benchmark is too unstable (typical for microbenchmarks with runtime measured in nanoseconds), "best of 20".
The statistics is probably wrong because these distributions are not Gaussian. Execution time of programs is bound from below for several reasons (no time machine, CPU throughput) but not from above (if you really unlucky, the computer may stall and the program will never complete), this factor creates large asymmetry in the distribution.
It's hard to selectively slow down code to emulate performance profile. You'd want same power consumption, and same load of external devices like disks and GPU.
ASLR has nothing to do with the type of memory layout discussed here. ASLR only impacts compiled code/data, and only entire shared objects/executables at a time.
A bad memory layout can have a huge impact on perf. A simple example is iteration order of a 2d array, where not doing sequential access can result in a ~5x slow down.
> Computer thermals when running the test can contribute 40%.
Only if you forgot to apply thermal paste.
I know all that stuff, but the presenter doesn’t talk about RAM layout of data structures. They talk about a few [kilo]bytes offset caused by differently sized environment variables, and layout differences caused by linking order.
> Only if you forgot to apply thermal paste.
Try benchmarking a single-threaded code in 2 cases, in cold state, and when the rest of the CPU cores are running something like CPU stress test (but not accessing IO or L3 cache, i.e. not directly consuming any shared resources). You will easily get above 40% difference, despite the thermal paste.
Memory layout is most definitely important just because memory accesses have such a high latency. This cost may be hidden by prefetching and the cache hierarchy, but keeping the caches well fed is exactly why memory layout matters.
Indeed, but that’s not what the video is about. They don’t discuss how to implement cache friendly data structures.
They tell how small random differences introduced by the size of environment variables, and linking order, affect performance. The statement seems to base on that article: https://users.cs.northwestern.edu/~robby/courses/322-2013-sp... The problem with that article, it’s entirely based on one synthetic test. And that test is rather unnatural IMO, that’s not how people are usually writing performance-critical code.
In my case, working on postgres, this is partially caused by the old school recursive row-by-row query executor model...
OTOH, I did observe up to 20% randomness based on other choices made by optimizer, compiler and linker. In my experience, the main source was different decisions what to inline, especially for release builds with LTO/LTCG. For me, a good workaround was compiler-specific forceinline/noinline function attributes.
However, my C++ code is probably very different from what's in postgres. I usually write and optimize manually vectorized and OpenMP parallelized numeric stuff, FP64 or FP32, doing little to no I/O.