Building the perfect memory bandwidth beast

Where do you work and how much does it cost to work with you? ;-)

Memory hard crypto POWs such as Equihash or Autolykos spring to mind.

Volume rendering.

Factorio, interestingly. Their dev blog talked about how it was typically constrained on memory bandwidth.

Maybe a low latency hybrid cache & query engine where data is mostly in memory but some computation (eg: aggregation at different percentiles on the fly) with low latency requirements would benefit.

Generally speaking, something where:

1. The data is in memory or can be made to be, for the duration of the need.

2. The overall throughput isn’t limited by some external I/O operation (eg: cache servers might seem like “memory hungry” things, but will bottleneck on network throughput before memory throughput [note: latency is definitely not throughput here]). - the CPU operations involved once data is fetched from memory are very cheap, but also generate a sequential but high volume of writes. Maybe an ideal example is incrementing every integer in a large array by 1 since reads and writes are predictable and SIMD instructions can push the theoretical per clock CPU throughput even higher.

Disclaimer: I might be missing some other scenarios from a lack of creativity. The question you asked got more interesting the longer I thought about it, and I think it might have something to do with why this “Memory Bandwidth Beast” hasn’t yet had its day in the sun.

Fast Fourier Transforms

Looking up many items in a large hash table. So like an in-memory K-V store could probably be designed to work well around such an infrastructure.

Memory bandwidth is 1000x lower than CPU bandwidth, so as a rule of thumb any algorithm whose work scales linearly in the amount of data being processed will be memory bandwidth bound, and also any algorithm which can't be structured to do a lot of work on one memory region at once before moving onto the next one.

Examples (for large enough inputs that it's relevant) include shuffling, sorting, kmeans clustering, branch and bound sudoku solving, vector addition, dot products, and so on.

Moreover, writing a particular piece of code is often easier if you ignore memory bandwidth as a constraint. The classic example is matrix multiplication -- it can be structured such that even disk bandwidth isn't relevant compared to CPU bandwidth, but doing so is a little fiddly compared to the naive n^2 dot products approach, so writing it yourself usually results in a memory bandwidth bound solution for large matrices.

Similarly, writing two passes over your data rather than doing a mega-loop, the choice to use classic kmeans rather than one of its approximations (when it would be appropriate to do so), or not enforcing sortedness at some reasonable boundary and having to do additional passes over your data. It's easy to write code that hoovers up way more bandwidth than it needs to, and often faster algorithms that come out don't do anything different than access the right data at the right time to reduce that pressure, like a trinity rotation [0].

Caveat: Benchmark everything, especially as you're building intuition. Trying to fix what you think is a memory bandwidth issue can result in pipeline stalls and all sorts of fun things, especially when your server has more faster caches than your dev machine, when data in prod doesn't match your micro benchmark, ....

[0] https://github.com/scandum/rotate