Hacker News Reader

Comment by MPSimmons | Hacker News Reader

Can someone please project that curve out and estimate where it starts to flatten?

That's not very useful. That curve might be a combination of a few different bottlenecks, l1 cache misses, scheduling bubbles, efficient scheduling of instructions within a certain window, l2 cache misses, cache contention in L1 or L2, AMX utilization, memory utilization (latency or bandwidth limited), MMU contention/page misses, etc. etc. etc.

The percentage of all those can change as you change the number of cores, and the different levels of the memory hierarchy get different levels of contention, latency limits, or bandwidth limits. So in an ideal world you can draw a graph and extrapolate, but in the real world you might do significant better or worse. There are cases (admittedly rather rare) where performance increases by more than linear in relation to cores.

> There are cases (admittedly rather rare) where performance increases by more than linear in relation to cores.

Ooh that's interesting, do you have any leads or examples?

It's called super linear scaling. If you have a fixed size problem, that doesn't communicate much, and is CPU intensive you can run on a single core and be limited by memory bandwidth or latency. But as you use more cores you can fit in L3, L2, and even L1 as you add more cores.

It's seen sometime even on multinode jobs. But it's definitely the exception to the rule, but it does happen. It's helped by the fact that each level of cache is approximately 10x faster, so fitting in a cache is a huge win. So running in ram on 1 node might be 20x slower than running in cache on 10 nodes.

Reply on news.ycombinator.com