An analogy: what's the difference between a supercomputer, and the same number of CPUs scattered across a few datacenters? It's that in a supercomputer, those CPUs are packed physically as close as possible with expensive interconnects to allow them to communicate as fast as possible. (For many applications, the supercomputer will finish long before the spread out nodes ever finish communicating and idling.) But you need to improve both or else your new super-fast CPUs will spend all their time waiting on Infiniband to chug through, or your fancy new Infiniband will be underutilized and you should've bought more CPUs.