Gigabit Ethernet is mainly used as a control network. Each FPGA has 4 RocketIO links (2-3Gb/s) and they are directly connected in a 2-D torus of point-to-point links. The RocketIO network is used for nearest-neighbor communication patterns. For reduction operations such as global sums, they call back to the host CPUs for MPI reduction operations to be preformed over the GE network.

Also one of the main goals was to build the supercomputer from commodity parts and "plug-in" FPGA cards, so they most likely had to go with FPGA cards there where available and maybe cheaper then the latest/greatest.

The paper would have been more interesting if they had compared this machine to other types of supercomputers then just CPU clusters.