I think I may not have been clear enough. I think we can take it as a given that there will be some kind of high-speed interconnect between the CPU and the FPGA (as you say, PCIe or QPI or whatever). But this is not the I/O problem I'm worried about. What I'm wondering is how the FPGA gets its access to the outside world. Does everything have to go through the CPU (which would be slow) or would the FPGA be able to use it's own multi-gigabit transceivers to talk directly to I/O devices (like NICs or HMC or Flash).

I always imagined that the best use of FPGA's in systems like this would as an I/O coprocessor. If the only way to get to the FPGA is via the CPU, then most (all?) of the benefit is lost.