We're not that far from the speed of light barrier. There's a fixed amount of time it takes for any information to get from one end of a chip to the other and back. Only shrinking the die will decrease that.