Flops are cheap. Bandwidth is expensive. Latency is physics.
We aren't at full lightspeed latencies ram technologies, but we're closer than you may realize, and it's unlikely we can do substantially better any time soon without revolutionary advances.