Memory bandwidth for unbatched decoder-only autoregressive inference should be the only bottleneck. The listed numbers imply you're only saturating about 500 GB/sec, ignoring KV cache traffic. Implies there's still quite a bit of inference performance left to squeeze out of the 960 GB/s spec.