Tensor parallelism, so you only need to store a fraction of kv cache per gpu.