It would double the size of the KV cache, which can be significant (multi-GB) at larger context sizes.