This does indeed improve efficiency significantly: https://arxiv.org/abs/2207.07061