I'm also reminded of top-_k_ training/sampling, where you throw away the inconsistent samples: https://arxiv.org/abs/2002.06224 "Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples", Sinha et al 2020.

Which seems like it would be roughly equivalent to throwing away gradients if they are inconsistent...?