I believe that performance degrades, as the models end up training on data that is more homogenized and less diverse when compared to original data.

At least according to this paper (I think, FYI I am not a expert) https://arxiv.org/abs/2305.17493