The issue is because of the current nature of LLM. Basically an LLM is attempting to model its inputs but it tries to auto regularize/generalize, which means information loss. And then if you use that LLM to generate more content to train another LLM, once it is regularized/generalized it will have further information loss.

This isn't that different than re-compressing images over and over again. They are disconnected from reality and are just simulations of reality and with each iteration they become less rich and faithful.

AIs need to learn from the world or at least as close as possible to it, rather than from their own reduced representations of the world.