This is probably the most important response / question in this discussion.

Assuming for a moment that AI-generated content will be ubiquitous enough to impact the sources of a new generation of AI tooling, what is the mathematical limit of this recursion? Is it a Mandelbrot set? A (Douglas) Adams-esque 42? Will it be an expose of ultimate truths? The seed of the singularity? Or a bunch of grotesque amplifications of the worst parts of the human condition? Perhaps all of the above?

Since I don't have the time, I certainly hope some forward-thinking grad student, or suitably motivated genius is experimenting with this now.

I believe the result might be similar to when one does oversampling to avoid class imbalances. When doing oversampling it happens that exact duplicates are trained. This is equivalent to increasing the weight for these examples. This might fix the class imbalance but at the cost of increasing the bias of the model. This means the model might overfit to the duplicates shown in the training set.

In the case of LLM trained on mostly LLM generated data you will see a similar increase in bias of the model and a overfitting to LLM generated data. This might lead to limitations of the performance of LLM in the future.

I'm sure there are still going to be sources for training data.

Academia should still be good to train on. As far as "public-facing" sources, maybe we'll be able to prune away AI generated content somehow?

The system as a whole will probably optimize for engagement, much like what is already happening with current media, just at hyper speed.