There's some cursory indication that in the long tail, training LLMs on LLM-generated data causes model collapse. Kind of like how if you photocopy a photocopy too many times the document becomes unreadable.

This isn't really surprising though. Neural networks at large are a form of lossy compression. You can't do lossy compression on artifacts recovered from lossy compression too many times. The losses stack.

Sure. I'm producing human output from human input in a generally unconstrained, limitless way.

This is producing approximated human output from approximated human input. That second level of abstraction will be constrained, ultimately, by the limits of input.

These new text books could be great at simplifying the subject matter and making the material accessible or they may just never have fully understood the materials and are misleading.

Now imagine that over and over again, imo it's pretty likely to introduce inaccuracies if just taking a naiive approach.

1. Generate inverted problems which are easier to produce than solve. For instance, create an integration math exercise by differentiating a hairy function and reversing the steps.

2. Create (simulated) environmental data.

3. Use adversarial model competition, e.g. self-playing chess or training an artificial image generator/detector model pair.

There's evidently a commonplace myth that information quality starts pristine and exclusively gets degraded by systems thereafter. That's just easily, demonstrably false in myriad ways. That's why it's absurd to conclude that LLM output being in internet training data will cause model collapse.

Information-rich synthetic data can be created without humans, and it works. (Check out Phi, for instance.)

Also, ever read your old journals? You are training on generated data.