Well, you’re learning from data you generate.
Sure. I'm producing human output from human input in a generally unconstrained, limitless way.
This is producing approximated human output from approximated human input. That second level of abstraction will be constrained, ultimately, by the limits of input.
These new text books could be great at simplifying the subject matter and making the material accessible or they may just never have fully understood the materials and are misleading.
Now imagine that over and over again, imo it's pretty likely to introduce inaccuracies if just taking a naiive approach.
Also, ever read your old journals? You are training on generated data.
Of course we still need real world data, but it seems like generated data should also play a role. Humans don’t weight dreams equally with reality, however, and that’s a distinction I feel is missing here.
1. Generate inverted problems which are easier to produce than solve. For instance, create an integration math exercise by differentiating a hairy function and reversing the steps.
2. Create (simulated) environmental data.
3. Use adversarial model competition, e.g. self-playing chess or training an artificial image generator/detector model pair.
There's evidently a commonplace myth that information quality starts pristine and exclusively gets degraded by systems thereafter. That's just easily, demonstrably false in myriad ways. That's why it's absurd to conclude that LLM output being in internet training data will cause model collapse.
Information-rich synthetic data can be created without humans, and it works. (Check out Phi, for instance.)
There's some cursory indication that in the long tail, training LLMs on LLM-generated data causes model collapse. Kind of like how if you photocopy a photocopy too many times the document becomes unreadable.
This isn't really surprising though. Neural networks at large are a form of lossy compression. You can't do lossy compression on artifacts recovered from lossy compression too many times. The losses stack.
If you want ML to do it... well that's a bit of a catch-22. How would an ML algorithm know if data is good enough to be trained on unless it has already been trained on that data?
Eventually AGI's will need to learn by experimentation as we do, but in the meantime ability to well predict a potential training sample could be used to decide whether to add it to training set or not. At the moment it seems the main emphasis is on a combination of multi-modality (esp. video) and synthetic data where one generation of LLM generates tailored training samples for the next generation. I guess this synthetic data allows a more selective acquisition of knowledge than just adding surprising texts found in the wild.
This seems doable, and, I think something like it is already done?