If feeding AI generated stuff to train an AI is bad then what does that tell us about humans consuming AI generated stuff?
If I give a pig something to eat and he throws up I'm surely not touching that stuff myself
If I give a pig something to eat and he throws up I'm surely not touching that stuff myself
Whether it's an AI eating its own training data, a human stuck in an empty cell and forced to live with just his/her own thoughts, or a culture believing its own myths and aggressively punishing insiders and outsiders who challenge them - it's all solitary confinement. And we know how that goes.
(/s of course - although the self-referentiality of memes is balanced by the mad scramble for novelty.)
Or to put it in your analogy, If I feed my cat milk, they'll throw up. That doesn't mean milk is unfit for human consumption.
If AI generated stuff is so bad you cannot train AI with it why would humans use it to train themselves (aka learn stuff)?
a more relevant analogy for llms is a bullshiter that don't know shit about anything but like talk about everything and they're good at that (talking like they know), I'm sure we all at least know or knew in the past someone like that.
at a low enough percentage these bullshiters can thrive and get praise from other people but when there a lot of them and not enough of the real deal, well everyone is in trouble including the bullshiters since they no longer have who to mimic.
This space is where the Venn diagram of marketing and genocides overlap.
At least this is what our specilist at the hospital explained to us.
Obviously ethically we probably should not normalize eating people though.
It's not unqualified bad. It is bad if you do it for many times in a loop, and without any external inputs. But you can put fresh material right in the prompt and it won't suffer from retraining on synthetic data.
If you train a small model from scratch on purely synthetic data, like the Microsoft Phi models, it comes out competent and 5x more efficient than models trained on web text. So there's the flip side. You can do it in bad way, and you can do it in a good way.
A pig (or AI) throwing up on an input should increase your priors that it may be bad for humans, but it does not prove the matter either way.
If you interact with people / fresh data regularly you'll avoid that.
I'm not sure how much is down to evolving requirements and data formats vs degradation of the training data. The idea sounds common-sense but it's also triggering a bullshit warning for me, because I'd expect more consistency per AI model, and differences to occur on new model interations - regardless of the dataset.
Just because it doesn't work with AI now, doesn't mean it's something that wouldn't ever work.
AlphaZero/MuZero game AIs do very well feeding themselves. Often better than previous AIs with hardcoded rules and training based on human games.
Training AI on AI output is less like organised education systems and the less formal written & oral traditions that predate them, and more like dark reddit/chan/other communities eating each other's rhetoric, or some political¹ and religious groups in the wider world. These relatively insular (but sometimes large) groups often descend into a pit they have difficulty reasoning themselves out of.
Maybe the answer for AI is to work out what sort of external mixing helps to keep humans on track, and if it can be emulated in the training models used, so their reasoning continues to grow instead of falling into this pothole.
This might be a crap idea (too little sleep, caffeine has just made me tired and jittery) but perhaps the scale of the information we pile into the process and the fact the retraining at least partly on AI output seems inevitable, means we need to look at training an AI more like training a small population of humans rather than a single unit human.
----
[1] mostly the far right, but that might in part be because they tend to be loud so noticed – far anything is a problem
There is a difference to talking only to yourself and talking with other people.
We know that there is more than one AI software but there is not as much variation as for humans.
And as someone else already wrote here, humans have something like a ratcheting mechanism. I would paraphrase it as "Humans have learnt to stand on shoulders of giants". AI does not have this.