Wasn't ChatGPT3 trained on web content with a cutoff of 2021? Future ChatGPT versions will likely be trained on data tainted by AI-generated fluff, and will face the same challenges Google is facing with today's web content.
Most of those SEO-spam blogposts are based on older internet content, and written by non-expert copywriters that can write about 10 different subjects on a given day. A lot of the work is just information compilation and rephrasing, taking some items from a few Buzzfeed lists and changing the text a bit. Some stuff like SEO-spam medical advice can be downright dangerous if done without care. And a lot of companies in this field have been experimenting with AI for at least 10 years.
The problem however is that they require sources, be it freelancers of AI. And there is so much corporate blogspam today, especially in the areas that use content marketing, that it is becoming harder and harder to find "first generation" content written by an actual expert. Google isn't helping by prioritizing newer content.
Note that often enough the "first generation content" never existed in the first place. Content marketers are professional liars, they have zero incentive to find expert sources to plagiarize, when they can write plausibly-sounding original bullshit instead. Writing plausibly-sounding bullshit is exactly what LLMs specialize in, which is why content marketers are interested in them.
We humans have been repeating and imitating ourselves forever. That’s even the way babies learn to talk.
And we are not all clones, we don’t all think, say or do exactly the same things. But most just repeat and consume the same content and ideas (movies, music, books, languages, media).
I think the important part is to have a minimum of filtering. Humans consume knowledge brought by others humans, but most of the time we cherry pick the true and useful knowledge and reject what turns out to be false (ideally).
I think what AI brings to the table is automation and speed, but the quality is not better. So if AI starts consuming its own content, will that decrease the quality of knowledge* in general?
(*Here I mean the first knowledge you quickly get from a search or asking some AI, not what you could get after hours/days of research.)
All incentives of massive industries like SPAM, "content creation", "news" publishing, and advertising, are against it becoming better at rating the quality of its output - or rather, just have it become better at being undetactable but still a cheap fast mass produced wall of text...
Humans repeating and imitating others has historically been a "volatile memory" keep-alive mechanism. Basically saving culture and knowledge in people's minds and helping it transfer, as literacy was low and (manual) writing and book copying scarce, expensive, and time-consuming.
Even when, with the advent of typography writing was easier to reproduce, but still somewhat costly (cost of materials, typesetting, distribution, etc.) and gatekeepers (publishing houses, bookstores, etc.) ensured that most stuff is somewhat original, not just copies or random permutations of the same content. Indexing was also costly manual labor (creating dictionaries, curated bibliographies, books with oversight on a subject matter and references to what the main science/wisdom/etc about that is, library collections, etc.)
Now, however, SPAM and AI-SPAM is on mostly permanent storage - and permanent storage, indexing, and duplication/reproduction costs close to zero and happens automatically at huge scale.
So, no, it's not the same thing. The same way breaking a quick little wind is not the same as having full-blown Taco-Bell-inspired diarhea.
No, it's not. Babies learn to talk by babbling, and then by taking cues from their parents which babbles elicit a response and which don't. If you want to translate that to AI, it would mean that the AI would spew random garbage and then learn to filter its garbage from the responses to its outputs. That's pretty far away from what ChatGPT and other language models are doing right now, because they stop learning before they start producing any output.
So you are already doing a kind of real time RLHF in the chat. That is how DAN was prompted to exist.
He just never specified which direction :)