Same with stable diffusion, AI art. And it'll be the same with LLMs.
Eventually the whole internet would be flooded with cheap AI generated content and clearly AIs need human generated content to train on so it'll be the snake eats itself.
Geniune human-generated content will retreat to account-gated networks and private group chats where everyone knows one another. The rest of the internet will just be incestuous AI-generated chum.
In a way, it'd be an improvement. Genuine connection doesn't scale, so let's be honest about keeping it away from random parasites online.
The single most intellectually valuable website on the entire Web is very likely Library Genesis, where the only Web content is a catalog of books you can pirate by clicking a link, and it's, like, a lot more valuable than any other site (even Wikipedia). It may well be more valuable, in those terms, than the entire rest of the Web combined.
If serious book publishers survive a while longer and if non-fiction books aren't overrun with dubiously-accurate AI bullshit, things won't actually change all that much, I think.
As far as written content goes, the (public) Web is most useful for opinions or product discovery, and even those can already hardly be trusted because of all the marketing astroturfing. AI garbage barely changes that already-toxic dynamic.
Video's another matter—some video content on the Web is great and has ~no at-least-as-good replacement anywhere else, in any other medium. But it's also all but completely monopolized by Youtube and hardly participates in or factors into the broader Web.
A chat bot, like search index, need to updated for new and current events to stay relevant. I can’t see why it can’t be deceived by spams
WebText and WebText2 referenced in their papers are corpuses based on Reddit submissions which had a 22% weight in their training model.
https://openwebtext2.readthedocs.io/en/latest/
This is larger than Wikipedia (3% weight) or either of their two book corpuses (8% each).
The only other data included was a filtered set from Common Crawl (weighted 60%).
---
I was imprecise with my language before but hopefully that at least provides some clarity.
This is fair—also in fairness, whenever my Google speaker answers a question, it always tells me which site the answer is from.
For the time being but there is no reason AI couldn't produce genuine original content. Real life human artists also use previous content for inspiration.