This is what happens next. The web becomes full of AI-generated stuff, be it art or text or whatever, and people scrape that and train models and it starts consuming its own shit and regurgitating the same kind of stuff. And then human-created content becomes valuable again, and people find new sources of inputs, etc and we go on..

They won't. It is easy to detect AI generated answers (link in the parent thread).

Also the algorithms for detecting fake facts (e. g. "politician_name was born at <random date>") also exist; for knowledge graphs like Google Knowledge Graph it is fully automated; if a website consistently outputs answers that contradict other sources, GKG increases penalty coefficients for all facts from this source.