A pathetic scenario but somehow consistent with the rules (or lack thereof) of the game
Search engines will have to rely more on signals outside the content, such as links from other authoritative sources, but it does not look like a qualitatively different world.
I also agree that authoritative sources become critical. Yet those typically rely on very human assessments (with their own pitfals and controversies) and in any case much more slow / costly to develop.
How exactly this all will play out is not clear (to me). But the naive technosolutionism of deploying "AI at scale" and believing that it will just work as advertised seems misplaced. The human condition is very reflexive.
I can imagine good enough AI being able to spot truth even better than what humans do - by veryfing sites and commenters with sources of real information to estimate their credibility.
E.g. in a theme similar to Page Rank, you could have an AI that has some sites as a source of objective truth (Wikipedia, science journals, reputable sources of news etc), and then use that as a basis of estimating trustworthiness of a material.
Also, AI could find, for a given subject, opposing opinions, and estimate which ones are possibly fake, and which ones are real.
In essence - do what current fact-checkers do, but for every single website and comment in existence.
Look for english version of article about Nord Stream... Compare with any other langage (no need to know these other languages).
There is something fishy going on here.
https://en.wikipedia.org/wiki/Nord_Stream
https://nl.wikipedia.org/wiki/Nord_Stream
https://de.wikipedia.org/wiki/Nord_Stream
The Dutch and German are a lot longer than the quite short English version. But ... that's just a matter of organisation: in the English the editors chose to make separate "Nord stream {1,2}" articles, in other languages they folded it in one article. On the German one in particular it's just two huge sections.
In short, it's fishy in the same way that bread tastes like fish: not at all.
One month ago, they were no english version available from the french page on the article, only a 3 lines 'simplified english' version were linked.
The thing that makes me question that is the data that is used to train those models to begin with. To disect truth on the internet, can you use the internet as a source of truth to train it?
the irony is that finding "objective truth" is a very non-trivial human game but in all cases costly. E.g journalism has been decimated after losing their traditional ad revenue. Wikipedia and science journals survive because they rely on informal and formal public funds etc.