I saw a quote somewhere to the effect of “LLMs are lossy compression of the internet” and it seemed about right.