> The average webpage on the internet is so random and terrible it's not even clear how prior LLMs learn anything at all.
Perhaps we should stop exposing humans to them, as well?
Perhaps we should stop exposing humans to them, as well?
The key word in that quote is “average.” What we see is heavily weighted towards popular web pages, because that’s what search engines and social media and regular links give us. We don’t see average.
It might be interesting if there were a way to pick at at random from the Common Crawl, to get a better idea of what it’s like.
That's why we're all armchair experts in other domains.