Is it the AI that's the trouble or the hostile new information environment we're expected to navigate and survive? Expecting us to remain sane amidst these torrents of information without new tools for querying and filtering it is cruel.
Specifically, they mention that "knowledge collapse" is the narrowing of the "human working knowledge" and the universe of things "worth knowing" or "epistemic limit", which may result in obscuring/hiding what they call "the long tail of knowledge" (they define all these terms). They explain this better than me, but I think it's all about constraining the human relation to knowledge by these new "recursive AI" processes, without any malice involved.
I don't know if the paper mentions this, but I've thought about this (in less smart terms). It's probably not that AI is a radically new process in this sense but, like a lot of tech, it makes it faster.
We often say "well, things were like this with [written language|printing press|the internet]" but isn't it possible that a relatively harmless process could be accelerated to a harmful level by technology, even if it's not "fundamentally" something new?
Can we always fall back to the belief that "people from previous generations complained of $THING and it turned out ok, therefore $NEW_THING is probably OK too"?
The real issue as I see it is people who have that kind of knowledge need to share it and have it be seen by those who are interested in it.
There’s a huge problem with search engines serving junk ‘narrow middle’ SEO-optimized content to the endless horde, as opposed to real, quality knowledge. This issue predated generative AI.
This is what happens now, but do we know it will remain so? If AI is carelessly integrated into the teaching loop, I suppose it may narrow the knowledge even there. And then, how will people know there's more to be known?
And even outside formal teaching, how will regular people know there's more to be known beyond what AI tells them?
How will you even know you "need to know more"? Who will tell you?
Already we see a lot of what people "know" (about anything: biology, WW2, ancient history, politics, art) is mostly informed by Hollywood and internet-spread "pop knowledge" (directly or indirectly, as in watching a documentary or reading a newspaper article that were themselves informed by regurgitated "pop knowledge").
(In case anybody is wondering, I don't consider myself outside of this. I know I'm impacted!)
So this phenomenon isn't new... but will AI make it worse?
Rephrasing McLuhan "The medium is the m[ea]ssage" we can say that "The AI is the medium and the m[ea]ssage".
It seems obvious to me that a significant step to combat this effect would be not calling the thing "artificial intelligence". I like the term gpt because it is descriptive and accurate. Gpt doesn't suggest in any sense that we can rely on the thing or are comparable, in the same way "cell phone" doesn't. Imagine if we really bought into the idea that phones are truth receivers, or called them the closest friend. But that seems to be the idea with AI.
Keep speaking up too. Just calling bullshit on the thing implicitly gives people permission to turn it off. The tech will not cause knowledge collapse among a population that doesn't need it. I do use it by the way, eg GitHub Copilot which is nice.
Every google search now returns loads of AI generated websites fill of low quality information. You could call the things "bullshit generators" and the people who really want to make websites full of bullshit will still use them to flood the internet with that bullshit. We all suffer the degradation of the internet when that happens.
Again, it's good that you call bullshit on it. Things are not as set in stone as they appear -- the tech is finicky and expensive, and we still have the wealth of "legacy knowledge" online and offline. There is enormous investment being made in this sector currently. Remember the huge amount of bullshit that was spread all over during the height of the cryptocurrency craze, uberization, social media, war on terror, etc. As I said, "AI" is a new part of the complex. Turning these things off is a way to stay sane, no more or less.
That is a google problem, not an AI problem.
I think it’s worth asking questions about how our data is being used to fuel corporate interfaces to that data, who owns the rights to that data and those systems, and consider how we might maintain independent creators on such a system. Will iFixit still have the resources and means to take high quality documentary photos and write ups for new devices if everyone consumes that information via an Apple, Google, or OpenAI interface? How will you post a structured, expandable, localizable repository of your own data to share, aka how will you publish web pages and how will people find them?
I believe society will solve these problems via the actions of many individuals and organizations, but with certain organizations holding so much power it’s important to consider and inquire on how that power is being used.