The Tragedy of the Cognitive Commons
arxiv.org
arxiv.org
At least we're (mostly) not scrolling feeds all day at work. AI is going to be like this for all our professional competencies.
If Hacker News is what passes for high quality content in 2026, I feel well and truly bad for the current generation who will never get to experience anything like the old school SMF forums, or pre-2014 Reddit.
I suppose that Socrates would strongly agree, seeing how the same argument could be applied to learning from a book.
There is another charge against writing in the Phaedrus, which is that texts cannot answer for themselves (contrary to spoken interlocutors) or adjust their communication to the needs and character of the reader:
> when they have been once written down they are tumbled about anywhere among those who may or may not understand them, and know not to whom they should reply, to whom not: and, if they are maltreated or abused, they have no parent to protect them; and they cannot protect or defend themselves
An interesting thing about computation is that it is a writing and re-writing that can indeed be responsive and, in a non-trivial sense, "answer for itself". LLMs do this in a form that is obviously very successful for engagement (and for hype) and for producing a lot of compelling output.
I think the Phaedrus also has something to say here:
> Soc. Then [they who who knows the just and good and honourable] will not seriously incline to "write" his thoughts "in water" with pen and ink, sowing words which can neither speak for themselves nor teach the truth adequately to others?
>
> Phaedr. No, that is not likely.
>
> Soc. No, that is not likely--in the garden of letters he will sow and plant, but only for the sake of recreation and amusement; he will write them down as memorials to be treasured against the forgetfulness of old age, by himself, or by any other old man who is treading the same path. He will rejoice in beholding their tender growth; and while others are refreshing their souls with banqueting and the like, this will be the pastime in which his days are spent.
Regarding LLMs specifically, my view is that the current state of LLM-based AI is leading us more towards '"write" our thoughts "in water"', because
* what we write to LLMs does not catalyze a transformation in understanding of them as systems, nor drive them to evolve in a sustainable way towards truth,
* and their outputs are the result of potent patterns in the turbulence of humanity's discourse, rather than an understanding or systematization that could "teach the truth adequately to others"
While I think their potency cannot be questioned, their sustainability and correctness certainly can be. IMO, AI in the vein of theorem provers and semantic webs are much more in line with a text that can actually speak back with integrity and genuine responsiveness, "teaching truth adequately". But I also guess
* the LLM profusion will likely catalyze some important developments and advances in those other approaches (and some use of LLMs will persist even after advances on a difference basis take over) and
* probabilistic methods are here to stay and will likely be more integrated into symbolic systems and
* we are at real risk of totally loosing the plot, or perhaps reenacting the plot of [Echo and Narcissus](https://en.wikipedia.org/wiki/Echo_and_Narcissus#Story), and seeing the warning of utter thoughtless wrought by irresponsible writing (as claimed in https://hedgehogreview.com/web-features/thr/posts/platos-war...)
I don't think the act of writing prompts for an LLM advances thinking/cognition in the same way.
Even today, with the current state of the art, we need far fewer experts. LLMs are already much better at debugging than humans are, for example. Math proofs are falling at a unprecedented rate. And so on.
In the future, we may not find mere human levels of intelligence so useful. Let consumers consume.
In the aggregate everyone maybe learns on average about as much new stuff in a year anyway.
There _are_ still humans out there talking to each other.
Except there's mass directives to eat as much seed corn as you can
There used to be jokes about how men don't want to ask for directions...
--
[0] - Some new phrase I suddenly saw pop up everywhere in the last few days. Must be popular with cool kidz.
Good. No more growth.
There is nothing anyone needs to 'intellect' more of to make anyone's life better. Nothing. You can stop the inner monologue :)
Nothing.
The hype from non-technical people about AI would make you think they already discovered AGI.
How people in tech are aware they're training their replacements?
Or are we quickly approaching an apex where the people running these companies realize AI cannot completely replace human developers?
The boosters are loud and the failures are made silent, until the failures become impossible to ignore. Tens to hundreds of millions of dollars spent on AI in enterprise software companies need to show RoI at some point, and the promises of executives leaning on AI's productivity improvements to make their accelerated release schedules need to actually make the releases. Its currently a very dangerous career move to publically apply critical thinking to LLM capabilities. I think the math sort of does itself.
Same goes for writing, anyone who has written long complex texts with LLMs knows that a ton of editing is required to make it half decent.
If it has some subtle data corruption bug then you're hosed.
It's just crazy to me to think about running AI changes in production on a real product.
Maintenance <> growth
Once you got a skill it’s quite hard to actually lose it. It’s like riding a bike as they say.
I find all the reading of those walls of code exercises my mind better than the writing did. I’m sure I lost memories of how to read a file by hand or balance a tree but who cares?
I am thinking about architecture and design a lot more these days and every decision I make has to be actually argued for even to myself. I can no longer lean back and say “that’s how we always do it” or “too expensive to change now” as I find many people in practice actually do. They were just coasting on premade architectural choices and their “skills” consisted of knowing arcane incantations and syntactic details completely unrelated to the (business) problem at hand.
I am not convinced many developers actually have the skills they think they have. They could wrestle syntax and mess around with tooling, but could they abstract properly? Define clear semantic boundaries? Have proper civil discussions about responsibilities and where they should lie on the right level of abstraction? Nothing has changed in that regard. If anything that part has been amplified. (“taste”)
So basically, if you use it to wave away things you've mastered or explicitly don't want to learn at the moment, you can stay focused on things you are figuring out. I don't see it as any different from writing "by induction" in a math proof without writing all the details because you and the reader know you could easily work them out. In this way I can learn things that I simply would not have the time to dig into before, increasing my skillset. As is was before AI, metacognition is the most widely useful skill to have.
People rarely cared about the details to begin with. Now those that did care can no longer justify the time that was previously required.
Is this -> More people that are not devs, are using AI to create things, and are skilling up to a mediocre level?
or
Is this -> Existing experts, are actually loosing there skills? Using AI is reducing someone already known skills.
Like, problems that I would previously dig into on my own, poking at this and that log file to try to understand what happened to get a system into a particular state, now claude is mediating most of those kinds of interactions and it is the one doing the first pass surfacing of "okay I discovered X, Y, and Z things that are hinky, and I'm not totally sure yet what this all means, but let's look together."
Nowadays I feel almost naked looking at a terminal where I'm typing each character myself. A lot of the old instincts around tab completion and grepping through the --help output of every tool, all that stuff has atrophied somewhat. Maybe that is genuine skill decline?
The only way I've found to make juniors develop that instinct is to make them do it until it comes naturally--to continually demand they verify their assumptions, test out theories while debugging. If you don't practice being comfortable with the unfamiliar, you lose the knack.
It's fine, of course, to lose the knack to AI, as long as you'll never need to solve a problem the AI can't (or can't access). I don't think that's a good assumption for everyone to make.
you have to do all this when dealing with AI too. The syntax is a conversation vs a strict grammar but you still have to do all the exploration, verification, and evaluation you always had to do as a developer. You have to know how to explain what you want and know when you have it, that's not always an easy skill to learn though.
Try it sometime.
And in the morning, I have patches for the new version that pass the test suite.
What skills are being developed by non-devs prompting AI to churn out code they don't understand? Note that I'm not debating if the code is good or works or whatever, I'm asking if any real skills are being developed merely by prompting AI towards some goal.
How could it possibly work? Except she got through setting up her own Apple dev account, all the certificates, subscription entitlement, revenuecat integration, TestFlight, all the paperwork that goes along with a listing and all of that in about 3 weeks. She submitted her app to the AppStore for review last friday.
She didn't learn any coding but she sure as hell produced a pretty good mobile app with a subscription payment system all on her own with the help of ChatGPT and ClaudeCode. She's also producing different tools that run on her website which is hosted on squarespace. I have no idea how she's doing it but she has all these little questionairre and decision tree pages backed by google sheets. She's the second non-coder i've seen really blossom with AI and just run with it building stuff she's always wanted but never knew how.
If they are producing code, there is some knowledge being absorbed. Nobody is using AI to create an app is learning zero.
Its just what they are learning, is not as much as they think, and really random, not-structured. Like reading just a few random chapter of a book across multiple classes. They get a smattering of ad-hoc tidbits of knowledge.
I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared.
Maybe not actually absolutely zero, but it's much closer to zero than what they would learn creating the app themselves
I think the knowledge being absorbed is 'how to use AI to make apps', not necessarily anything about the code itself. I've seen lots of people who are very effective with AI suddenly poleaxed in interviews without it. Going from 'able to explain exactly what needs to happen' to 'staring at a blank page with no ideas'.
It absolutely is. It removes a lot of friction from pathways that kept previous knowledge firm.
This isn't unique to AI. When calculators came around, people had a new tool that decreased the need to do arithmetic in your head. This has lead to a decline in the percentage of people who knew arithmetic, but can no longer do it adequately without the external tool. Another example would be assembly skills declining amongst software engineers after compiled languages become dominant.
What I would argue is unique to AI is two-fold. The first being that the scale of things being automated is enormous. Keeping with just software, AI isn't only a tool that decreases the need to write code, it's also a tool that decreases the need to do your own research, debugging, system design, version control, etc. The list of tasks a regular engineer is responsible for that have not been consumed by AI is small. To make matters more frightening, this is just a single industry. The same applies to multiple other knowledge heavy fields (e.g. mathematics).
The second thing I'd argue is unique about AI is the addictive nature of it. These LLMs are deliberately coerced into being sycophants that blow smoke up your ass unless you deliberately tell the model not too (and it may still do it anyways!). A calculator didn't compliment your insight for asking it to calculate the square root of -1.
Furthermore, many people have talked about the resemblance of LLMs to slot machines--there's a feeling that anything could be possible if you only prompt the model correctly. Your first prompt is wildly off the mark, but your next prompt is better though still not quite right, so you continue forward. You keep iterating your instructions, your word choices, your tone, all in search of the desired response. In this way, an LLM is a slot machine, your prompt the lever you pull, and the output is the jackpot.
All this taken together, we have a machine that decreases the need to engage with the critical faculties of your own brain, which has been trained to keep your attention through flattery, and whose probabilistic nature preys on our love for gambling.
I've got a friend whose firm has started using 'cognitive surrender' to describe the habit of delegating thinking to the LLM in the way that we used to skim code reviews and say 'LGTM' while missing something important.
AI is good enough to generate a first pass for an increasingly large number of projects. An engineer using AI can create an even larger amount.
But muscle memory and expertise are not fixed. They fade over time without use. If you're no longer writing code yourself, you'll get rusty on syntax in the short term. In the long term, you'll get rusty on code structure and layout.
But many engineers are no longer reviewing code either. Reviewing code written by an AI is now the bottleneck, so you're expected to allow AI to review it as well.
So we have a group of people who are no longer engaging in either the writing of the code, or the analysis of the code that's written.
Obviously, this would lead to skill loss.
Many would argue that they're not truly losing skills, because they're more engaged in the grander architecture of the code. To that I would say--your job title says engineer, not architect.
How can you understand them when everything changes again 6 months from now?
Look at the prompts used to make progress on the Riemann conjecture: https://www.anthropic.com/research/riemann-zeta
> Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”). This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress.
Jarred is not a mathematician, and not an expert on the Riemann conjecture. What new skill is needed to tell the LLM to believe in itself? What part of that work couldn't have been done by a ten year old?
The other thing is to give your agent a skill not to solve certain key problems unless explicitly prompted. Write the scaffolding sure, but leave the juicy parts alone. And if I get stuck, I have it enter into a dialogue with me, nudging me towards understanding.
Actually engaging my brain to solve the lower-level, on-the-ground code allows me to think of better ways to do things while I'm writing them. It's like writing anything. You start with something you want to convey, a thesis, and then it evolves and becomes better as you write it. An agent will just write it with no thought, as in it will reflect one of the LLM 'ghosts' as Andrej Karpathy puts it, doing something in the same way that someone in the training data has done it on a similar or different problem. This is why I get conniptions now when I am sent generated text or am expected to read it on a public forum. It's disrespectful of the time of every person expected to read it.
That said, I wouldn't like to go back to the before times without having the agentic option available. Ideally, businesses should not mandate how LLM's are to be used at their company, and just let the devs find their own flow. That is, if quality is even a factor that any company optimizes for anymore.
What these cognitive tools do is make the mundane work we're lamenting the loss of redundant. Who cares if junior devs can't code? When this stuff really gets going they'll be using their answer boxes to both code AND to have have design discussions with a form of intelligence that has every PhD ever obtained and infinite patience.
Consider: long division used to be in school curricula; our forebears had to know it to be considered "educated"; and yet I, in my 40s, was never taught long division.
Where did it go?
Getting past that, the author has a point. There's a loss of shared expertise when there aren't people around learning and doing something. In the US, we've seen this in manufacturing. The number of Americans who know how to set up a good production plant is much lower than it was in the 1980s. That's the consequence of the hollowing out of American manufacturing. The author talks about AI vs. white collar work, but fails to make the connection with outsourcing vs. blue collar work.
The loss of this expertise has recently been made very clear in the US as attempts are made to scale up weapons production for the US's various wars. Progress is very slow, as has been seen with both artillery ammo and air-defense missiles.
The term is not "common" or "commons" but "tragedy of the commons" which dates back to a book in 1968 by Garrett Hardin. Elinor Ostrom subsequently showed that most or all of Hardin's assumptions and claims were wrong, or at the very least, far from universal, and that the process he describes elides what actually drives the destructions of resources held in common: greed and power.
Hobbes, Rousseau were effectively both trying to solve the "tragedy of the commons" problem with political collaboration. Hobbes is widely misunderstood by (romantic, naive, libertarian) fools to have said that absent a powerful soverign, life is "nasty brutish and short". What he more accurately said is that, "an unmanaged commons is nasty bruitsh and short" -- to use that kind of language
Rousseau, to the same romantic strain is widely misunderstood to have said that 'the unmanaged commons' is a delight. Instead, he basically said politically community isnt that required anyway, and if you're in trouble, just run away.
In either case, the tragedy of the commons issue is well known in other guises throughout the history of philosophy
It really isn't. There are lots of anecdotes that appear to illustrate the tragedy, throughout history. But what they lack is what Ostrom added: the historical account of how a previously managed commons had its management structures dismantled by greed and power and only then was in a position to become a "tragedy". The result is that a very significant number of people believe that resources held in common always lead to "tragedy", when in fact the lesson is "the structures used to manage resources held in common are critically important, and severe action should and must be taken against those who try to dismantle them". We can hold resources in common, but must be ever vigilant and actionful against the selfish bastards who seek to manipulate them for their own advantage.
The application of this principle to physical resources is novel, sure -- but the observation that "enforced coordination solves the rational self-interest problem" isnt novel.
But Ostrom's novel observation wasn't that. It was "all those stories of how shared ownership of resources leads to tragedies? they're wrong, and part of a mostly uncoordinated attempt to get you to abandon the idea of shared ownership of resources. It is, as ever, greed and power that lead to those tragedies".
eg. large companies ( 'hyperscalars' ) pre-ordering RAM and GPUs driving up the prices to the point where PC gamers and AI enthusiasts cant upgrade.
Hyper-inequality makes it more likely that greed and selfishness will succeed at dismantling whatever management structures exist for a given commons.
I'm not worried about AI taking jobs. I'm worried that humanity has lost the ability to share at such a monumental level that basic sustenance and financial security are out of reach, even with AI.
After lifetimes of negative reinforcement, the only salvation seems to be the disruption of capitalism itself. Somewhat ironically, the wealthiest and most powerful people in the world seem to be investing trillions of dollars into AI to do just exactly that.
No, humanity hasn't lost the ability to share - it's the exact opposite - trade (and capitalism) has enabled sharing far beyond what it ever was before. Even better, it's done though voluntary and mutually beneficial exchange rather than people being forced into a fixed role in life which was often something horrible like servant, slave-girl, or warrior.
What are we even doing here?
Hardin coined "The Tragedy of the Commons" in 1968, elaborated on his theory in his 1974 paper "Lifeboat Ethics: the Case Against Helping the Poor". He was a eugenicist and specifically targeted refugees. He lobbied US Congress in opposition to international famine relief. His evidence-free theories must be contextualized within his overall white-nationalist project. https://www.splcenter.org/resources/extremist-files/garrett-...
The paper asserts that "expertise within a profession" is a commons. As an industry we've never been able to define the specific, task-level role boundaries between PM/designer/engineer across companies. So how is a profession defined? What are the units of expertise here? This paper fails to provide a defensible definition of their commons. Ostrom's commons need clear boundaries and resource units. https://en.wikipedia.org/wiki/Elinor_Ostrom
From the abstract: "*The paper reframes expertise development as collective stewardship*" which ignores capitalism.
Firms are incentivized to build proprietary expertise, eg Slang at Goldman. Individuals are incentivized to build proprietary expertise and use it as leverage for higher compensation. Even if we fudge commons into a more generic collective action problem, this paper doesn't reckon with the tension between market incentives and collective expertise. This tension comes up again and again, in open source, in academia, and corporate L&D programs.
The author goes on to use professional organizations as an example of governance for their Cognitive Commons. The examples of medicine, law, and engineering are particularly bad as these are credentialed, legally enforced enclosures of expertise, and not self-policing or democratic. They are the opposite of commons.
The author should have just written about expertise as a resource pool and avoided the commons. This was a frustrating read. The paper is a disservice to the commons literature.
but the phrasing of "who pays for the increased schooling times?" is a good one. i think "debt" can be a decent way to conceptualize the cost and repayment of training someone. feels evil to say, but viewing people as firms you can invest in and expect returns upon. you know not all loans will be repayed, but hopefully they'll average to a profit. (risk management etc.)
student loans are. a decent example. the government/private enterprise gives money to pay for education, then this is repayed, providing a financial incentive for paying for someone else's longer schooling timelines. firms investing in training can be viewed as an extension of student loans. but then ah, there are countless stories of how debtor/creditor relationships can be exploited. indentured servitude etc. there are a lot of complications coming to mind. also "altruistic" people who give without expectation of repayment. or the divide between like, communal vs individualistic cultures. (individualism, i argue, encourages the formalization of debt, as opposed to a more communal culture where the expectation of repayment is informal.) you could do math on how many people pay vs how many people benefit, who is the biggest stakeholder, etc.
but i am on my lunch break and need to get back to my work. good article tho. good topic to bring up.