Are large language models a threat to digital public goods?

> They used stack overflow to make their case, and report that user engagement has gone down after the release of ChatGPT. Could it not be the case that SO is less adept at finding related/duplicate questions than ChatGPT? Given the later's facility with the language, I would expect it to be. So I look at the paper to see if they accounted for that, and find this.

The moderation team and community in general on Stack Overflow is so toxic I'm not even sure you could control for that effect well enough to arrive at this conclusion. I would argue people are leaving because it's easier to ask ChatGPT your question than be flamed and banned for asking how to do something. A half right answer from ChatGPT is better than getting marked duplicate and closed because the moron moderation team can't detect nuance.

In 2013 or maybe it was 2014 I was tasked with building a couple of things for our Sharepoint 2010 that ran our intranet. I’m a programmer but it was pretty far out of my area of expertise and it was hard to find anything documentation related, probably because Sharepoint 2013 had been released. Anyway, I made things work by doing a lot of google programming, of which a lot led me to SO. Most of it was build with JavaScript but I still needed to figure out how to work with Sharepoint APIs and Lists. Fast forward to 2023 and I’m tasked with working with our current Sharepoint. It’s still far out of my comfort zone, but the Microsoft documentation is on point and it turns out to be a fairly trivial task to engage with the APIs. This story isn’t really unique to Sharepoint, and while obviously very anecdotal, these days it’s very rare that your average blog spam or SO answers are more helpful to me than official documentation, and often the official GitHub code examples. Now I’m more experienced today than I was in 2013 so that helps as well, but for the most part, I think the key reduction for my personal google programming has been how much better we as software developers have become at documenting our tools.

So in 2023 I almost never visit SO, not because of GPT but because the people who write frameworks also supply you with documentation and implementation examples.

Again anecdotal, but I do use GPT quite a bit. Though rarely to help me figure something out. It writes my documentation, unless the code is very secret. It sometimes writes some basic code like auto-generating a docker file or basic API classes. Things that aren’t in competition with SO. I’m sure it’ll take over the role of SO for some parts, but to me personally, it seems far more likely that SO’s decline is linked to a range of other challenges as well.

the hypothesis is wrong to begin with

they should ask people that ask questions, not people passively looking for answers

the experience asking a question on stackexchange sites is horrible! each tag is its own community with edicts and customs you have no idea about which derail your path to actually getting an answer, the auto-moderation system is completely broken because it thinks your question should be replaced with one from 2013, or the human moderators unilaterally fix your question which then causes the auto-moderator to think its similar to the one from 2013 and unceremoniously replaces it

With chatgpt you dont need to prove anything about what you tried, you dont need a reputation score to do anything, you dont need to be a steward of every upvote for all eternity, you dont need to engage in meta discussion to get your post out of deletion, and it answers faster. You don't need to be told to ask a separate question if you have any followups, with all of the same gamble of problems, while chatgpt already assumes what you will have a followup question about and just tells you all the gotchas. Because its read all the forum posts and has seen what the recurring issues are.

its better, faster, and cheaper time wise

its not a direct comparison, chatgpt is more like a pair programmer. If stackoverflow had an adhoc pair programming live session you could hop into it would be a better comparison.

> We observe no change in the votes posts receive on Stack Overflow since the release of ChatGPT. This finding suggests that ChatGPT is displacing a wide variety of Stack Overflow posts, including high-quality content.

My alternative hypothesis: ChatGPT is eliminating all the long-tail, garbage questions, so the vote patterns don't change because the missing questions weren't getting much of them anyway.

To expect change in vote counts and/or ratios, you have to assume that users have sort of "vote budget" that they are going to spend one way or another. If voting is mostly independent variable and mostly depends on particular user and particular post then you should not expect voting patterns/counts to change meaningfully.

> Could it not be the case that SO is less adept at finding related/duplicate questions than ChatGPT? Given the later's facility with the language, I would expect it to be.

Given the later's facility with the language, I would expect it to be a better search engine. I would expect that ChatGPT is replacing Google as sort of tokenizer. I.e. search pipeline changes from "form natural question -> input to google -> go to SO -> if first few links do not yield answer post new question" to "form natural question -> input to chatgpt -> extract keyword tokens -> input to google -> go to SO".

There is an important bit in the article:

> > Using data on programming language popularity on GitHub, we find that the most widely used languages tend to have larger relative declines in posting activity.

Here we can form a hypothesis that reduction in post frequency comes from entry level posts with posters not knowing what to search for. Under this hypothesis ChatGPT has strongest effect on users using SO as knowledge base rather than Q&A forum. This user type distinction would affect posting frequency much more heavily than voting patterns.

Some of my colleagues are using ChatGPT and blindly follow its advice. ChatGPT's answers often look more persuasive than human answers on SO.

Still, I find ChatGPT answers much harder to validate. In programming Q&A, the answer is usually a series of calls. When you try to apply the human solution (assuming it's at least somewhat correct), the error often lies in input data mismatch or some minor changes in requirements when the answer is not perfectly aligned. Rarely the calls themselves are wrong - they might be deprecated if it's an old answer, but generally, they do at least exist. With ChatGPT, you never know which particular part of the answer has been hallucinated.

Also, as SO has more stringing question requirements, you are forced to construct a minimal case to reproduce the error. Often, composing an SO question got me straight to the cause of the error.

What happens after LLMs kill off SO and then seek more updated training data?

It seems like these “deaths” are either temporary or something else will pop-up that continuously improves and trains an LLMs with proprietary data.

There’s def a shift in the social contract of the internet. We’re shifting from “publish a few things you know a lot bit in exchange to read stuff other smart people have shared” to “solve a problem with an LLM and train it in a manner that can be helpful to others”

What’s unclear is how the middle of that will be monetized. Search engines figured out how to do it for the “publish and share” paradigm and made a boatload of money off of it after paying for the massive infrastructure required to index everything. How will LLMs do it without killing the data that trains it?

I'd expect user engagement to be going down in the same trend that has been occurring over the past five years as Stackoverflow is less and less useful.

There is a weird sort of analogy where universities spent the last 20 years putting their classes online, or associate professors and grad students uploaded YouTube deep-dive videos, etc. Point is, enrollment at Universities are way down now and you could probably draw some kind of link between the two.

However, there are also links between jobs that no longer care, and GenZ is a smaller cohort, and also maybe more of an interest in trades as a path forward.

I'm not suggesting the paper linked here is right or wrong, but we should keep an open mind about all the other potential reasons why things suddenly shift. One random reason could be that during this overly oppressive year of layoffs engineers have either stopped needing to search or haven't had time to work on a project that needed it.

Also there's smarter IDEs which are likely the #1 reason I've stopped searching the internet for one-off small things like "what's the API to deep copy an array in Rust." Documented autocomplete has gotten really good and things like copilot are just icing on the cake.

Putting all knowledge in the hands of a for-profit company is the opposite of “democratization of knowledge”.

The problem with this sort of comparison is that up until now, new ideas came from humans and technology advances merely helped to spread ideas, or greased the wheels. Now you can generate new ideas. Often without any skill of your own.

This is bad at first glance because look what happened when smart phones became popular: no one remembers phone numbers of their family, people can’t remember directions when driving. Now you don’t even need the ability to generate ideas.

And it’s not clear to me that LLMs democratises information. LLM have guard rails, are potentially trained on copyrighted data. Need to be run by large companies…

Doesn’t seem like a valid comparison

True democratization of knowledge would mean LLM's are open source and usable by everyone (the way of Wikipedia).

Even then you'd want attribution to survive.

Eliminating or changing the incentives people have for doing things is the absolutely surest way to change behavior.

Postulating that all published work can be appropriated at will by certain private for-profit entities is the death knell of the knowledge economy as we know it.

> improved propagation of ideas has led to improved dissemination of ideas, and it will this time too

I sincerely hope you are shitposting. So called Web 2.0, the platformized web has placed huge incentives to stifle propagation and dissemination of ideas. However, these incentives are somewhat distributed under human review, therefore there can be bubbles with opposing ideas. LLMs centralize censorship, by design.

Currently you can find "BrawndoLounge" and "ToiletWater" subreddits that heavily censor opposing ideas. Even if one is full of hate speech or whatever, there still are opposing ideas there. If LLM guardians decide that one of these sources is "toxic", in an LLM-led web that viewpoint simply vanishes.

> improved propagation of ideas has led to improved dissemination of ideas

Prove this.

If you're only pasting backtraces, you're not taking advantage of its coding ability. You can describe the problem, have it write some code, then iterate by telling it what to add, cases you want it to handle, add features to the code. It's been doing a pretty good job of helping me designing a database down to the actual CREATE TABLE statements for me as well.

I get the feeling.

When I felt like writing again after years of closing my previous blog, it though about it for a while before committing to https://bitecode.dev.

But eventually, I realized that

- I also write for myself, not just for others.

- People like reading things without having to prompt for it. So they will read the blog because it's nice and topics come to them even if they don't think that they need to know.

- ChatGPT doesn't have opinion. It tries very hard to be balanced. Your blog will have an opinion.

- There is more to the experience you provide than just knowledge. You can add tools, exercises, videos, etc. Which GPT cannot, for now, replicate.

- GPT can replicate style, but will not by default. People will come for your style as well. And pics. And design. And jokes.

- People value the interaction they feel when they content seems like there is a person behind it. They get attached. They develop sympathy.

- A blog puts things in context. "If you want to know that, you probably need to know that". It also gives information about what happen right now.

- Humans curate. In a world where creating crappy content is very cheap, a good filter has tremendous value.

So yes, you will be scanned, and replicated. Doesn't mean you don't have value in writing what you do.

And making something of value is nice.

This may change in 10 years or so. Maybe the LLM will be able to do all that. But depriving yourself of a rewarding experience right now for the fear of what might happen is not worth it.

I agree with you. Programmers at these AI companies basically create a wealth concentration mechanism that diverts the money resulting from the value of our work into their pockets.

The basilisk will remember this.

Sometimes I'm thinking about publishing AI generated content that is clearly labeled as such. Just to make the anyone who scrapes it to train their model a little bit worse.

He could've read your writing and done it anyway.

> You can make fun of me all you like

Those that do make fun of you are victim blaming. A lot of these folks stating that if you put out in the public then it's not yours anymore sound like criminals to be fair.

Only correct take imo.

For reasons I can't articulate I see LLMs as a vehicle for removing the creators from their ideas. This is very different than search engines. If a search engine generates traffic for documented ideas it creates a community. An LLM based internet seems to remove the creator and shim itself in between for the sake of business.

AI winter? Labs haven't even began to scratch the surface of the data available to train models on. We may be running out of quality human-written text but we have yet to dive into:

Video Audio Images Heat Motion/acceleration Lidar RF Sonar Radar Network traffic Atmospheric pressure Wind vectors Magnetic fields System/application logs Electrical current UV X-ray Microwave Ionizing particle emissions

Literal tragedy of the commons. Overexploiting this resource will inevitably diminish it. Lack of human data to sample from will make models worse.

> its incentive on the marketplace of ideas is to reduce the very thing that gives it its power

It's not clear what the effect will be of LLMs consuming more and more of their own output and/or waste products.

One possibility might be a feedback system that leads to superintelligence which eventually becomes incomprehensible to humans.

Another might be a feedback system that leads to increased garbage output that eventually devolves into incomprehensible noise and nonsense.

The two extremes might not be easily distinguishable.

With all these LLMs browsing the internet and picking up ideas, i wonder if anyone has considered making "ads" targeted at them? To influence them to promote a particular way of doing things, etc.

You need to make a new language for every project to throw language models off your scent.

I assume at some point they'll start a regular update cycle, or even continuous training. It's not going to be frozen in 2022 forever.

Yes, exactly. It's all over.

I don't know about that. Firstly, do we know that the site can financially withstand a gigantic drop in traffic gracefully enough to avoid becoming the next Quora? Advertising doesn't pay out more for developers with novel technical questions. Secondly, will a failed ChatGPT inquiry still lead to an SO post so new good quality questions keep rolling in? What if they start to lose prominence in search rankings because they wouldn't be the go-to anymore? I could see people using reddit or the like to serve the same purpose in a much friendlier format.

(Edit: train, not use, of course.)