New questions on Stack Overflow are down 77% compared to 2022

Learning to predict what word will lead to a successful solution (rather than just looking like existing speech) may prove to be a richer dataset than SO originally was.

There are a few times on Reddit that I want to explain something that I know well. But it will be a long post.

I’ll be lazy and ask ChatGPT the question, either verify it’s correct based on what I know, ask it to verify its answer on the web - the paid version has had web search for over year - or guide it to the correct answer if I notice something is incorrect.

Then I’ll share the conversation as the answer and tell the poster to read through the entire conversation and tell them that I didn’t just naively ask ChatGPT. It will be obvious from my chat session.

Just considering Stack Overflow for a moment, they exist to profit from their product of consolidating questions/answers. When the LLM can answer most questions more efficiently, they've lost much of their value proposition in terms of product... and perhaps their business along with it.

Many of the community forums, however, tend to not be businesses per se. Sure they'll see less traffic, but that might not matter to them. In fact, it might even be better to an extent because they often aren't monetizing their services and so LLMs carrying some weight can help reduce costs. Under those circumstances, LLMs may not be nearly so bad and they, themselves, will still have sources of training data.

For example, I read the Elixir Forums for language announcements, feature discussions, occasionally to ask questions that I can't resolve with research, and even to answer some questions. I've also got LLMs fairly well integrated into my workflow. I don't see that pattern changing: neither less Elixir Forum or less reliance on the LLM. What has changed is I don't use search as much as I use to, nor do I use Stack Overflow as much.

So I do expect the big aggregators to go away, those not tied to monetizing their knowledge transfer I expect to see less overall traffic, but not less meaningful and substantive interactions.

Historically, most of my SO usage boils down to: 1) finding how to implement something esoteric that results in finding a clever solution or a under described feature flag in a function/tool 2) finding a workaround bugfix for a broken feature in some software (>70% of the time finding link to a github issue in the description

If we consider that LLMs are functionally an information retrieval function containing natural language program subroutines. In this context, a web-browser enabled LLM should be able to determine go to source and return a functional answer on a model that is not pretrained on the source.

So as long as there is good documentation on a particular piece of software, we should theoretically be able to generalize to non-existing tools. At least long enough for there to be a newly-created training dataset from people hitting the problem for the first time.

Side note: In some sense, the foundation model labs are aggregating the Question-Answer pairs (typically from stackoverflow) from their user data. I wouldn't be surprised if they created a stackoverflow clone at some point to opensource the dataset creation and labeling efforts.

This is basically what community notes is for X and now Facebook

You can see that comments on those videos are students who just entered CS and are afraid (questioning whether they should switch majors.)

It was nice to be able to find answers on the internet while it lasted.

It does speak to the culture of editing/moderation on SO, though. For many people, myself included, it simply does not work.

Remember Monica? She was a Stack moderator who got in trouble over Stack's new Code of Conduct a few years back.

The Code required using people's pronouns if they listed them. Monica always used gender-neutral pronouns and wanted to keep doing so, but Stack took issue with this.

Pronouns are a pretty standard part of interacting with someone in English -- and so is the "thank you" in GP's Stack post. So why is one social convention mandatory while one is forbidden?

Because it's not about what's on topic and what isn't. It's about archetypes. And the highest archetype for a certain kind of programmer is the Vulcan. Be respectful, be noble, but don't "waste time" on pleasantries that are just there to convey warmth and friendliness.

The problem is that not all programmers are Vulcans, and more and more non-Vulcans have become programmers as coding has become more and more mainstream. But Stack essentially set up test cases for how Vulcan its users are. That tends to be an alienating culture for those who aren't.