Hacker News Reader

This doesn't work at scale. Stack overflow as a platform has been handling user generated input via moderators, voting, and testing. This is fine when there are only 26.8 million coders on the planet, most of which aren't posting on stack overflow regularly. With LLM's all of a sudden there is a huge influx of mediocre content on the platform that people can't handle. Inevitably this will erode trust in the platform. When someone posts a answer I assume they actually ran the code, and can verify the result. LLM's can spit out seemingly correct code that just doesn't work.

How are they going to check for LLM usage?

I think it's way more likely that poor answers won't mention the usage of LLM's to generate the answer, while good answers aided by LLM's will more often mention it.

Punishing honesty just seems incredibly counterproductive.

Automatic detection is downright dystopian... being censored by an algorithm because it mistook my effort and work for a LLM.

Agree with the middle part - At the moment, the policy implemented by corporate is "Don't ask; don't tell". If someone says they used GPT or other AI for their answer, it's disallowed. If they try to hide the fact, there's not much the community can do to get it removed.

And while I'm not a moderator, as just a user I've flagged over 1,200 answers on Stack Overflow (and several of the smaller communities like Ask Ubuntu) that were subsequently removed. Automatic detection was never the sole criteria that was used to determine if it was AI - It's entirely possible to spot GPT content using multiple methods. I don't publicly talk about most of these, since we do have a group of users (sometimes spammers) who attempt to hide their use and make it more difficult to detect. See some of my additional notes on the topic on https://meta.stackexchange.com/a/389674/902710

> This doesn't work at scale.

See also: the Clarkesworld saga of them being bombarded with mediocre AI-generated short stories. Filtering out bad submissions has always come with the territory, but they're suddenly drowning in them with the advent of LLMs which make it trivial to churn out vaguely story-shaped text on an industrial scale. The generated content isn't good by any measure, but it's "good enough" to pass the smell test and waste a curators time before they realise it has zero actual merit, and there's so much that it becomes a sisyphean task to sort through it.

Likewise with image generation, it's now incredibly easy to churn out images that look like something a person might make to express themselves, which are actually just a loosely guided slice through a statistical model of pre-existing images, passing the smell test for "good art" despite having zero actual intent or substance. It's spam, but for culture.

This seems like a problem you can only solve with an invite or credential system. If you are an invited writer (or have some sort of literary degree) you can submit content, otherwise you gotta let people invite you. AI content is still allowed, and if you post garbage you lose your posting privileges.

Has that observably worked for Citizendium or lobste.rs, which have tried exactly that for years, though? Have they been widely recognized as superior to Wikipedia and Hacker News? Have they in fact been widely recognized?

If your answer is that Wikipedia and Hacker News still get the recognition and haven't collapsed, then I suggest that there are already examples to learn from that the same idea for Stack Exchange won't work.

Exactly, it’s not that useful if the answer I’m looking for exists on the platform but I can’t find it because of the signal vs. noise ratio. To me, usually, the context of the answer is more important than the answer text itself.

> This doesn't work at scale.

Sounds like a job for AI

Again, you can handle this by rate-limiting and standard anti-abuse measures. To elaborate: don't allow new-ish accounts to post more than one question/answer per day, don't allow allow accounts to more than one question/answer per week/month if their previous content hasn't reached a certain quality threshold of votes, and so forth.

It's entirely possible to set up the system to prevent it from being flooded by content that moderation can't handle. In fact, StackOverflow has already been largely set up that way, and this will just require just a little more tweaking of the types of existing policies that have already been in place for a long time. People attempting to flood internet forums with low-quality content or outright spam isn't anything new.

This works in theory, if people abide by it.

However, in practice this sort of approach would likely mean people who don't have anything invested (especially new users) would create multiple accounts to be able to post multiple times.

Rate limiting only works well if there's a stickiness that makes changing accounts more difficult than waiting out the rate limit.

---

While Stack Overflow was set up to handle moderation, the culture evolved to one that disdained any appearance of gate keeping, preservation of any attempt to answer, and that moderation and curation actions on a post were personal attacks on the individual who wrote it.

As tooling was taken away from community moderation and curation it became harder and harder to maintain quality. Additionally, the rule of 90-9-1 (aka The Rule of Participation Inequality - https://www.grazitti.com/blog/the-90-9-1-rule-is-over-its-ti... ) applied to people who are doing moderation and curating means that once it scales above a certain point it becomes impractical if not impossible to curate all of the incoming material.

A little more tweaking may have been possible a decade ago. However, both the culture of people asking questions and the corporate "engagement first" approach have made being a person trying to curate the material fighting against the tide.

There are 3.3k questions that have had a close vote cast that need more people to review them. There have been only 313 reviews today as I write this ( https://stackoverflow.com/review/close/stats ). And that's ignoring the countless thousands of reviews that have timed out.

    year close tasks
    2016     581,204 https://meta.stackoverflow.com/q/340815
    2017     .......
    2018     440,336 https://meta.stackoverflow.com/q/378415
    2019     318,431 https://meta.stackoverflow.com/q/392550
    2020     225,745 https://meta.stackoverflow.com/q/404558
    2021     213,104 https://meta.stackoverflow.com/q/415250
    2022      96,495 https://meta.stackoverflow.com/q/422885

A trend with community moderation is clearly visible and likely too far to be corrected with tweaking.

>>If answers are good, keep them. If they're bad, downvote them.

>This doesn't work at scale... with LLM's all of a sudden there is a huge influx of mediocre content

The GP's answer may not work at scale - however LLM detection doesn't work at all. So the only semi-workable solution is aggressive filtering and banning users who post trash (LLM or not).

Also, there's a need to think about score and trust mechanisms - the same mechanisms which can be used for filtering also provide an incentive for LLM use, is there a way to avoid that?

>When someone posts a answer I assume they actually ran the code, and can verify the result

I wish we lived in a world where this assumption wasn't naive.

Yep, and if you aggressively ban bots/LLM content, then you'll see everyone accuse and report each other for said content even if it's good content.

For example here on HN we have a rule if you see bot content you don't mention it in the thread. You report it and let the admins decide. Anything else just turns into flamewars.

And there's the problem on SO. Previously, we could do exactly that - Flag the content for a Mod to review. Now Mods are pretty much prevented from taking any action when we (the community members) and they believe it is a bot.

I saw one user yesterday post 10 lengthy, detailed answers in an hour, in 3 different programming languages. But the Mods aren't allowed by SE to consider that (or pretty much anything) to be an indicator that it's AI-generated.

> If answers are good, keep them. If they're bad, downvote them. If they're redundant or off-topic or gibberish, delete them.

Yeah, lets us keep providing free labor to help train somebody else's models, improve somebody else's infrastructure so that they can even more effectively dominate.

I think you missed the fact that you are mainly doing this to make other users find relevant answers, on a free website.

Or maybe you would want to pay for a version of stack overflow that is only curated by employed people ?

But I guess you would lose much content.

As you say, Stack overflow is free to use. ChatGPT isn't and its scraped web training corpus is not available.

So it's not symmetric

How is that any different than providing free labor to help someone learn how to build the next [X] so that they can more effectively dominate?

But more importantly banning AI generated content from Stack Overflow doesn’t solve the “problem” you’re describing.

> How is that any different than providing free labor to help someone learn how to build the next [X] so that they can more effectively dominate?

Its an absurd equivalence. A human can never "dominate" no matter how much they learn.

Training for free a closed, for-profit private enterprize infrastructure that will eventually make your preferred free collaboration platform obsolete is like the ultimate of naivete (or vested interest).

>A human can never "dominate" no matter how much they learn.

Somewhere an autocratic leader is laughing at you.

All content on Stack Overflow is licensed under Creative Commons CC BY-SA. So, in theory, the community could create a new Q&A website and import all the questions and answers from Stack Overflow. But that doesn't overcome the network effect, I admit.

It's not really an equivalence, if you remove the sensationalism, it's just a point that the free work is already being provided.

Put more succinctly I'm making two points: 1) The free work is already being done. 2) StackOverflow has been crawl-able since forever.

You'd have to address both of those problems before the concern that you're raising becomes relevant. Banning the posting of new generated content doesn't address either of those things.

Ok, it makes more sense now. But you need to consider the significant incremental benefit to the model training process when creating labels exactly where the model underperforms.

The baseline you mention is already an issue that is being debated hotly.

My comment was that humans reviewing and fixing new AI content is exactly not what should be done in the current circumstances.

If you as an answerer uses an LLM to generate content, then verify and vet it yourself based on your own knowledge before posting, I'd think it's fine.

But spamming thousands of answers an hour automatically and wanting the community to do all the work is just not sustainable I feel. It'll also kill the sense of community if half the actors are bots.

Agreed - That's the basis of my "responsible use of AI on SO" post at https://meta.stackexchange.com/a/389675/902710

> If answers are good, keep them. If they're bad, downvote them. If they're redundant or off-topic or gibberish, delete them.

In practice this isn't possible. Lots of accepted answers are bad. Often for subtle reasons! An answer with a SQL injection vulnerability might get plenty of upvotes and be accepted, but it's objectively bad (even if it answers the question).

The problem is that there is no AI that's accurately fixing answers. AI only generates mediocre answers, it doesn't have the capacity to moderate mediocre answers. Humans simply can't keep up, or don't have the acumen to pick up on the subtle inaccuracies in accepted-but-bad answers (after all, that's often why they're looking for the answer).

Even with protections like rate limiting, I'm not sure you could prevent the majority of the damage that crappy AI can accomplish. Simply paying (pennies) for proxy servers with residential IPs gets around much of that, anyway.

haha. no.

answers are not either good or not. it's not binary. there is a lot of nuance and sometimes the questions and followups contain details that can take the answers in a completely new direction.

just telling if an answer is good or not is a lot of work. sometimes it requires and expert to figure it out. when the answers are given by a human and it's a good faith effort both people in the loop benefit from it. when the BS generation is automated there is zero incentive for a human being to even look and correct the answers. what is the incentive to do so?

> when the BS generation is automated there is zero incentive for a human being to even look and correct the answers. what is the incentive to do so?

You hit a good point here. If users can't be bothered to spend time, effort, energy and cognition into answering a question, why should the readers and correctors do so?

The volume is the real problem, as you mentioned. People are worried about AI content because it’s hard to even review it or accurately detect it. I can’t remember the last time I looked at my email spam folder, even though I’m sure there are some useful messages in there, mostly because it would take too much time to go through it. If the inbox starts being filled with AI content that isn’t quite spam, it might start feeling like a chore to go through too, again mostly because of the volume. We will have to see if the existing mechanics of unsubscribe, report, downvote, etc. will be sufficient to hold back the tsunami.

My guess is the spam detection arms race will evolve into a content moderation arms race and we will end up with advanced AIs filtering and moderating content for us, to varying degrees of success and things like email, forums, and Q&A sites will become increasingly hands off.

Agreed. I have found myself sometimes asking Chat-GPT for guidance related to obscure error messages and occasionally its more useful than google or stack overflow.

When it doesn’t hallucinate, which in my case it’s most of the time.

Sure, but if you try it out, you pretty quickly realize it's a hallucination. Unfortunately the type of GPT content we're now getting on Stack Overflow and its sibling sites is mostly unvalidated GPT hallucinations.

I generally agree if the content can be kept clean. Certainly, human users will have more to contend with.

There are just so many interesting questions by having chatbots interact with StackOverflow:

What happens when AI-powered accounts are allowed to vote?

What kind of questions might AI-powered accounts be asking or be interested in?

How will the questions/answers/responses change between model revisions?

You can imagine things spiraling out of control. Perhaps there needs to be a chatbot version of StackOverflow to service all the questions that interest chatbots. :P

I guess the premise is that it is easier to identify AI-generated content, with a low hit rate (?) of correctness, than to identify correct / incorrect answers based on merit alone.

Also StackOverflow is kind of gamified (a rare case of it working IMHO) and the rules of the game don't work so well when kind of good looking content is easy to generate. SE answers are hard to verify but easy to write. If writing becomes too easy, it is a recipe for spam - as has indeed happened.