Hacker News Reader

Comment by ericbarrett | Hacker News Reader

I have moderated large internet spaces. There are people with severe mental issues who, when confronted with direct moderation, will make it their sole purpose in life to ruin the forum. I'm talking 20 hours a day of spam, ban evasion, doxxing, DDoSes, direct messaging other members, and whatever else an intelligent-but-disturbed human can come up with, solely dedicated to ruining the space the moderators are trying to protect.

Shadowbanning is like a dodge—instead of head-butting such an explosive personality, you let them slide by and rant on into the ether. It can be, and is, abused, but it is also extremely effective and time-efficient when dealing with "the weirdos." (Generally they will go away after some time not getting any engagement, positive or negative.) That can make or break a volunteer force, and if it's a paid moderation team the cost reduction is real.

View on HN

Just an anecdote from Second Life, there's one person that keeps coming around despite being permanently banned and more or less not liked by most folks in one hub. He's made it his mission to dox one person who didn't actually do anything to them other than not hang with them in games like Red Dead Redemption and the like after they had one bad night. And he doesn't just do it this person but to several and all of them women. I think shadowbanning wouldn't work here but I wonder if it would mitigate the spread of such doxing if it were tried.

Sounds to me, a better approach would be transparent but automated moderation.

Automated moderation would need some way to reliably answer the following questions:

- are two accounts controlled by the same user?

- are two posts talking about the same topic?

Then an approach would be to publicly issue a "content ban": User X is prohibited from talking about topic Y. Then use bots to enforce the ban and identity sock puppets or alt accounts if the user tried to evade the ban.

I think with the current progress in language models, we have a much better chance to answer those questions with usable accuracy.

I've seen this tried before. My thoughts:

First, no language model (or human judgement) will ever be perfect. Therefore you will eventually have a false positive, and a respectful contributor will be banned.

Now you must either:

a) Provide an appeals process—which takes time and effort, and will surely be abused to the maximum extent possible by the maniacally aggrieved.

b) Or, if the judgement is final, you have implemented undisputable algorithmic perma-bans (Google's much-hated approach).

So you're not really solving the original issue, which is the amount of time and resources that disturbed individuals can consume when confronted.

Second, this approach does nothing to prevent external issues like legal harassment, doxxing, real-life staking, and so forth.

There should be laws that allow to sue and effectively punish those things.

Yes, it could be abused by authorities, but it should be possible to sue someone if there are enough proof.

Big online platforms might also be able to clean a lot of trolls too.

Probably, people with mental health issues that spam fora with nonsense fifteen hours a day are judgement-proof. They have no assets to attach that would justify the effort of legal action against them.

I don’t think that juice is worth the squeeze.

Besides, imagine whatever laws you think would help here and now imagine a toxic actor making it their life mission to provoke everyone on your forum but without breaking your imagined rules. You’re back where you started but now you have overreaching laws.

Reply on news.ycombinator.com