Possible routes for distributed anti-abuse systems
dustycloud.org
dustycloud.org
Federated reputation is the right idea, but it's only a means. The real hard problem is distributing the mental / emotional labor of moderation. I don't think this decentralizes well. Good moderation requires focus, a carefully trained subjective sense, along pragmatic acquired experience for how to act on issues that come up. If you agree that the most efficient/appropriate route is communities having human moderators, then that puts a ton of constraints on any solution, and it begins to resemble the forum / IRC op / reddit moderator structures we've seen before.
It's pretty simple to create a federated platform with a strong anti-abuse component, but the solution is not what most proponents of decentralized platforms want to hear.
You need:
* definitive scopes in which certain moderators have a ton of (possible unauditable!) power,
* consensus on which moderators exist over which scopes,
* encrypted group channels for communicating with moderators,
* and, the most important part: a tremendous amount of dedicated human elbow-grease and emotional labor.
Consider SMTP, which is a true peer-to-peer system. In SMTP, abuse is a solved problem - spam filters work great, and 33mail/mailcatch (disposable email) improves things tremendously. At most 1-2 recruiters/SAAS offering marketing automation/etc slip through my filters each week, forcing me to expend about 3-5 seconds/week clicking "report spam".
The "abuse" problem on twitter is similarly not really a filtering issue; if it were, then it could be easily solved by a moderately better twitter client or switching off notifications of @mentions.
No, the actual problem people face is that they want to stop others from speaking, and label the speech they wish to stop "abuse". What's scary to most (including me) about Twitter is that an angry mob may form and decide they hate me, and then use Twitter to coordinate real life actions against me.
That's not a problem of abuse on the platform; that's a problem that the platform enables people to speak among themselves in ways I dislike. Lets be clear that this is the problem some folks wish to solve. It's only by identifying the problem that we can properly solve it, or decide if it's a problem that should be solved.
There's multiple things going on here. We've seen use of communications channel A to organise harrasment on channel B - which is hard for B to stop ("brigading", gamergate). Gamergate escalated into real-life death threats against its targets. We've seen on-platform organisation of harrasment of people on and off the platform (this is ultimately what got Milo banned from twitter). We've also seen "news" organisations get into finger-pointing and condemnation of random citizens (e.g. Daily Mail outing a teacher as trans, resulting in her suicide).
We've seen the legal system get involved clumsily. There's seemingly no middle ground between "joke in poor taste" and "arrested for bomb threats" (#twitterjoketrial). Occasionally people resort to libel law, and even more occasionally the less powerful, famous and obnoxious person wins (Katie Hopkins vs. Jack Monroe).
People are I hope aware of the ability of abuse reporting systems to themselves become channels of abuse. Facebook's real names policy is exploited by anti-trans campaigners to force people off the platform, for example. Anyone should be aware of the risks of automated threshold systems being abused: if all you have to do is get 100 accounts to press "report abuse", that will be abused very quickly.
Local standards also present problems. Do we really want to go along with e.g. Pakistan arresting people for blasphemy?
No it didn't. It really didn't. No more than Twitter normally is. 'I hope you die' and 'I hope you get shot' aren't death threats, they're horrible things to say. There's a difference.
(mind you, the whole question of #twitterjoketrial was whether "Crap! Robin Hood airport is closed. You've got a week and a bit to get your shit together otherwise I'm blowing the airport sky high!!" constituted an offence against the Communications Act 2003 by being of a "menacing" character.)
That should tell you what was going on, especially when what set it off was an abuse victim warning that his abuser was capable of anything for self aggrandizement.
Candace Owens / Social Autopsy found that out when she started getting threatening and hateful emails from "gamers" on an address she'd only given out to progressive "anti abuse" activists.
Local standards also present problems. Do we really want to go along with e.g. Pakistan arresting people for blasphemy?
It's definitely true that anti-abuse systems can be themselves abused, though most of the systems that you're talking about are partly due to the anti-abuse systems being centralized, right? I also see a lot of comments here along the lines of "but that's censorship!" But the article is discussing decentralized anti-abuse systems which allow individuals to set up their own opt-in filters which apply to themselves and their communities (which means different people might have different filters). Do you think that's different?
The situation where A is sending to B something that's harmful to C cannot be dealt with by C's filtering and can only be addressed at a higher level in the system.
Those are the technical distinctions, but there's a lot of possible things covered by the second case: leaked nudes, lynch mob organisation, slander, leaked intelligence, compromised party documents, names of human rights activists being leaked to secret police, copyright infringement, child porn, fake news, real news in fake states, allegations that invitations to pizza are evidence of child porn, and so on.
Or various right wing ideas now softly suppressed on Twitter/Reddit.
Huh? If C knows A's public key and the content is signed, why can't C filter A's content?
The problem does have a way to solve itself, but that probably involves the model we see here on HN, which admittedly is preferable to the model of a locale such as Reddit. That said, each platform is going to want to address the problem on a faster timeline, because the problem directly hits their bottom line.
I agree that the technology exists (and plenty of other solutions mentioned in the article). But saying that because the technology exists and hasn't been deployed therefore there is no abuse problem is a hilariously bad non sequitir.
I linked your comment and slides from the blogpost now, btw!
Closed doesn't have to mean secret, just that it uses a whitelist for who can post.
Slap on some access controls to my idea here and you might have a decent start;
https://roamingaroundatrandom.wordpress.com/2014/06/01/a-dec...
(I think I've even mentioned this to one of you guys before)
In the "web-of-moderators" model no content would be outright blocked. Each user would have a personalized view, depending on his personal moderator network. One side effect of this would be that trying to outsmart the moderation would be harder, since all the bans would be shadow bans. Spammer would not know from how many people in his network the content is hidden.
This solves elegantly one problem hard with moderation: allowing freedom of speech while giving a safe place for people to be.
You can imagine a forum where you have official moderators, and non official moderators. By default the forum display the very clean, politically correct moderated content. However, if you create an account you can switch your moderation source, and see more content.
People ok with dirty jokes or extrem opinions would be able to enjoy the site as well as children.
That's actually a fantastic idea even outside of the web-of-moderator concept, just as a implementation in a regular centralized software. Even better in a global ring of course, but one is harder than the other.
But yes, I often wish I could see deleted comments on reddit in places like /r/askscience: I want to see what stupid shit people have said to get banned.
ALTHOUGH, like everything discussed in any of these threads as 'solutions' to the so-called 'problem' of 'abuse': it really is just going to result in more echo chambers.
And they are sure to be right, as much as we are sure we are right.
That's the problem with freedom of speech.
One of the solution is to allow:
- anything that is legal geographically - anything that is not followed by destructive actions IRL - or both
And even that is hard and very imperfect.
Morality is always arbitrary, it's impossible to draw a perfect line in the sand.
Which is why systems life freenet are both a fantastic place (you got everything man, even the craziest informations) and a terrible curse (you got everything man, even a 10 years old getting raped).
The web-of-moderator is an elegant solution to this problem. You don't draw one line in the sand, you draw many and let people choose a line.
This doesn't solve the moral or legal issue, but it does solve the "i'm offended" issue. Which is more that what we could do up until here.
>This doesn't solve the moral or legal issue, but it does solve the "i'm offended" issue. Which is more that what we could do up until here.
I entirely disagree. It does not solve that issue at all. The issue there is that people feel they have the right to filter out everything they disagree with or find uncomfortable. They don't.
The TV news doesn't avoid showing images that are graphic, they say 'graphic images upcoming, turn off if you can't handle it' then show them.
Yes but this is a way harder battle than creating a web-of-moderators :)
The "problem" that the whole distributed paradigm brings to communities is the same as its greatest strength: if nobody controls the system, then anyone has just as much influence as their peers. This means that systems will have to be put in place to give some power over others in order to control the community.
This is where I believe the crux of the issue lies: in the distribution of power.
If anyone can decrease someone's federated reputation, then so can the trolls.
If you require people to "join" a community, whether by participation or time or other means, trolls will do that as well if it is easy to do so.
If I were trying to create a community of people who are at a higher risk of being harassed I would have my community structure take some cues from HN, Reddit, and the like.
One of the things I love about HN is that you need a certain amount of rep to downvote. This alone eliminates a lot of the issues Reddit has where everything turns into mob rule as everyone, including community outsiders, have the ability to influence discourse. Going one step further, I would propose an invite system. While an invite system would mean slower growth initially, we're trying to optimize for quality in the community, not quantity.
I had some alarms go off in my mind while reading this article's seemingly nonchalant approach to what is effectively censorship, but in a distributed system I guess this is of no concern as everyone can just "fork" the community and start their own.
As time goes on I'm beginning think that no matter what the original design of the platform was they will all inevitably regress into something like IRC.
The rules will be changed so that groups can split of or filter and focus on some specific area to try to keep the discussion on topic and productive. And inevitably there will be some group that captures control of that group and turns it into a dictatorship.
Dissenters will start a new group if they can or leave the platform. Any other design will just become so noisy and unproductive that it turns into a dumpster fire.
The reason why HN is not a total shit show is because they sometimes put down the hammer and knock out off topic discussion from the core set of topics that revolve around the practice of programming.
> Going one step further, I would propose an invite system.
Some of the most productive people who make the best posts are hermits. There people who rarely check in to social media to see whats going on and have little or no friends they could ask for a invite.
And on the other hand some of the most crap people are online all the time and have huge networks and can get an invite to where ever they want.
It differs from Eigentrust and Eigentrust++ in that peer nodes don't manually propogate their trust ratings to eachother on their own behalf, nodes just use their own view of their overlay networks when it's time to recompute trust ratings.
Simulation framework with threat models: https://github.com/Psybernetics/Trust-Toolkit
Peer-to-peer caching proxy PoC: https://github.com/Psybernetics/Synchrony
There is probably still more work to do and this (IMO) does not scale to Layer 8 that easily.
When all your email filter is doing is filtering things as 'spam' 'not spam', it's fine to have a binary file. But what should be moderated on a forum is a completely different kettle of fish. I want Twitter completely unmoderated (except for literally illegal content). Some other people, clearly, want Twitter restricted to only things they agree with. We should both get what we want.