Trust-based moderation systems
cblgh.org
cblgh.org
To find the most trusted peers, we use Appleseed, a peer-reviewed algorithm and trust metric which was proposed in the mid 2000s by Cai-Nicolas Ziegler and Georg Lausen from the University of Freiburg. Appleseed operates on precisely the kind of weighted graph structure we have described, and it is also guaranteed to converge after a variable (but finite) number of iterations. Appleseed, once converged, produces a ranking of the most trusted peers.
Appleseed produces its rankings by effectively releasing a predefined amount of energy at the trust source, or starting node, and letting that energy flow through the trust relations. The energy pools up in the nodes of the graph, with more energy remaining with the more trusted peers. After the computation has converged, each peer has captured a portion of the initial energy, where the peer with the most energy is regarded as the most trusted.
ie: Can I become the most trusted for that one glorious moment of ripping the table cloth out from under everybody?
Then raise the stakes more and consider the US presidential candidate nominations process.
Push it to extreme situations, and finally in the end compare it to the Chinese social scoring system.
This kind of journey of morals is what ultimately led to me transferring ownership of my own online community and walking away for good. It’s tough and I don’t envy the folks who keep up with these things.
If there was any trust left here, the upcoming election cycle should have already destroyed it. The likely Republican candidate refuses to debate, the Democrats all seem to think the president shouldn't run again but won't stop him or allow a challenger, in all likelihood the two final candidates also won't debate each other, and we're left picking between two candidates that much of the country likely aren't happy with. That's neither democratic or trustworthy.
The Democrats effectively aren't running a primary at all. There are technically two challengers[1], though there are no debates planned [2]. RFK was running on the ticket and had plenty of support in polls to warrant primary debates, though he's now running as an independent after what he viewed as a refusal by the party to allow a primary process to challenge Biden (his opinion, I can't vouch for inner workings of the DNC).
[1] https://www.reuters.com/world/us/democratic-candidates-runni...
[2] https://deadline.com/feature/2024-presidential-debates-sched...
As a non-American, who's RFK? Not the long-dead Robert F Kennedy, of course, but my ability to search that acronym pretty much only brings him up
Honestly I never really think to refer to him as a Junior, his dad was assassinated shortly before I was born, but I should have not only avoided using initials I should have also included the "Jr"
Third parties can not gain traction. (Hence all the effort spent on co-opting and riding one of the existing parties like a zombie.)
Third parties effectively exists as free research for what positions will be popular next cycle
A lot needs to be reformed with regards to voting and election systems in the US but to claim it is impossible for a third party to win is factually incorrect. How else would independent politicians like Bernie Sanders or third party candidates like Abraham Lincoln would have ever been elected?
Who wins is often not determined on election day, but in coalition making after the election. If you have 5 - 10 parties, it's often not immediately apparent on election night who can gather a majority behind them.
The parties in the US have changed around something like 5 or 6 times. In the last 2 or 3 changes the party names didn't change but the parties did fundamentally change. After each shakeup, we've ended up with a two party system that's structurally unworkable for independents to win at the highest level.
How much longer they can keep this illusion running has got to be THE most interesting thing happening at the moment if you ask me.
When push comes to shove the UN has very little power here and is at risk of going the way of the League of Nations if it pushes too hard without having any meaningful enforcement mechanisms.
It would better prove out that the UN is also a psychological operation to a non-trivial degree.
> Would the UN kick Israel out if they continue the war despite a UNSC vote?
The US is the military muscle of the UN, and as far as I can tell, Israel largely controls the US, at least with respect to Israel. Who Epstein worked for, and who has has on tape seems like a relevant aspect of this issue, that hasn't gotten much traction even on TikTok.
> When push comes to shove the UN has very little power here...
The UN, or at least the idea of the UN that has been distributed among the minds of people in Western countries, has massive power. Control people's beliefs and you can control the world, and the US and Israel are both masters at that game. For evidence, I present the conversations that take place on social media, including HN.
> ...and is at risk of going the way of the League of Nations if it pushes too hard without having any meaningful enforcement mechanisms.
Dare to dream! I can't think of any "reasonably" likely scenario how that could come about, but maybe I lack imagination.
I do however think the decades long psy op that Israel and the US have had going in this specific region is at serious risk though. However, I do not underestimate the "Public Relations / Journalism / etc" skills of who we're dealing with - I think one well designed "event" could easily put most people right back into their trance.
Its definitely unlikely in the near future, though it will die eventually and likely will happen quickly. I don't see many ways that happens soon, though depending how serious UN member states actually consider the war in Gaza, they could walk away seeing either how useless the UN is when one state can veto or when it becomes clear that a UN without teeth is only useful during times of peace.
I also don't expect the UN to dissolve anytime soon, but I can't really put any reasoning behind that other than it feeling crazy to think the UN could be gone tomorrow or 6 months from now.
Presumably, if enough nations take real issue with Israel's war they could get frustrated enough with political roadblocks and general lack of accountability and enforcement that they decide to walk away. If a few walk it may not take much for it to quietly slip away.
Unlikely. Is there even a process for the UN to kick members out? What good would it do anyway? At most, they get demoted to a non-voting member, but probably the US and UK won't both vote against them in the security council, so do they really need a general assembly vote?
> Would countries go to war with Israel specifically to uphold the UNSC vote?
No, but they might enact a no-fly zone, or do a peace keeping mission with lots of possible rules of engagement.
But the UN is really a means for the powerful countries to justify their extraterritorial actions, when there's enough concensus and the other powers don't care enough to turn away the rubber stamp. It does some other important stuff, but if the UN had significant power in the Israel/Palestine conflict, there's a series of adopted resolutions that could have been de facto enforced.
I'm not sure how that would work here. Given that they are primarily Israeli jets in the air, UN member states would need to be willing to shoot down Israeli jets to enforce the no-fly zone.
> if the UN had significant power in the Israel/Palestine conflict, there's a series of adopted resolutions that could have been de facto enforced
Now you got me curious. What are those existing resolutions they could reach for? Is it mainly just enforcement related to economic sanctions and similar non-military actions?
Sure, but enforcing a no-fly zone isn't going to war. And I'm sure they'd shoot down Hamas jets as well, so it's fair.
> What are those existing resolutions they could reach for?
There's a big list of resolutions [1], resolution 54 [2] from the security council in 1948 ordering everyone to desist from further military action is probably technically still in force and applicable.
In the general assembly, you've got resolution 181 [3] which sets a partition plan for Mandatory Palestine of 1947 which didn't happen. And there's many years of annual calls for Israel to withdrawl from the occupied territories, etc, between about 1967 and about 1984. Unfortunately, many of the Wikipedia links to general council resolutions don't currently work, so I'm relying on titles.
Anyway, if the UN orders a partition in 1947, and it hasn't happened by 2023, but they still say oh hey, do this thing... either the UN doesn't have much power or it isn't willing to use it.
[1] https://en.m.wikipedia.org/wiki/List_of_United_Nations_resol...
[2] https://en.m.wikipedia.org/wiki/United_Nations_Security_Coun...
[3] https://en.m.wikipedia.org/wiki/United_Nations_Partition_Pla...
Of course, that's not doable in a typical chat room setting.
* Profiles don't have to be tied to real identities like on Facebook. Screen-names are fine as long as people know who's who, like on AIM.
Old Facebook already did it. Twitter could be it if there are no recommended / globally popular accounts and you just repost stuff you like. Both found it more profitable to shove stuff at people and create popularity contests.
LMAO. Bernie Madoff got started by roping in friends and family. Friends and family are literally the biggest targets in every MLM scheme. Family is frequently implicated in identity theft (they have access to your files). Gen-Z has figured out that they can scam their families and cry gay persecution from them to avoid accountability. The vast majority of rape cases (70-90%) are acquaintance rape.
Real acquaintances are the most likely to be victimized in literally every form of crime, including murder. Trusted networks only function when people have decency and don't exploit their insider access.
> That's why OG Facebook without groups, pages, global recommendations, etc was actually very civil.
I'd argue it's because the networks were smaller, and no other reason. When you have a large family, you can afford to fuck over a few members and live with never being invited to Thanksgiving at their house again.
Large network isn't an issue in of itself. Most people indirectly know tons of people. Overly connected is the issue, cause it incentivizes bad things for popularity. Nobody directly knows like 1M people IRL. Even worse if there are no real-life consequences. Maybe we're saying the same thing.
That's an interesting angle. Trust is the fine line between friend and stranger though. The stats would be different because "acquaintance" no longer has meaning.
> Overly connected is the issue, cause it incentivizes bad things for popularity.
Yeah, I thought about it some more and shit really went downhill once FB replaced the timeline with the algorithm. That surfaced the worst behavior for attention. You're right about extended networks.
Obviously not proof of anything cause it's only a model, but I agree with the theory. Yes FB did go down the toilet with the algo feed, maybe Twitter too. So this is how I judge social networks now.
So in other words, all you need to do to game the system is generate a bunch of accounts that publicly link to you, follow you, or whatever this systems' input of 'User A trusts User B' is. And not get caught by whatever anti-spam or anti-bot model they're using.
As long as it can be guaranteed that one specific individual can only ever create one account, it's not a problem at all.
Which would seriously reduce the number of people willing or able to use the web.
The only way I can think of to try to avoid this is to simply make it more lucrative to continue with good behavior rather than liquidate the trust you’ve built up.
Also, respond very quickly to recent trust issues, but slowly to long term trust issues. So any trust violation gets punished quickly & heavily, but has little lasting effect if turns out to be an isolated issue.
It’s an interesting problem. I don’t see how we can keep scaling and increasing the value of social connections (high trust in useful networks has almost unlimited potential vslue) without automating & decentralizing the quality maintenance.
The appearence of multiple relatively new users all pushing extreme takes that are on the edge or well over gets shorthanded as an edge lord raid - it's coordinated and can have multiple root causes from simple social drama to actual distraction with an ulterior motive (eg: prompting moderators into rushed counter reaction can expose passwords | permissions | inter operator side channel communications, etc).
Slow trolls like the drama but dislike the boot .. they'll wheedle their way into communities and build a support network then pivot into being "that uncle at thanksgiving" .. you know the one.
These are very rough descriptions, it's very much the case that you might not be able to pin particular behavious down but you'll know 'em when you see 'em.
If you can create identities at zero or modest cost, no majority-vote scheme will work.
Amusingly, what does work is Second Life. Space keeps everything from being in the same place. You can shout at most 100 meters, and the 3D world is the size of Los Angeles. There's no broadcast system built in. Jerks are a local problem. Local landowners can kick people off their land. Spam consists of buying small land parcels and putting up billboards, and is rarely profitable. Influencers have small circles of influence. Everything is local.
If it's hard for one person to reach large numbers of people at low cost, moderation becomes far less of a problem. This is alien to the concept of social networks of course. It does raise the question, do you need to give everybody a bullhorn?
[1] https://trog.qgl.org/20081217/the-why-your-anti-spam-idea-wo...
Sounds like subreddit moderators to me? Still builds on human labour, this time unpaid and from community members.
Also I don't think there's much hope for one-size-fits-all solutions to trust tracking. Some applications are slow and iterative, like the evolution of reputation in communities, and they must allow for redemption. Others are critical, where even a single, brief defection would be a disaster. I guess this one is aimed at social media chat.
Of course, dang based moderation also works well but you need a dang for that.
I can imagine this being reduced by at least a factor of 10 without much impact on the trust score. But note the “random subset” means different people will have different trust scores for the same peer :shrug:
For (2) yeah we probably need to lower our expectations on privacy for the time being; it’s a masters thesis and privacy in open distributed systems is very tricky.
If we do care about privacy, any moderation system should first be designed to meet that goal. Its not worth designing a moderation system if we don't first know it will work with one of the core requirements.
I am trying to innovate on moderation systems and I run/code a whitelist moderated forum [0]. You can only see posts and comments from users that you follow. It's a very simple system and there really aren't any gaming vectors. One implication is that if a new user signs up and posts, no one will see it unless they follow. I've actually never used any typical censorship moderation.
(I think your comment is on topic for a post about a moderation system.)
I don't get it. How do you even find users to follow if you can't see their posts or comments?
You can't. Inevitably, the forum is slowly taken over by some self appointed dictator and cohorts and the more saner voiced are driven out.
[1] https://link.springer.com/article/10.1007/s10796-005-4807-3
Explicit definitions of malicious behavior in the forum guidelines may or may not be enforced if the forum is controlled by interests seeking to covertly amplify certain narratives while suppressing others, even if those narratives do not explicitly conflict with site guidelines.
One plausible approach to this situation is to use a LLM agent as the forum moderator - one which only uses a publicly-available explicit set of moderation rules to flag comments and submissions. Something like this is almost certainly being used at Youtube, X, etc., with the caveat that the rules being used are mostly hidden from the public (e.g. X feeds don't seem to have much interest in amplifying stories about UAW's efforts to unionize Tesla, etc.).
This could lead to a regulatory approach to social media in which the moderation rules being fed to the LLM must be made publicly available.
It seems here you can only "trust" someone into being a moderator, and then they have to do this part.
In practice most central systems don't explicitly charge money for accounts, but instead require verification of something that would make it inconvenient to register large numbers of accounts all at once. For example, if you want a Gmail account, you need to verify your phone number with SMS. Phone numbers cost money to obtain, which means that you can distrust ones used to create spam accounts and the spammers actually lose something.
This is also why Fediverse moderation puts so much emphasis on defederating instances rather than banning individual accounts. In the Identica/OStatus era of the Fediverse, defederation was actually very controversial! But here in the Mastodon era, the only way to actually punish bad instance operators (and there are plenty of them) is to defederate their instance. This works because instances are referred to by domain name, and DNS is a centralized[0] system that costs money to register, so you can distrust a domain and actually cost the abuser money.
[0] The distribution of domain records is decentralized, and you can delegate subdomains forever, but you have to have a chain of delegation leading back to the root servers. Top level delegations cost lots of money, second-level delegations less so, and subdomain delegations are basically not worth anything and can be distrusted with wildcards on the first private zone in the domain (e.g. ban .evil.co.uk, .evil.net, etc).