> 1. Would the system discussed have anyway to diminish "cancel culture" like attacks?
I would differentiate between "cancel culture", where folks gang up and react disproportionately to a misdeed - versus a smear campaign where people try to weaponise the moderation system by spreading malicious reputation (e.g. your ESB example).
The idea is that both could be solved by having an open ecosystem of relative reputation lists. If a trusted authority (e.g. iwf.org.uk) were to publish a vetted list of hashes of known bad content that you definitely don't want on your server, then you'd obviously expect them to review and verify entries to that list. If they got sloppy and let randoms try to smear each other by submitting ESB-haters to the list, then there would be a scandal and people would stop trusting that reputation list. Alternatively, if ESB-fans started framing ESB-haters with child abuse imagery, then obviously that's a problem for the cops rather than the moderation list verifiers.
Alternatively, if some random vigilante decided to start publishing up a reputation feed of people they happen to think are are child abusers, and then started to stuff it with other randoms they happen not to like (whether that's ESB-haters), then the hope is that armed with appropriate visibility on the filters which they are applying, people would spot that this is a disreputable source of reputation asap and run a mile. For instance, you might have accidentally subscribed to the #child-abuse-vigilante reputation feed, and your Matrix client might say "btw, 98% of the rooms in this community have been filtered out by the #child-abuse-vigilante feed data". You might click on the link to check the names of the rooms which have disappeared, and if they turn out actually to have names like "ESB was overrated" then you might choose to dig further and then ring the alarm that #child-abuse-vigilante is not to be trusted, and unsub. You could also publish a reputation feed for reputation feeds (no, really).
> 2. Is there a way we could test this outside of Matrix?
Absolutely. There is nothing Matrix specific here, or even decentralisation-specific. We're just trying to prove it in Matrix because we have a relatively large community, containing a fairly representative mix of different actors, but we're not so big as to make it hard to experiment - and because this is frankly an existential threat to the long-term success of the project :)
The room for discussing this is #matrix-reputation:matrix.org (although public convo is a bit sporadic; this should change as we spin up more experiments in the coming weeks/months however).