So, should we just try to ignore the larger-scale exploitation of trust?
But I think there's a risk of sending a lot of bright, productive people off to tilt at technological/interface/regulatory windmills in a way that leaves fundamental trust issues out of scope.
Even if Amazon, Twitter, Facebook, etc. could wave a magic wand and remove every fake review, counterfeit product, scam, personal threat, or piece of false/fraudulent media, there would still be a trust issue. We still have to trust that they did what they said, that whatever disappeared was correctly identified, and that they didn't wrongly remove many legitimate items in the process.
Even if the magic wand that makes these decisions has the utmost ethical and logical integrity, there will still be some mix of skeptics, cynics, malign actors, competitors, bots, etc. who chum the waters with accusations to the contrary. We'd still have to choose whether or not to trust this process and the actors behind it.
So, I think it might be productive to focus on some smaller questions first. To be semi-arbitrary: can we find a protocol for reliably building a 50-person trust network with a limited scope/focus (maybe identifying reliable providers of a single service), where each participant knows a small fraction of the network, which is capable of meeting both its purpose and is capable of detecting and reforming or ejecting exploiters? If the first is tractable, can you expand the scope/focus of the networks and retain these properties? Can you compose a higher-order network that retains these properties?
These kind of emergent niche communities can only exist when the broader network gets sufficient scale to have a critical mass in the niche.
So in limiting scale overall let’s take care not to throw out the baby etc.
I'm imagining something more along the lines of: can we learn how to turn what small/niche networks do well into building blocks, and does that knowledge teach us anything we could use to (I guess adversarially) re-structure/reform some environments that are currently trust bonfires.
References? Credit Score?
Both of those are easily gamed in the absence of trust. For references the mechanism is obvious. They list co-conspirators as references who then lie about their titles.
For credit reporting it seems like you could do the same thing over time. Create fictitious entities to make positive filings with credit reporting agencies. I assume the reason that isn't more popular is that it's even faster to do it the other way and steal the identity of somebody with good credit.