Blueprint of a distributed social network on IPFS, and its problems
beyermatthias.de
beyermatthias.de
Two things that need to be solved:
a) The structure of moderation. No moderation does not work. Been there, done that, it gets very ugly very fast. But what can be done is to make moderation more visible, and allow people to choose who they want to be moderated by.
b) Scaling. 1-100 users are fun, 100-1000 are exciting, 1000-10000 are a little nerve racking, and after 15k users, you notice the reality of how nodes and networks behave has diverged from your 'reasonable' assumptions. Around 20000 or so, I stopped maintaining it. Scaling distributed networks is much different than scaling regular networks where you can add more hardware and servers and sharding, it requires new behaviours from each node as the system grows. The software that works at the level of a million users in a decentralised system will be much different than the one that works with a billion users.
Funny enough, I've recently quit my day job and I'm focusing on the second version of it, so the memories from 2013 are pretty fresh.
If you don't like the moderation/government of your instance, you pick another.
That could be solved by a robust way of confirming that 2 identities are the same person: so all traffic for user@example.horse is also directed to user@example.golf and vice-versa.
Coming at the same sort of thing from a distributed approach (I'm thinking of Scuttlebutt): you could make it possible to subscribe to another person's followers and blocklist, delegating your moderation to someone else.
I suppose there would soon be well-known and popular options to delegate to, much like popular ad-blocking lists; but still, each person can choose which ones to use.
I don't think subscribing to moderation is particularly useful, I considered such an approach at some point but the approach fragments the network too much (IMO)
See [Against Consensus](https://www.scuttlebutt.nz/principles.html).
https://redmatrix.info/help/accounts_profiles_channels_basic...
I had a similar experience with ZeroNet where extremist groups swiftly occupied the ground and toxified the budding discussions.
So in conclusion, are the idiots keeping us from having a decent solution for a social network?
I remember being really frustrated and annoyed when I tried to sign up for Mastodon. Trying to decide what instance to sign up with by looking at a list of possible instances, where they all look the same, and knowing basically nothing about how Masto even really worked was just stupid.
Personally I think the federated model, as it has been implemented, is not a good solution. It just adds complexity and confusion.
Basically, think of the most annoying person you can imagine, give them infinite free time, and a uncontrollable urge to do EVERYTHING they can think of to make your life miserable on said social network (doxxing, abusive messaging in every way possible, creating dozens or hundreds of accounts, etc...). If you do not have facilities to deal with this, your network is garbage.
With the focus on "distributed" and "resilient" and "unblockable", you're effectively creating the perfect environment for every abusive stalker, internet creep, and troll.
Lord of the flies is not a functional governance mechanism. Moderation and control have to be designed into the network from the outset.
There's no reason I can think of that some sort of "distributed control" isn't possible, but you'd better have it planned out first. Bolting on tools for handling abusive users after-the-fact will likely be much harder.
Its a bit like the internet itself. Lots of shoddy sites, but no one needs to visit them. Search spam is the exception but let a Google handle that.
A group of people who all subscribe to each other could be a minimalist network, but how would things like conversations work?
If you want to support conversations with people you don't follow then you need to add support to your RSS feed for comments or pingbacks.
There are some obstacles (like phone number verification) that centralized systems can benefit from and that are harder to have in a distributed systems because of trust issues - but they're not really working at any scale and only thwart least persistent persons.
I wonder about the ability to set up privacy canaries to detect information leaks. Because with stalkers and the like what you are looking for is to make sure that they don't have side channel access to you. So it's not just being able to block them, but block people they associate with. And importantly have ways to detect when they've gotten shared info they aren't supposed to.
Decentralized networks (such as the web) work differently from centralized network (such as cable TV and Facebook) and have different strategies for coping with misuse. You can not moderate a permissionless network. That's literally part of the meaning.
In a decentralized network, I follow you by subscribing to your data. Conversations usually follows that data naturally, just like a mailing list is centralized but email is not.
Mostly so, but it is worth the trouble.
About 90% of emails I receive are spam. So, email is 90% garbage. The 10% left is the sinews of my life.
Fortunately, the industry found ways to cope, by contributing to mostly centralized DNSBL services such as spamhaus.
The same goes for the Web: centralized services like Google or (is there something else? ) filter out the irrelevant, dangerous, and fake (with varying degrees of success).
Moderation? Inevitable. Distributed moderation? Not so much.
The parent post is right: if you don't make it built-in, it will not be how you like.
You can say that you personally want moderation in your networks, but you have no basis to make general claims about moderation being necessary.
Gmail spam filtering is moderation, which is being argued that without which, email would be near unusable, which is true for most.
> ... get less than 1% spam,
> handled by gmail spam filtering.
It's almost like there's a pervasive force, trying to make people use gmail.
I, for one, don't trust gmail at all any more.
All of the above work by permitting the end-user to moderate to the extent he / she wishes, or not at all.
I had kill-lists for Usenet, I can DNS-blacklist websites I never want to see, I can grey-list or silently drop troublesome e-mailers. No-one makes the choice as to degree or method except me and yet it works.
[1] of course the beauty of a distributed system is that those who want to see gore can not be denied this right.
In a distributed scheme, people I "block" or even content I filter out should only happen in regards to me fetching content from the distributed network.
In particular, if the content pathways are P2P (which they should be, even if other measures in place such as relays or caching), inappropriate content ends up being reduced by a result of less peers propagating.
Once the right solution appears for the network, ideally there will be multiple clients working through the UX of using the network.
Personally the best I've seen in this space is Secure Scuttlebutt. I just wish they'd drop the write only aspect for the log, and switch pubs over to acting as relays.
A client side solution can’t stop any of that. Anyone thinking about social software needs to look at what motivated assholes like 4chan have done and think about how they’d stop the next weev, GamerGate, etc. If you don’t have a good answer all you’re doing is building weapons for them.
Also known as "please don't pull this idiot out of my killfile" problem known on the usenet for decades. Maybe you won't see the messages directly, but most probably you will see the quotes of the messages.
I think in a way those early examples might have held recognition back because a lot of the most authoritative people had learned “no big deal” back in the era when the Internet wasn't integral to our lives and harassment techniques were far more primitive and less routine (pre-SWATing, revenge porn, social network-driven harassment of people you actually known, etc.), and because so many of the people involved were affluent white men who just didn't tend to attract the kind of persistent hate campaigns which became famous years later. Kathy Sierra should have been a wakeup call but a lot of people shrugged and said “don't feed the trolls” as if that was useful.
Human scale networks are small. Our brains can only handle a few hundred people in our social circle, maybe up to 1,000.
I'd change that into companies or even governments that actively try to subvert your system.
- fake instances / relays meant to vacuum personal data for sale and profit - identity thieves, both the broad-and-shallow, and the target-a-specific person kind - governments who got a taste of being a surveillance state thanks to FB and now can't kick the habit - massive numbers of people who decide to mess with someone at once (like when Twitter or 4ch put a bullseye on some person, whether it's deserved or not) - bots. bots bots bots. - maybe your client is super tight and solid, but what if company X decides to fork your client and lures people into using it? Now they have access to each user's stuff, and all their friends? Nice.
Obviously you can't realistically defend agains all of that. But it's worth thinking how best to mitigate before launching.
Perhaps even in a way that allow multiple regimes. For example, one user/group/admin could prefer one style of moderation, while another would prefer another style.
I think a new social network should leave as much "open" as possible, so that not just the software is federated but the ideas on which it is based are also federated in a sense.
https://www.scuttlebutt.nz/faq/misc/related.html
ZeroMe – Decentralized Microblogging on ZeroNet https://news.ycombinator.com/item?id=15464156
An off-grid social network [Scuttlebutt] https://news.ycombinator.com/item?id=14050049
--
A recent discussion of building on IPFS, where a developer was honest about drawbacks:
OpenBazaar 2.0, powered by IPFS https://news.ycombinator.com/item?id=16702684
haven't solved the decentralized reputation problem and reviews can't entirely be trusted
--
I am interested in this space because first mover advantage is still kind of up in the air until something works well enough to build a network advantage.
I think "federated" is the term for its model, moreso than "decentralized" (?).
[1] https://joinmastodon.org [2] https://instances.social/list/old
(General note)
Federated is a bit different from decentralised. For example, email is federated, but not decentralised. In a federated network, if "your service providing server" is down (like gmail.com), then you don't have service, though others on other parts of the network would continue to have service. With a decent sized network, there is no "your service providing server" and other machines can take over when one fails. Decentralisation always goes along with redundancy whereas federation doesn't require redundancy.
Obviously a lot of this is very squishy and overlaps a lot even within the same system.
Look at DNS. It's nominally a centrally-rooted hierarchy, but the root is operated by consensus rather than monarchy, so how do you classify that? Then each domain can be operated by a separate organization, so essentially federated but not exactly because it's still a hierarchy. Meanwhile the recursive resolvers and caches are fully distributed -- use any of them and the results are (supposed to be) the same.
IPFS is nifty but I'm not convinced of its ability to support an intergalactic workload quite yet. I've looked at it vs. CDN for a video delivery platform and have a much more robust system built on primitives DNS -> CDN -> HTTP -> HLS segmented MP4 + static HTML site w/ SPA chunks in just a few weeks. I explored similar ideas with using encoded public keys, hashing, and side-channel crypto key distribution to support some interesting workloads. This is cool because it's a no-voodoo solution, just HTML and RSS and JS and chunked video files, and the CDN handles the details invisibly. With IPFS on DTube I've only been able to author a little data before running into queuing blocks where I was unable to continue adding content.
I have some videos on my YouTube channel talking about IPFS and giving demos in the past 1-2mo, 'iSpooge Daily'.
It's an IPFS based social network, running entirely in the browser. Posts are JSON objects of content and metadata, with links to the previous post and the author's profile. Currently updates are propagated over a single pubsub channel, so it only works for real time communication. I plan to replace this with IPNS once it is implemented in js-ipfs.
Long term, I want to implement a web of trust to facilitate search, discovery and spam control. Here's some good reading about how this might be done without central authorities: https://www.scuttlebutt.nz/stories/using-trust-in-open-netwo...
On Facebook, LinkedIn, MySpace, Twitter, Google, Github, GnuCentral, etc., I type in Waldo's name and instantly discover Waldo's profile. On Facebook it even shows me Waldo's profile before I consider discovering him.
Also, how does the system make it likely that when I search for Waldo I get Waldo and not his evil twin Odlaw?
Also, how do you incentivize adoption so that the community doesn't end up rationalizing "Error 404: Waldo not found" as the proud badge of an artisanal social network?
1. People who want to consensually maintain two-way contact with people they talk to in normal life.
2. People who want to anonymously get life details of classmates they have a crush on, ex's, etc
Building a network to support #1 while maintaining true anonymity and privacy is much more doable. Not being in group #2, I don't worry about it.
4. People who want to follow events, share events, and follow which friends are going to what events.
Anyway, still horribly incomplete. Somebody should have put this list in the requirements, and put it up for discussion. System design starts with requirements!
Denying that there is profound value in digital communication between two consenting adults who have never met in normal life is a Luddite position.
People are just saying that they have no interest in being part of a network that tracks their videos, location, actual name, phone number, text messages, behavior across the whole internet, email address, and builds a psychological profile on them in exchange to look up college buddies (and therefore have deleted their accounts).
Many of us want a network that errs to the side of too private, and it's okay if not everybody is on that network. Many of us would rather have no network as compared to one that's too public.
Facebook got its start for one reason and one reason only-- you could use it to find Waldo.
Are you saying that global discoverability is not a valuable feature of a usable social network?
I though vanity was the main driver for Facebook adoption?
The news feed (and consequently vanity, people begging for attention etc) came much later.
This was largely behind the relative failure of RSS, in my opinion, even more than the Google Reader debacle. Various friends and family still ask me how I keep informed, I show them Reeder for iPad and they immediately want to get on board; then I explain how you go around to the news sources you like and gather feeds or use a bookmarklet and their eyes glaze over.
I hope this brainstorming on a distributed/federated social network goes somewhere, but if the creators don’t even bother considering why Facebook worked in the first place, their efforts are doomed.
The SSB network is very active, take a look - https://www.scuttlebutt.nz/
While I'm a huge supporter of blockchain projects and their allies, in the case of social networks I don't see how this is going to turn out any differently than FOAF.
In this model it might be possible to assist uptake by having a class of power user who can manage their group over onto an ad free network.
Example: I have a private github account with repos I share with various friends who are not paying github users but are nonetheless now engaged in the github platform.
A "social network" can mean very different things to different people. Even among FB users you find a very wide range of primary use cases.
So if you're going to propose a new social network at least start by defining the desired characteristics of your network, and even more importantly in this day and age, talk about the kinds of threats you are going to defend against and those you won't.
Then you can explore possible technologies to see what fits best. And if you did that you might come across the DAT protocol[0] and find it's better suited than IPFS.
Characteristics I'd like to see discussed before launching into a solution:
- Is this meant for users to connect to wide audiences (like Twitter, Reddit), or is it meant for me to keep up with my known friends/acquaintances privately?
- what features are you going for? Groups? Events? Chat? Just basic timeline?
- What's the visibility model? If a friend replies to your post, does that mean friends of that friend see your post? [there are hundreds of questions similar to this one. laying out some general principles before starting on the solution seems like a good idea]
- is end-to-end encryption supported? How do you limit the distribution of private information when many of these decentralized protocols assume that knowing an unguessable url is enough to get the contents at the url.
Anyways sorry if this comes off as grumpy.
Everything is encrypted, including metadata and we try not to expose your follow list to the network. It's inherently multi-device, independent of DNS and even has streaming E2E encrypted videos playable in the browser (streaming is disabled by default at the moment because it only works in Chrome) and even hides file metadata like size, name and directory structure. It's still pre-alpha though.
What we seem to need is a solution that does what Facebook does right, which corresponds to the networking needs of billions of people, combined with a transparent and ethical content policy approach. Plus: that doesn't leak data at an individual level (except after a court order), to name some of the requirements that seem most important.
On the other hand, a non-corruptible utility-type service like this might be too much to hope for as well ... ?
I think what will eventually happen is that the traditional telco providers will gradually improve their service quality, prices and infrastructure and eliminate the need for parasitic communication platforms like WhatsApp and Telegram by introducing standardised and provider agnostic solutions. For example why not extend the protocol that is used to route phone calls to a protocol that allows to route arbitrary packet or streaming data transfers.
Yeah well, finding anyone is not a flaw; it's a one of Facebooks killer features, without which it would not have become the de facto global registry of single individuals.
> "The only advantage services like WhatsApp and Facebook have that it is really cheap"
I believe this is a gross misunderstanding of the utility of social networks compared to older systems like telephony. But granted, the brave new world comes with a number of problems as well.
Who can at any time reach out to another average person on the other side of the globe, be it acquaintance or stranger, and discuss matters of mutual interest, without jumping though hoops or even knowing how to google, and (because of the real name policy) be reasonably safe from harassment.
But hey, to each his or her own.
Then build a system that supports both. Let people opt-in to a lookup directory (phone-book), but make it anonymous by default.
That's going to take a long time. Maybe 50 years? Working in the telecom business has taught me that federation has pretty ugly failure modes. The result of the structure of the telecom business has been the SIP "standard" and the horrible abuses of SS7 before that. The horrible reality of telecom is that most work is integration work. New features or similar is a secondary priority. Did you know that telecom providers actually test every single new phone on their network and add workarounds? It's the Tower of Babylon, except made from technical debt and divergence from expected behavior.
I've been thinking about this a lot, and I've arrived at an explanation: Each telecom provider is large enough to do their own development, switching costs are high, and vast majority of traffic is internally in a country. This situation incentivices creating new features and services that only need to work internally in a country or even just in a single network on a low budget using the path of least resistance (read:abuse of standards, bad solutions, technical debt). Most advanced functionality break down at country borders. For advanced functionality it's common to only support a small set of clients.
I'd suggest not waiting for Telco providers. They're going to become dumb pipes because of their incentives. Centralization works extremely well. Any decentralized network needs a lot of thought put into how it will avoid the traps of the telco industry.
The privacy problem is especially interesting because I work on a distributed key management system called NuCypher. We use proxy re-encryption to do this and have a distributed network of nodes that perform the actual re-encryption on split keys.
If anyone is reading this article and wants to work on this problem, we have a split-key threshold PRE scheme called Umbral that we used to decentralize the key management part. The link is below:
Filter bubbles, propaganda and competition for users' attention with clickbait and other cheap stories might even become worse, since a centralized platform could at least tweak their recommendations algorithms to break the bubbles (which Facebook doesn't seem to do, but it could).
Basically, I'm pessimistic because a distributed network only solves the (comparatively) easy technical challenge without addressing the problems of social networks.
https://pol.is/home presents a more viable option. But people self-grouping (eg. vile people only seeing other vile people) could also create echo-chambers.
For example, as a libertarian I’d love to see comments by anyone with “skin in the game” - regardless of political bent. How could we achieve that?
Now if only we could have the requirements posted and discussed first :)
I see something on the Internets which I think my wife would like, but I also want her/our friends to see it and I post it on her wall/timeline. I don't want to post it on my feed, that won't help, and I don't want to private message it to her, that also won't help.
Any of these social networks add this feature - then I will champion them to the end of the Internet.
And perhaps add support for events. But that can be an adjacent but compatible application.
A year ago, I started making this prototype: https://www.dearnation.org/ "Dear Nation is a living archive of the letters we write to our nation's leaders." I've been torn about pursuing this concept. I think I'm particularly scared of how messy it is to moderate and foster democracy online. I'm hoping that by working with tools like IPFS it will inspire me to give more free time to ideas like this.