The unrealized potential of federation
drewdevault.com
drewdevault.com
1. Among other things, because of a dead-simple user experience that lets even grandpa post content after a few clicks. Federation, if it adds any level of complexity past the grandpa UX, would defeat the goal.
2. Next, identity. If grandpa can post, so can the bots, how do you ensure accounts don’t go to fake entities? What if an account has to pass a quality threshold before it can publish in public, but can be used privately any time. HN has similar model, but here you limit public posting until 10 random other users have seen and upvoted the new user’s post. Add to that possible additional authentication for payment of a pro feature, or a human check mark, and you’ve put a hurdle on spam.
3. To keep culture from running amuck provide positive and negative feedback. Public content has a span problem, not so much private behavior. Content ratings and discussion licenses can come into play as well, where any group can limit posts to PG content, or discussion that gets moderated automatically or manually if it passes certain thresholds.
Happy to suggest more if needed. I run https://www.dreamlist.com
Invite only. If I'm running a federated service for me and my family, the moderation is going to be astronomically better than anything big providers are doing because a couple minutes of moderation by me every week is astronomically more on a per user basis than what someone like Google or Facebook is doing.
Drew's point is that people who can have or acquire that expertise are force multipliers — it's easy _enough_ that one person can run a node that carries less-technical users into the ecosystem. But I don't think we're really there yet, and worse the value isn't there in federated social media to offset the costs of time, money, and change.
We've seen distributed networks capture enough imagination and be usable enough to get mass adoption, but they did things like distribute tremendous proprietary media libraries on the cheap and without permission.
Services like Mastodon aren't as easy to spin up for a group as, say, running Napster was, or even seeding a torrent is now. But people, even non-techincal ones, either put up with those services' occasional unreliability or recruited more technical users to work them on their behalf because the relative value was so great.
So to me the problem with federation is making the value exceed the cost. It's a squeeze from both sides. We won't get there until an ActivityPub service is as easy to stand up as running an app on your device, AND the value of running one is large enough to make it worth the trouble.
It feels like messaging should be where the value end should have already taken off, but federated group chat is still a rat's nest of incompatible or feature-starved services. Federated social is still just on the wrong side of complexity and the value it provides honestly isn't that great to a mass audience.
Sandstorm is/was most of the way there; it makes installing server apps really easy (assuming a hosting provider installs Sandstorm for you).
Not for 99% of the users, for now. But there is an increasing faction that is taking issue with what any company does with their data.
At first, caring about one's own data was mostly a political thing. Now, abuse of data is getting so ridiculous that how much time you spend in the supermarket queue may depend on the aggregated data supermarkets have on you (and yes, supermarkets often aggregate their customer data via third-party reputation aggregator companies). Not to mention credit interest rates, insurance premiums, visas, and airport security checks. Soon people will too realize that taking their data off from ad networks is a good way to save money. At that point, one either goes silent or searches for a limited, well-protected social network.
I research in that direction: http://mro.name/ShaarliGo#install--update
Perhaps extend that one or two levels into the graph, with exponential decay. So if you invite someone, and that someone turns out to be a spammer, they get banned, you get banned, and I lose posting privileges for a week. And the person who invited me loses some privileges for a day. Etc. Add some level of manual review and algorithmic detection of clusters of accounts that exist only to invite other accounts that invite spammers, and...
... well, did anyone ever try this out? What are the obvious holes in this system that I'm missing?
It's worth not just "a shot", but lots of earnest attempts.
Another feature that I wish other places adopted is a public moderation log (https://lobste.rs/moderations) where users can see all the actions by the mod team. This helps keep the mods accountable.
If you are, while i think email has proven that moderation is much much easier at scale if the attackers are spammers (astroturfing is a bit of a different story though)
Yes, protection against a million fake accounts is the key scientific problem to solve (Sybil attack).
Our university lab has worked on this problem for 15+ years, its hard. Early indications are that ActivityPub and federation are not going to scale. Bot attacks are too ferocious for amateur admins. Big instances attract more people, however a single instance fails to scale.
Nobody knows how to build trustworthy self-organising systems with "self-governance". Somehow you need to deduce the reputation of others from their signed interaction history. Their privacy needs to be protected, but you can still calculate a reputation score. After 20 years of scientific progress around "distributed reputation systems" with 100+ scientific articles they have one thing in commons: they don't protect from the Sybil attack.
Server federation is not going to help if there is no trust fabric to build upon. The effortless onboarding plus mobile-first requirement of today means you even need to have smartphones talking to eachother directly, P2P-style. <plug> thankfully students in our lab now have carrier-grade NAT traversal operational between Android devices. Google: "Technology Stack for Decentralized Mobile Services" (quotes required).
First movers always take the majority of the market share if there are no compelling reasons to use a similar service.
'Privacy' itself is barely a feature - people do care, but it's rarely #1 on their list.
Techies viewing this from the more strategic, systems perspective have to get into the headspace of individuals to see it from a 'user centric' perspective - because the answer to many of the questions lie there.
For starters, very few people care that Facebook for example is 'centralized' - they care about the material value offered to them 'right now' which is to say, access to their friends, posts, information etc.. Everything is is a distant second. Issues that do not directly affect their experience an even more distant third.
The same seems to be pretty much true of "federation".
That rules out my children's grandparents then.
One of the big things facebook did is you couldn't customize your page. No autoplaying midis with seizure inducing gifs. Also helps with brand cohesion.
Federation gives people freedom. Quite frankly users cannot be trusted to maintain a good experience if given freedom.
You could have stopped there, that's pretty much the only reason. But that has little to do with federation though. And yeah the ugliness and bad UX problems of free software - software that's freely unusable - have been known for literal decades.
That's the point though, design for the lowest common denominator has been the modus operandi for the giants because it gets them the most profit. There is no proof that this is the way to create the most "useful" social network. Often I'm tired of the "organic spam" that users with easy access create non-stop in a lot of the communities I'm in. A technical barrier to entry isn't necessarily a bad thing in this case.
One of the reasons amoral commercial efforts like fb won is because this modus operandi happens to harmonize standard commercial MO with what a network is.
There's definitely value in alternative MOs. As you say, a technical barrier to entry can have its uses. Wikipedia is a nice example. I don't think simplifying the editor UX to an intuitive one would necessarily be useful. But wikipedia needs a throttle on editing.
I'm not sure this relates to federation though. Niche communities don't need to be federated.
No that is not true. Judging from my experience with various subreddits anything above 100k quickly becomes trash. You don't need as many users as possible.
>One of the reasons amoral commercial efforts like fb won is because this modus operandi happens to harmonize standard commercial MO with what a network is.
For what facebook wants to be i.e. a profile for all your contacts that is indeed the case. For hobby/activity related communities that's not true.
>I'm not sure this relates to federation though. Niche communities don't need to be federated.
It does relate to federation because right now if I open a new subreddit I am bound to the wishes of the company and they can change the UI in such a massive way that any community that benefits from long-form text posts quickly dysfunctional. I have noticed the gradual decline in content and comment quality on reddit since the redesign and the push for mobile app usage and it's slowly creeping into every subreddit out there.
With a federated alternative to reddit I could launch a server on my own and have control over how the community evolves over time( or rather, people who have the community's best interests in mind can in general). I hope someone takes what Mastodon does and make a reddit alternative.
... the problem with that is there are a lot of damned smart people who are good at what they're good at precisely because they view "wrangling tech" as a complete waste of their time.
Artists and musicians are a really good example, and are the key reason I use Twitter (pretty much the entire indie game industry lives on that platform), but this principle also extends to doctors, lawyers, scientists, teachers, historians, practically everyone in all the sciences, trades, and humanities ... except computer programmers. I've had aspirations towards being a polymath, and one of the bitter things I've learned about the programming-autodidact death march is how much of what someone learns isn't important fundamental knowledge, but is just byzantine details that will evaporate half a decade later when the software in question goes obsolete. It's excruciating to waste time learning something like that (which will become worthless) when I could be learning core programming fundamentals -- or could be learning something in a different field like drawing that will stay valid for the rest of my life.
I was a desktop developer looking-in-from-the-outside at web development, and I timed my entry into the field specifically to avoid the hot mess that was the API differences between netscape (yes, Netscape Communicator 4.5) and IE 6 at the time. As a programmer with enough of a nose for bad code smell, I took one look at that and thought to myself "this is completely insane." I could have knuckled down, busted the books and gotten all those api minutiae down just pat. Just in time for jQuery (or future browser versions) to show up and render them irrelevant forever.
It wasn't insane because it was a mess - it was insane because it was definitely going to go away, and soon. And once it did go away all the time I'd spent learning and mastering it would have been utterly wasted, sure - but wasted at the expense of not doing something else that's more important. That's time I could have spent learning the piano. Or a foreign language.
Now - not all programming is like that - eventually you've gotta bite the bullet and decide something's "good enough to get the job done". Many tech stacks actually end up in a nice sweet spot where they're coherent enough to learn quickly, and stable enough (in the sense of an API) to get out of the way and let you approach long-term projects without the ground shifting under your feet. But if you just blindly vacuum up proficiencies as they come off the treadmill you'll quickly discover you're doing nothing else with your life.
--
This is the relation most humans have with tech. Especially the smart ones who are really into something else.
It's a means to an end - and can work wonderful magic. But it demands a price.
The power of Facebook, Twitter and such is largely their feed. The feed is an answer to point #1: grandpa simplicity. The feed is the thing that needs to be protected from points #2 & 3: identity and culture.
The feed is that will motivate participants, content creators, etc. IE, many will create content for the feed. Reddit/Digg/HN upvoting were a simple way of doing the feed that proved this point and paved the way for Facebook & Twitter. A lot of mass adoption services created after went extreme. Feed only. The Tinder UX is a feed of datable people, for example. Spotify's importance is being solidified by being a feed. Importance in spotify's context is largely "being the place where music gets popular."
One way or another, the feed issue must be addressed. New content UIs will either have a successful feed or transcend the feed some way.
I'm not sure how this relates to federation. A feed seems a centralizing element by nature. Successful federated systems of the past (the the WWW) tended to leave the feed ("discovery," in a pre-feed context) to someone else... which turned out to be google, fb, etc.
However the labyrinth is is designed, what matters is how the feed works.
Because they get free money from the Federal reserve through cheap loans and government contracts and money attracts people. They can use the money to saturate the media with their brand until everyone ends up using them. Then they can use the money to crush all potential competitors.
These problems are so fundamental and universal that you pretty much need "a new form of Internet" that solves them, so then apps running in it can compete with centralized services.
So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. Handshake and Holochains are another interesting projects in this space, but more of a pieces of potentially bigger picture.
Coke is number one for a reason. Sure there is Pepsi, and yes there are small soda makers that either get acquired or close, but not too many people make their own soda.
For the same reason, there won't be a federated social network...
Unless, people don't even realize they're using one, but that gets back to your point, we'd need a new form of internet.
Sure there's a reason federated networks like Usenet died - they were killed off by a combination of the old/new AT&T/Verizon duopoly and various government organizations ( https://www.cnet.com/news/n-y-attorney-general-forces-isps-t... ). Yes, it happened for a reason - the old Bell last-mile monopoly and active government measures to quash it.
This has repeated a lot in the history of the western civilization.
Please note: I am being very careful with the claims I am making here. I have a little more than a layperson's understanding of Rome's history (years of Latin classes paying off at last!) but not nearly enough to call myself an authority on governments through the ages.
My point -- my only[0] point -- is that "The Internet started off federated and has become centralized / nobody has gotten federation to stick yet" is not enough of an argument to support "federation is doomed to failure".
My comment does not totally disagree with its parent, which seems to think that alternative approaches may succeed. We certainly agree that it's important to learn from the past.
I'm having trouble wrapping this up, but don't want to spend more time, so I'll leave it on that somewhat unfinished note.
[0] edit: I suppose there is also a secondary point that systems design is, like governance, an inherently social issue. This makes any analysis a lot more complex than if we restrict ourselves to purely technical merits.
If you want a classic example, some of the Greek city states might count. Though keep in mind that many of the Greeks considered sortition (ie random selection) the hallmark of the democracy, and considered that voting would just lead to aristocracy or oligarchy.
The German Free Imperial Cities are an interesting object of study. And there's lots to be learned from eg China as well.
Well, looking at the US at least, they weren't wrong.
Eg the German electoral system is also a bit messy, but seems to work well in practice. That might be a better yardstick to measure what voting can do, before we dismiss it.
(My favourite sortition variant right now:
- everyone can register as a candidate, if they want to - every voter can pick one candidate to put on their ballot - we select a few hundred ballots at random to fill up a parliament
For completeness sake, add some mechanism to handle a single candidate being selected by more than one randomly drawn ballot. Eg you could draw again, or you could give that candidate extra weight in parliamentary votes, or you could form a chain and add a person of the double-candidates choice to parliament, or you could have voters write down multiple candidates in order, and just pick a later one on the voters list etc.
Parliament can then select the executive, like it is already done in the UK or in Germany.)
The sortition system I sketched would still reward specialisation of labour, but there would be fewer threshold effects: if you convince thirty friends friends to ballot for you, you'd have a ten times higher chance to make it into parliament compared to only convincing three friends.
With voting you have threshold effects. In all but the smallest towns, getting thirty people to vote for you has exactly the same outcome as zero votes.
I did intentionally avoid clarifying this (also, technically the US is a republic) because I was trying to paint in broad strokes — I don't think my point needed details to be convincing and so I didn't want to get bogged down in them. I'm happy for the clarification as a reply, though :) A better phrase, which I would have used if I'd thought of it at the time, would be "representative government".
What about GNUnet? They’re trying to build pretty much what you described, from the ground up. https://gnunet.org/en/
Superficially it looks to me like bunch of theoreticians overcomplicated everything, and it will never be practical. Probably great project to advance your academic carrier though. But maybe I'm wrong.
Email would beg to differ. Google and Microsoft may have a lot of users, but interoperability is astoundingly high.
Yes, there is a high barrier to sending emails from a new provider, and yes, Google/Microsoft have a large chunk of the market. Still, there's little barrier to using ProtonMail or Fastmail or whatever free host you want, and everything interoperates shockingly well. You don't typically have to think about "what if this person has gmail?" or "I don't know if I can send to that host from outlook".
Looking at stuff created in the last 10-15 years, are there any federation successes? I can't think of any. (Blockchain excluded for the reason stated in the article)
Of course, there are few if any political leaders or CEO's posting there, but that's not a definition of success to me.
I have thought about what would happen if someone like, say, Musk would proclaim that they're quitting Twitter and setting up a Mastodon instance instead. I'm not sure the Fediverse would be better off if that happened.
I have met many new friends on all the networks that I have joined however.
I don't think that is necessarily a bad thing, but I think for federated services to catch on, you need big players that provide reliable service and a recognizable brand. The average person is not going to want to search through thousands of small federated servers to try and find one that works for them. And running a single server might be simple and inexpensive, but making it reliable and secure is not.
Yes, but importantly, those few big providers federate. That means that the barrier to entry for the little guys is much lower. This situation, while not ideal, is vastly better than the non-federated alternative.
That's just because installing and maintaining your own email server is extremely complicated [1]. Lots of people maintained their own Kazaa/Limewire servers for their friends (and the bigger piracy community) back in the day, because it was as simple as double clicking setup.exe and installing the software, and then configuring it.
Email does require higher reliability than Limewire. But I imagine if someone made a 100$ box that I could plug into my router, and so could 3 of my friends, and that had an email server installed on it or could be easily by clicking a installation file, and the 3 could act in concert to provide email services to the 50 people, maybe you would see more uptake.
[1] I am a tech-savy user who has been looking for a simple solution for a decade now.
It's not anymore [0]. However, good look on getting Microsoft's or other's mail servers to accept your e-mail. That's a big reason why even tech savvy users give up on hosting their own mail server. I've been running my own mail server for over two years now and the situation hasn't improved. My last problem was that a mail server from a government agency straight up refused the SMTP connection and contacting their postmaster mailbox yielded no response.
(For anyone wondering: It's not a problem with the configuration of my mail server, I've checked it about 10 times now. It passes all MxToolbox checks. However, I'm using an .xyz domain which might contribute to my problem.)
[0] mailcow.email
This includes, having out of the box configuration, that makes the email acceptable by Microsoft/Gmail etc.
[0] https://mailcow.github.io/mailcow-dockerized-docs/i_u_m_inst...
It's a bit more complicated than that but the simplest solution I know.
> This includes, having out of the box configuration, that makes the email acceptable by Microsoft/Gmail etc.
This is unfortunately impossible. You can have a perfectly valid configuration including reverse DNS, SPF, DKIM and DMARC, yet Microsoft's email servers will still let your mails vanish.
media organizations, pols, and public sector institutions do. Your grandma doesn't need to use Mastodon and it's not about her. It's about those who will be followed and have something important to transmit.
And if your not part of the main group, what's even the point of federation? It's practically "servers" at that point.
No amount of technology is going to fix the 'a majority of people don't like you or interact with you' problem though.
it's more interesting because the UX encourages different styles of posting (no QRTs for dunks, for one) that people are developing into slightly different social idioms. also no ads, chrono timeline, pick your own image crops...
That's nonsense.
I run my own instance and don't use any sort of ban list, and I can interact just fine with both the "main" instances, and a whole host of smaller, more niche, instances.
I run my own instance. As far as I can tell, this is false. There are some ban lists floating around, though I'm yet to see a common, standard policy of blocking instances because they don't subscribe to a particular blocklist.
Spam is ramptant. Running a server is a nightmare.
Standards are a tar pit.
It "works" only for fairly loose definitions of "works".
Using spam-assassin, clamav, and a few recommended tweaks (greylisting, HELO rules, blocking .ads, .party, and friends) and I'm down near zero spam.
Each urbit instance is a peer. The stars and galaxies provide discovery and initial routing, which is a pretty neat solution, they're basically fancy DNS servers.
What I think is exciting about urbit, is that each identity can host services. So you can launch a chat room, and it's yours, other peers can show up and say things in it, but this is very different from federation, where there is a sharp distinction between administrators and users.
Anyone who comments in your chat room can have a chat of their own, and the cryptographic identity of you and them is identical and consistent, wherever in the fleet you may encounter each other.
Stability and performance have improved, but there's a long way to go before a smooth one-click user experience is a reality.
But the basic design is solid, and it has a chance precisely because it isn't federated, but distributed. Secure Scuttlebutt is another truly distributed system, of very different design; it's locked in some bad architecture but I found it quite pleasant to interact with.
Don't expect anything useful from Holochain, ever. Just... I don't want to get into inside baseball, but, nah. Never gonna happen.
SSB is neat, but ineed could use rearchitecting. I could imagine ssb-like system + handshake to work quite OK for certain things.
Thanks for info on Holochain. I've been only lurking in there and reading docs, so my understanding is rather shallow and theoretical.
The entire form of current global human society is federated so you can't simply state it does not work. For the internet so far we have a few empires that battle it out, but I don't think they are manageable long term and this has started to become apparent.
Would you care to mention them?
Email's federated nature has kept it alive for decades, and will ensure that it long outlives more proprietary forms of social networking.
Federated systems already work in practice and at scale.
edit: Some other examples with widespread adoption:
- The phone network (PSTN)
- The internet (via BGP)
- The postal service (federated between countries via the UPU)
- Podcasts (and RSS more generally)
Am I missing any?
At the same time, I know several people who gave up their email addresses and only use Facebook now. Maybe the next iteration of social media behemoth will be even more effective at that.
Though to be honest I expect they just use their phone if they need to reset their password.
Email in particular I don't think really fits the mold of social network. I think principally because there's not really a public post concept it's all essentially point to point communications. There are some abstractions like mailing lists but they're still an express list of people who will get it. I guess that might not be so different from a 100% private profile but the lack of any public functionality excludes it from the social media category.
nowadays mostly for contacts that either dont use a smartphone or dont have mobile internet connection (yes! those exist).
are u sure u are using sms? asking because apple overlays imessage and iphone penetration is said to be highest in usa.
And in general I consider it a terrible method of communication.
What percentage of email traffic do you think is social in nature (and not just notifications from an actual, separate social network)? I'm guessing <0.1%.
Podcasts exactly exhibit what I fear for with federated systems. As podcasters move to "Spotify exclusive", Spotify still calling those podcasts, and those podcasts not being available through your general podcast subscription app, could lead to most podcasts being de facto centralised again.
The same could happen to the Fediverse: some player adds some feature that makes large swaths of people migrate there, those features are not interoperable, and then that service turns off federation altogether...
Am I missing out on content from Facebook/Twitter/Instagram because I'm not connected indirectly to everyone in their gardens? Sure, but I'd argue it's low quality content that was only a drain on my mental health.
I guess I'm trying to say, I think it's about the individual success and not the % of market share.
You could argue maybe A) won't work or isn't possible or needed, but I feel B) myself a lot. For example I meet someone and want to talk to them, but it's likely they might suggest we chat via Skype or Discord or something like that, and if I don't want to use that service I'm often out of luck.
That, personally, is what bothers me the most - that network effects seem inherently anti-competitive because they force me to use a platform I wouldn't otherwise want to use, only because they purposefully make their software unable to communicate with competing platforms.
For all of the open protocols and specifications we have from HTTP to TLS to HTML and literally one thousand others, we just re-use them over and over to create closed systems that can't talk to each other, which feels like going backwards. Sure it's great all of our computers speak TCP, UDP, TLS, HTTP, and so on. But they don't all speak iMessage, Zoom, Skype, Discord, Reddit, Twitter, Mastdodon, Matrix, Slack, etc etc. I still feel constantly locked-in to all of these platforms just when I want to communicate to someone that happens to be on it, and the area is certainly not progressing because the fight to gain this huge market share of users is never-ending.
Ideally, that is what I would love to end via federation. If I could actually use some type of FOSS and open protocol, and actually talk to any other person I wanted to with it, and I wasn't forced to use a certain platform, service, company, and give them all of my data in the process. I don't see any way for this to be achieved, but I can always dream.
My favorite video game is Halo 3. It never died, I just left.
Okay, so you should subscribe to 5 services to see all the latest anime you should see. You want to support the artists. But, some of them have horrible experiences, and you are locked into it because you want to _legally_ watch this show.
It's that if your platform is exclusive enough, you also become very slow to innovate.
We've seen time and again that most innovation comes from constrained settings. But big corps just want to maximize money, they don't care about improvements. That is IMHO the block stopping us from actually tackling a lot of identified and solveable problems we face this century.
There certainly are UX problems that the current federated social media projects have. They're not technically unsolvable, it's just that current federated social media projects don't want to solve them for some reason. That's what I'm trying to address with my project (that's not yet ready to be demonstrated, sorry).
The primary problem that the current federated social media projects are facing is that there's no global user search. At all. You sign up for a Mastodon account on the instance of your liking, and now what? Sure you can follow Eugen and like the cat pictures he reposts. Cats are cute. But how do you find people you already know without asking every one of them whether they have a Mastodon account? You don't. And that's how you not build a useful social media service. Every single social media service that has had any semblance of success offered an easy way to bootstrap your social graph — be it a detailed global search feature and rich profiles to go with it or the ability to import your friend/contact list from somewhere else. That alone should be telling.
There are technical solutions to this, like making a DHT out of instance servers that could be used to link the user IDs on centralized platforms to ActivityPub URLs. Even if said centralized platforms refuse API access to instance admins, there's still machine-readable GDPR data export to take advantage of. It can be done. It's just going to take some time.
> "You’re certainly familiar with another federation which is not based on ActivityPub: email."
Self-contradicting article?
Email has a lot of spam, but a federated system which keeps this in mind from the outset needn't suffer the same fate.
EDIT: Incorporating the replies to my comment, I also want to mention that the lack of large, centralized players like gmail also prevents spam. If there are a lot of intances, and they're all small, then cutting one off for bad behavior is a lot less difficult. Instance admins can be held accountable for the behavior of their users.
How is distributed identity solved in the long term? Nobody would run a server to communicate spam; that's rarely how email spam is sent these days. Most of the spam I get is from accounts hosted on GMail and Yahoo!. So the weakest link is a single server that's community-respected, but doesn't have adequate spam protection. Federation makes that much harder, because the whole point is that every server is run separately.
Mastodon gets no spam today because it's smaller than Hacker News. There's relatively nobody on there.
There's nothing saying any particular instance will moderate, but if they don't and it becomes a problem they'll quickly find themselves cut off from federating with other instances.
Same way email handles it -- through the use of the domain name system. You'd expect that an a federated server with the address "https://mastodon.mit.edu/" would consist of those with a tie to the Massachusetts Institute of Technology (faculty or students). Your identity is provided by the system you are a member of and handled by your specific system administrator.
background info:
https://github.com/WebOfTrustInfo/rwot9-prague/blob/master/t...
We see the complaints about this every time someone posts about an email server. It's hard to get the big guys to play ball. On the other hand, every time email comes up, people complain that there's too much spam.
That's a problem that can be fixed from the get go if you design a federated system that doesn't assume all actors are charitable. The major players could cut down on a huge amount of spam by only accepting signed domains, but email wasn't designed to force all email through only it's home server, so that's not as backwards compatible as you'd like.
That's not however a strict federation problem, it's more of an 'email was designed to assume everyone is a good actor, before we had much proof to the contrary' problem.
I think the lowest cost solution to this would be a WordPress site on a shared host somewhere, with this plugin installed:
The technology exists in the current system. We just need to activate it. This isn't a technical problem.
The status quo with email is not like that, unfortunately. Either you federate with GMail and accept the possible spam, or you disrupt significant part of communications of your users by refusing to accept email from Google's mail servers, with Google not losing basically anything (and having no intrinsic reason to fight spam). So it becomes a loss-meh situation instead of win-loss.
Awesome, so users will naturally gravitate to the servers that spent the most $$$ money fighting spam. You end up with Gmail all over again.
I don't know why you are using Gmail as the spammer in this situation. Arguably the 2nd biggest reason Gmail got so big is at the time their spam detection was lightyears ahead of everyone else. Then as they got larger they killed off smaller servers that didn't do the magic song and dance, and those users couldn't talk to most people as everyone was on Gmail. There's nothing stopping this from happening again on some new federation platform.
This is my biggest gripe with federated services. They all start from the point where "the Internet is centralized" and offer some proto-Internet to undo it. The Internet started federalized and the easiest solution to many of the problems that came with it was centralization. It's always unclear how you prevent this from happening again.
Or hopefully all servers have good enough open-source anti-spam built in so that it isn't a centralizing force.
Machine learning mitigates this somewhat - doesn't really matter if the code's open source if even the people who wrote it have no idea what's actually happening in the black box.
Of course, that method of anti-spam has an entirely separate set of problems. But we've got those either way.
If the model weights aren't available you won't get many instances of the server running.
Honestly, I'd like to see more yammering about federation by people aware of the early federated systems of the internet and some perspective on their uses and failings.
Or, better, less yammering and more realizing. Preferably more so than small networks where Nazi Twitter is the biggest node.
- it's a distributed system and not really a federated one: you can set up your own SMTP agent and send mails to anyone without prior registration to a federation.
- anyone can impersonate anyone: if I want to send an email pretending I'm bill.gates@apple.com there's no built-in mechanism to prevents that.
Theses aren't fundamental features of federations, and came from a design which just overlooked how many bad actors email would attract when it became mainstream.
The way the impersonation problem is dealt with in ActivityPub is very similar to DKIM in the SMTP world: the sender uses a cryptographic signature (HTTP signature) to prove that he is allowed to send messages from that actor. Of course this doesn't prevent someone from sending spam from their domain, in either case.
Which makes me wonder, how come we call these systems "federated" in the first place?
Federations I'm familiar with feature a central authority composed of representatives from the member states. That authority controls admissions and membership, and usually has some level of additional authority over members, along with capability to enforce it. None of that is present in these "federated" systems.
But looking at how the terminology is used in practice, "federated" refers to systems with N users per host while "P2P" describes systems with one user per host.
There is actually some debate as to whether it's really a federation if the central authority can override member states (as per https://en.wikipedia.org/wiki/Federation)
One could say publishing an MX record on your domain is that registration.
To the extent that this problem is email-specific, the ultimate root cause is a failure to deploy DMARC. Any new federated system will require a DMARC/SPF-like mechanism for preventing request forgery.
These questions are too much for laypeople, and I can't imagine a scenario in which your friendly local system admin is your service provider for a critical service in your life like Facebook. Could you imagine email working that way?
I see federated systems naturally ending in centralization in the form of a couple big megacorps that the average person trusts to not shut down tomorrow. Convenience and ease of use will win out. It happened with email, it arguably happened with xmpp back in the day. Change my view?
For me it starts with the basis of a communication protocol for services https://github.com/asim/mucp. And then moves onto something on a broader scale https://gist.github.com/asim/ad6e157294d39c2040a5d51ab3d3216.... I don't have all the answers yet, but I feel something is coming. An answer to our future needs and its not p2p as you say. P2P might be useful for trade but not services or long term ownership and privacy.
I think P2P ends up being closer to the goal of distributed control because it doesn't tend towards centralisation over time. My issue with P2P though is that it still requires trackers for boostraping in most cases. DHT only gets you so far.
I think there will be some interesting advances once IPv6 becomes more widely deployed. Hopefully this time around multicast is widely deployed on the internet. Imagine being able to subscribe to a multicast group and instantly receive DHT nodes over the internet without a tracker? Not sure how this would work in practice, but I have heard of some research being done here. I think the future looks very good for P2P applications, whereas federated applications are still on shaky ground.
Federated communication or social media works differently than P2P file sharing. In file sharing you're locating random anonymous peers but in social media that's the last thing that you want. When communicating with a specific person you need their identity anyway and you can use some system like DNS or a blockchain to find the server backing that identity.
Multicast on the public Internet is not going to happen due to state management cost, and it's not helpful for federation anyway due to the spam problem.
Yeah this was Gagron's fault. Running a flagship instance was bound to result in this. I make it a point to educate people about not joining mastodon.social and finding another instance that better suits their interests.
Who cares what the most popular Slack or Discord is? I'm sure there is one, I bet it's a lot like hanging out on ##hottub back in the day. Matrix is fantastic for organizations.
Mastodon is... I mean it's a Twitter clone at end of day. People want reach, no on should be too surprised that poasters tend to cluster in the biggest instance.
I think chats for companies/organizations/projects are a good fit for federation, and with the adoption of Matrix by the French and now the German government, Slack should be looking over its shoulder.
Whereas if I ran a Mastodon instance, then of course, I would want as many users as my droplet could carry.
I'm not knowledgeable about sex work at all, but a lot of the discussion I read noted that it seemed troubling how OnlyFans had so much authority over how the content creators did business. It's not uncommon to hear how sex workers or people in the adult industry are at the whims of opaque moderation on social media too. [2]
Sex work can be an exploitative industry, but I think federation could bring a lot of power and control back to the content creators involved online.
Disclaimer: Again, I'm not an expert on the topic of sex work or knowledgeable about it at all. It'd be a lot better to get feedback of the people involved or know the industry better.
[1] https://www.bbc.com/news/business-53979625
[2] https://jezebel.com/instagram-s-solicitation-policies-are-ex...
Without easy setup of payment providers, federation is difficult. Any new instance in the federation needs to go through the same pain.
Meanwhile, what does federation actually solve? The only fans model (and most sex work) seems to be a social graph that is a single producer talking to a bunch of clients. The producer needs a server, clients need to connect to that server, where does federation come in?
Disclaimer: I'm not at all knowledgeable either - literally just know what I've read on hn on this subject.
None of them really stayed totally p2p. With Gnutella, nodes would slow network connections and weird NAT conditions would pop in and pop out. So eventually things changed, and some Gnutella nodes which were online for some time period and had a fast network connection would self-elect as super-nodes (or ultrapeers, or various names). In Gnutella, theoretically ever node was a server and a client, but eventually, Gnutella server/clients would hook into to these super-nodes.
If you want the maximum amount of content, both centralization and decentralization have their drawbacks. You tend to find elements of both (with Gnutella and Bittorrent too).
Again, you end up with a system in which money is correlated with user experience, and the operators who most efficiently monetize their userbase will be the most successful.
Novel approaches to the business side of decentralized services will naturally allow for a re-think of the UX issue. (Business model -> default config for "grandpa")
~
An alternative solution approach requires faith in one's government. Can you trust it? If you can, then a national identity services (yes, the dreaded digital id) can provide the infrastructural back-bone to build federated systems.
At the end of the day, location and identity is -all- you need to "reach out and touch someone".
I think that is for the better.
People need to stop harping over the technical appeal of the Fediverse, lest they only are interested in courting the technical crowd, which is actually fine with me. If that's what they want more of then they should not be concerned with who's a part of it. I think these federated platforms should not be interested in trying to appeal to anyone at all, because it's resulting in imitations of the exact platforms that we want to get away from. Under the hood they may unique, but all most people are going to care about is the front end.
Personally, my preference for a new social networking service would be very, very plain. Chat-orientated, share documents, comment on shared documents. No hearts or arrows or "Rebeeps, repoots, and boobops". Is there a way to federate message boards? I just want to talk, read and talk about what I've read. Is there a way to federate Hacker News?
There's Lemmy https://github.com/LemmyNet/lemmy
Then one day, for no obvious reason, the repo received hundreds of stars within a few hours. It was a trending Rust repo and was recognized on TWiR, but there was no actual implementation of federation at all. Just a JS frontend, and I think a tiny bit of stub code in Rust.
I'm not saying there are upvote rings or botnets amongst any particular group, but I haven't paid attention to GitHub stars since then.
I would think that most any pub-sub system with a minimalist protocol that supports text and maybe some file types would be adequate to build on. However, I think marketing and social factors have more to do with success and failure of these platforms than anything else. There are certainly a lot of capable programmers who have tried or who are trying to make a social media platform without pervasive user surveillance.
Some of the more interesting open questions for systems that want to strongly enforce the sovereignty of logical nodes:
How do you design inter-node protocol strategies that exhibit optimal, robust Nash equilibria in non-trivial real-world environments i.e. uses resources efficiently in all plausible configurations absent centralized control? This is fiendishly difficult because relevant properties of nodes scale in complex, interacting ways and game theory is not sufficiently developed to solve some of these cases in practical systems. In any sufficiently complex and real-time federated environment, you want these mechanics to be almost completely automagic. Currently, there is a bias toward restricting the number of logical nodes in federated systems because that means humans can plausibly deal with the brittleness, which implies centralization.
How do you prevent nodes from being attacked/exploited from both the outside and inside, including malware, DDOS, fraudulent data, exfiltration of private data, etc? This is a separate question from the intrinsic theoretical robustness of the protocol design in terms of resource and scaling efficiency. Also, how do you make a policy easy to describe and enforceable with strong guarantees? The state-of-the-art in high-assurance internetworking (which considers these kinds of problems) is quite far from where it needs to be address these issues convincingly.
How do you audit user behavior that spans multiple sovereign federation domains to limit abuse and misuse of the system while leaking minimal private state?
Some of these are just Hard Problems, unless you have a very simple/restricted system, that we will need to carefully design around to minimize human involvement; most federated system designs just ignore these issues. For others we have plausible approaches theoretically but no real technical implementations with sufficient generality or functionality. And then there are the fundamental trade offs and edge cases that we tend to ignore but which are really important at scale.
tl;dr: federated systems make distributed systems look easy. The reason they are not commonly used is that a correct design for a non-trivial system is deep into the frontier of computer science. We don’t have a complete theory for how to do it.
And by dead simple, I mean something grandpa can do. All they should need is enough skills to install an app, click through a welcome wizard, and start writing on its WYSIWYG editor.
If the computer doesn't have a publicly reachable IP then that's a much tougher obstacle. That can only be solved by voting with your wallet for ISPs that do give you a public IP with reachable ports. Net neutrality is critical for decentralisation and self-hosting.
I only use that path because I have no idea what the windows equivalent is but it also ships with a similar feature.
Money is the only incentive which works in our modern age. Any federated or decentralized system needs to start with money because money is the ultimate tool for control.
> You’re certainly familiar with another federation [...]: email
Spam as a term transitioned from, “flooding a communication channel with the same message” to “ads sent indiscriminately” due to a green card advertisement on NNTP.
What we need along with federation is federated moderation. There are some characterizations of content where we can agree to a label but not whether the label is bad. You might trust me to label gore accurately, I might trust you to label incitement to doxing. We might never agree on what constitutes baiting. Or the crossover point where a flame war should be filtered. So you might ban an individual who I think is “keepin it real” or vice versa. But ten of us shouldn’t have to do the exact same moderation in order to maintain a federation. N log n would be great, 3N would probably be tolerable.
With today's IRC user count and today's server capabilities, you might have an easier time running on a single server, but there's maybe some latency benefits for users with geographically diverse servers (maybe with support for a server to server mesh instead of a tree, to avoid multi-hop message paths)
The fiat monetary system is a ponzi scheme. Watch "Hidden Secrets of Money": https://youtu.be/iFDe5kUUyT0?t=62
I also recommend doing your own research after this. The evidence is clear and overwhelming.
Cryptocurrency is merely a response to this scam and its continued success helped to further prove that the financial system is a scam.
Even if you assumed that cryptocurrency is a scam, what kind of financial system enables a scam to thrive for 10+ years? In such a system, how can you tell the scams apart from the non-scams?
Clearly the so-called 'free markets' are not capable of distinguishing the two.
What's to say that most big tech corporations aren't scams? They get a lot of 'free money' from the government through massive contracts to build projects which only add more bureaucracy.