Hacker News Reader

Comment by dpc_pw | Hacker News Reader

Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder.

These problems are so fundamental and universal that you pretty much need "a new form of Internet" that solves them, so then apps running in it can compete with centralized services.

So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. Handshake and Holochains are another interesting projects in this space, but more of a pieces of potentially bigger picture.

View on HN

> Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder.

Email would beg to differ. Google and Microsoft may have a lot of users, but interoperability is astoundingly high.

Yes, there is a high barrier to sending emails from a new provider, and yes, Google/Microsoft have a large chunk of the market. Still, there's little barrier to using ProtonMail or Fastmail or whatever free host you want, and everything interoperates shockingly well. You don't typically have to think about "what if this person has gmail?" or "I don't know if I can send to that host from outlook".

Well most people use one of a few big email providers, not small individual mail servers run by tech-savvy friends or family.

I don't think that is necessarily a bad thing, but I think for federated services to catch on, you need big players that provide reliable service and a recognizable brand. The average person is not going to want to search through thousands of small federated servers to try and find one that works for them. And running a single server might be simple and inexpensive, but making it reliable and secure is not.

> Well most people use one of a few big email providers, not small individual mail servers run by tech-savvy friends or family.

That's just because installing and maintaining your own email server is extremely complicated [1]. Lots of people maintained their own Kazaa/Limewire servers for their friends (and the bigger piracy community) back in the day, because it was as simple as double clicking setup.exe and installing the software, and then configuring it.

Email does require higher reliability than Limewire. But I imagine if someone made a 100$ box that I could plug into my router, and so could 3 of my friends, and that had an email server installed on it or could be easily by clicking a installation file, and the 3 could act in concert to provide email services to the 50 people, maybe you would see more uptake.

[1] I am a tech-savy user who has been looking for a simple solution for a decade now.

> That's just because installing and maintaining your own email server is extremely complicated

It's not anymore [0]. However, good look on getting Microsoft's or other's mail servers to accept your e-mail. That's a big reason why even tech savvy users give up on hosting their own mail server. I've been running my own mail server for over two years now and the situation hasn't improved. My last problem was that a mail server from a government agency straight up refused the SMTP connection and contacting their postmaster mailbox yielded no response.

(For anyone wondering: It's not a problem with the configuration of my mail server, I've checked it about 10 times now. It passes all MxToolbox checks. However, I'm using an .xyz domain which might contribute to my problem.)

[0] mailcow.email

Come on, this is the installation process [0]. If it's more complicated than apt/yum install mailcow, followed by a nice friendly browser based configuration page, it's too complicated for widespread usage.

This includes, having out of the box configuration, that makes the email acceptable by Microsoft/Gmail etc.

[0] https://mailcow.github.io/mailcow-dockerized-docs/i_u_m_inst...

> If it's more complicated than apt/yum install mailcow, followed by a nice friendly browser based configuration page, it's too complicated for widespread usage.

It's a bit more complicated than that but the simplest solution I know.

> This includes, having out of the box configuration, that makes the email acceptable by Microsoft/Gmail etc.

This is unfortunately impossible. You can have a perfectly valid configuration including reverse DNS, SPF, DKIM and DMARC, yet Microsoft's email servers will still let your mails vanish.

> Well most people use one of a few big email providers, not small individual mail servers run by tech-savvy friends or family.

Yes, but importantly, those few big providers federate. That means that the barrier to entry for the little guys is much lower. This situation, while not ideal, is vastly better than the non-federated alternative.

While they do federate, it's a challenge to get them to accept mails from your own mail server. Microsoft's mail servers are notorious for accepting your mails with a "250 Queued for delivery" and then letting them vanish in a black hole. I've also ran into problems with mail servers of government agencies. So yes, while technically possible, self-hosting your mail server sadly isn't viable anymore for the little guy.

Email was already too big to fail by the time Facebook and Google got big enough to be able to kill it.

Looking at stuff created in the last 10-15 years, are there any federation successes? I can't think of any. (Blockchain excluded for the reason stated in the article)

What's the definition of success? I spend far too much time on the Fediverse (mastodon) and if success is measured by the amount of time I spend there, it's the most successful service I use.

Of course, there are few if any political leaders or CEO's posting there, but that's not a definition of success to me.

I have thought about what would happen if someone like, say, Musk would proclaim that they're quitting Twitter and setting up a Mastodon instance instead. I'm not sure the Fediverse would be better off if that happened.

Wide adoption. Especially for social network type apps, which are rather useless without my friends in them.

To each their own, clearly. I don't think I ever joined a social network because my friends used it (possible exception of FB, but I'm not actually using that one. There could be correlation there).

I have met many new friends on all the networks that I have joined however.

Urbit is also not a federated system, but rather, a true distributed system.

Each urbit instance is a peer. The stars and galaxies provide discovery and initial routing, which is a pretty neat solution, they're basically fancy DNS servers.

What I think is exciting about urbit, is that each identity can host services. So you can launch a chat room, and it's yours, other peers can show up and say things in it, but this is very different from federation, where there is a sharp distinction between administrators and users.

Anyone who comments in your chat room can have a chat of their own, and the cryptographic identity of you and them is identical and consistent, wherever in the fleet you may encounter each other.

Stability and performance have improved, but there's a long way to go before a smooth one-click user experience is a reality.

But the basic design is solid, and it has a chance precisely because it isn't federated, but distributed. Secure Scuttlebutt is another truly distributed system, of very different design; it's locked in some bad architecture but I found it quite pleasant to interact with.

Don't expect anything useful from Holochain, ever. Just... I don't want to get into inside baseball, but, nah. Never gonna happen.

Yeah, for me a federated system taken to its logical conclusion turns into p2p system.

SSB is neat, but ineed could use rearchitecting. I could imagine ssb-like system + handshake to work quite OK for certain things.

Thanks for info on Holochain. I've been only lurking in there and reading docs, so my understanding is rather shallow and theoretical.

The current shape of the internet itself (BGP peering among autonomous systems, a decent number of large tier 1 ISPs, etc) is, on its face, a relatively successful federation. The applications running atop that federation are, as you say, centralized, but much of that is because residential ISPs forbid customers forbid servers and even if they didn’t (they haven’t always) residential customers are prone to accidentally running botnets of various types.

> Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder.

Sure there's a reason federated networks like Usenet died - they were killed off by a combination of the old/new AT&T/Verizon duopoly and various government organizations ( https://www.cnet.com/news/n-y-attorney-general-forces-isps-t... ). Yes, it happened for a reason - the old Bell last-mile monopoly and active government measures to quash it.

Usenet died from a combination of web forums being a better experience for most people, and a saturation with warez binary traffic becoming so expensive that ISPs did not want to pay for it.

> So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying.

What about GNUnet? They’re trying to build pretty much what you described, from the ground up. https://gnunet.org/en/

I'm aware of GNUnet, and been following it loosely for too long to remember. It's been around for like 2 decades?

Superficially it looks to me like bunch of theoreticians overcomplicated everything, and it will never be practical. Probably great project to advance your academic carrier though. But maybe I'm wrong.

I am not sure this is self-evident; it could be that there is no end state, but rather a cycle of centralized and federated systems, and we simply haven't seen enough of the cycle to identify it, yet. As a parallel, consider systems of government in the last 2500 years or so.

Could you share examples from the governance cycle?

I was thinking of the rise and fall of democracy from before the Roman empire and through the present day. Pick any time and look back only a few centuries, and the answer to "What is the natural end state of government?" looks very different. Zoom out a bit and it's clear that if there is an end state, we haven't reached it yet.

Please note: I am being very careful with the claims I am making here. I have a little more than a layperson's understanding of Rome's history (years of Latin classes paying off at last!) but not nearly enough to call myself an authority on governments through the ages.

My point -- my only[0] point -- is that "The Internet started off federated and has become centralized / nobody has gotten federation to stick yet" is not enough of an argument to support "federation is doomed to failure".

My comment does not totally disagree with its parent, which seems to think that alternative approaches may succeed. We certainly agree that it's important to learn from the past.

I'm having trouble wrapping this up, but don't want to spend more time, so I'll leave it on that somewhat unfinished note.

[0] edit: I suppose there is also a secondary point that systems design is, like governance, an inherently social issue. This makes any analysis a lot more complex than if we restrict ourselves to purely technical merits.

I generally agree. Though just to be clear: Rome was never considered a democracy.

If you want a classic example, some of the Greek city states might count. Though keep in mind that many of the Greeks considered sortition (ie random selection) the hallmark of the democracy, and considered that voting would just lead to aristocracy or oligarchy.

The German Free Imperial Cities are an interesting object of study. And there's lots to be learned from eg China as well.

> Though just to be clear: Rome was never considered a democracy.

I did intentionally avoid clarifying this (also, technically the US is a republic) because I was trying to paint in broad strokes — I don't think my point needed details to be convincing and so I didn't want to get bogged down in them. I'm happy for the clarification as a reply, though :) A better phrase, which I would have used if I'd thought of it at the time, would be "representative government".

> considered that voting would just lead to aristocracy or oligarchy.

Well, looking at the US at least, they weren't wrong.

Perhaps. I actually think some variant of sortition is superior to most voting systems to eg fill up a parliament; but to take the US as an example to argue that voting is bad would make the job too easy.

Eg the German electoral system is also a bit messy, but seems to work well in practice. That might be a better yardstick to measure what voting can do, before we dismiss it.

(My favourite sortition variant right now:

- everyone can register as a candidate, if they want to - every voter can pick one candidate to put on their ballot - we select a few hundred ballots at random to fill up a parliament

For completeness sake, add some mechanism to handle a single candidate being selected by more than one randomly drawn ballot. Eg you could draw again, or you could give that candidate extra weight in parliamentary votes, or you could form a chain and add a person of the double-candidates choice to parliament, or you could have voters write down multiple candidates in order, and just pick a later one on the voters list etc.

Parliament can then select the executive, like it is already done in the UK or in Germany.)

My post was a little tongue in cheek. I don't think a voting sysyem necessarily leads to an aristocracy/oligarchy. But it certainly can, and it needs to be carefully crafted to avoid that.

Well, voting leads to specialisation of labour, and thus a political caste.

The sortition system I sketched would still reward specialisation of labour, but there would be fewer threshold effects: if you convince thirty friends friends to ballot for you, you'd have a ten times higher chance to make it into parliament compared to only convincing three friends.

With voting you have threshold effects. In all but the smallest towns, getting thirty people to vote for you has exactly the same outcome as zero votes.

A cycle of democracies / republics / authoritarian regimes? A cycle of smaller independent states / large empires?

This has repeated a lot in the history of the western civilization.

E-mail doesn't work too? Federation works, if some walled garden obsessive lock-in freaks (like Compuserve and AOL of old or today's analogies of Facebook and the like) don't manage to mess things up.

Email sucks.

Spam is ramptant. Running a server is a nightmare.

Standards are a tar pit.

It "works" only for fairly loose definitions of "works".

Email is totally feature frozen as well. There is infinite room to improve email but it will never happen because it requires the participation of too many parties. With centralized services there is always someone who has the final say and can force change.

As an example, we still haven't solved the wrapping problem that's been known for 40 years and has a fairly simple solution.

Is this the line length limit you are talking about?

Well, there isn't really any limit as such, but yes.

Centralized servers are just as hard to run, if not harder. You just don't experience those ops yourself without working there.

Have you tried to setup a mail server these days? There's plenty of nearly turn key solutions. Even from scratch on Debian, Fedora, or Ubuntu is pretty easy these days, a few apt-gets and a few additional tweaks and away you go.

Using spam-assassin, clamav, and a few recommended tweaks (greylisting, HELO rules, blocking .ads, .party, and friends) and I'm down near zero spam.

How's your delivery? Anything that runs the risk of my mail getting blackholed by a clients provider is a nonstarter (even if it's only a few days every few years).

Been fine, I have noticed others with terrible results. My best guess is that large email provides use domain age. So new domains have terrible problems with delivery. My domain is many years old.

Sure. But it's still far better than any of the alternatives.

Still, it works.

If these problems are so obviously ineluctable, why talk around them instead of detailing or even naming any?

You mean, other than Mastodon, which has a few million users and growing.

Mastodon is proof at least that federation doesn't fix the issue of censorship. There's a common ban list in Mastodon that if you don't include, you will not be able to join the "main" Mastodon cluster.

And if your not part of the main group, what's even the point of federation? It's practically "servers" at that point.

It does fix the issue of censorship, in that 'blacklisted' instances can still have a world of their own and use the technology or protocol.

No amount of technology is going to fix the 'a majority of people don't like you or interact with you' problem though.

> Mastodon is proof at least that federation doesn't fix the issue of censorship. There's a common ban list in Mastodon that if you don't include, you will not be able to join the "main" Mastodon cluster.

That's nonsense.

I run my own instance and don't use any sort of ban list, and I can interact just fine with both the "main" instances, and a whole host of smaller, more niche, instances.

Well I'm really glad that that trusted circle of instances exists, I don't want anti-Semitism or racism in my timeline. So far octodon.social has been a step up from twitter for me, because it isn't designed to monetize outrage. The model where admins have to police their community or risk being de-federated with the mainstream means that an instance is limited to the scale they can effectively moderate, which is exactly where other social media platforms break for me.

So basically Mastodon is moderated in exactly the same way as Twitter, with the same set of rules. How is Mastodon more interesting than Twitter then?

the only way you can imagine something like Twitter being more interesting is more content Twitter prohibits?

it's more interesting because the UX encourages different styles of posting (no QRTs for dunks, for one) that people are developing into slightly different social idioms. also no ads, chrono timeline, pick your own image crops...

>There's a common ban list in Mastodon that if you don't include, you will not be able to join the "main" Mastodon cluster.

I run my own instance. As far as I can tell, this is false. There are some ban lists floating around, though I'm yet to see a common, standard policy of blocking instances because they don't subscribe to a particular blocklist.

My grandma will never use Mastodon, but she uses Facebook. Tech isn’t the only audience you need.

I think tech is a tiny minority on Mastodon. From what I can see there are far more queer folk and lefty ppl. Your grandma will never use Mastodon because there are no network effects luring her there, but the UX is about as good as a normal centralized social site.

sorry, but your grandmother doesn't have a few hundred thousand to a few million followers.

media organizations, pols, and public sector institutions do. Your grandma doesn't need to use Mastodon and it's not about her. It's about those who will be followed and have something important to transmit.

> Federated systems just don't work in practice because of inherent problems with them

The entire form of current global human society is federated so you can't simply state it does not work. For the internet so far we have a few empires that battle it out, but I don't think they are manageable long term and this has started to become apparent.

> because of inherent problems with them that don't have good technical&social solutions yet

Would you care to mention them?

You would more likely need a new form of people.

Coke is number one for a reason. Sure there is Pepsi, and yes there are small soda makers that either get acquired or close, but not too many people make their own soda.

For the same reason, there won't be a federated social network...

Unless, people don't even realize they're using one, but that gets back to your point, we'd need a new form of internet.

Reply on news.ycombinator.com