Hacker News Reader

Comment by echelon | Hacker News Reader

Peer to peer, not federation, is the way forward.

We should only build peer to peer social protocols.

Websites and communities should simply sample from the swarm and make it easy for non-technical users to post and consume. They should be optional and not central points of failure (or control).

{Twitter, YouTube, Reddit, Instagram, TikTok, WhatsApp, Discord} should work like {Email, BitTorrent, PGP}.

Bluesky and Mastodon are the wrong architecture.

The web, fancy javascript UI/UX, and microservices shouldn't be the focus. The protocol should be the focus.

A fully distributed protocol would dictate the solution to this exact problem.

View on HN

Bluesky is designed the way it is because of scale. How do you make a p2p app that can handle hundreds of millions of posts per day without beefy servers helping? Bsky is designed so that the microservices themselves can be decentralized and so multiple different types of apps can be built on the same protocol/infra.

Obviously, it’s early days, and hopefully there is even more experimentation in the p2p space. But atproto architecture is a very fair experiment in this space. I can store my data on my own server, use a client app I wrote, subscribe to a specific aggregation/feed service I prefer, use the moderation list I want… all while still being connected to the larger protocol & network. It’s pretty neat.

You use routers as the beefy servers. Unicast, multicast, broadcast.

Unfortunately that means the implementation needs to reach all the way into the network layer.

Multicast doesn't work on the global internet, and can't, due to problems of scalability and billing. It's sometimes possible to negotiate with specific ISPs to use multicast in specific ways on their network.

> How do you make a p2p app that can handle hundreds of millions of posts per day without beefy servers helping?

You design it with those requirements in mind? There’s no fundamental technical limitation at play here.

There kind of is: the computers the "p" run don't allow incoming connections and don't allow long lived processes.

You invert the problem.

People want to build store and forward systems because that is their mental model of the problem. store and forward system are fine, and there are many advantages to them, but direct request systems scale much better. basically have each user fetch their messages from the locations they want rather than delivering the messages to them. think how the web works vs how email works.

RSS! But you can't make a personalized timeline with a pull model, and that's where the money seems to be.

I think finding where the money _isn't_ is a fun way to find interesting projects.

> How do you make a p2p app that can handle hundreds of millions of posts per day without beefy servers helping?

Presumably by fusing the P2P and federated models together. There's no particular reason those two models can't coexist within the same protocol. It just hasn't been created yet.

Similar to how a good mesh networking implementation will make use a high bandwidth backhaul such as the internet if it's available.

ATProto may be the closest we'll get to that. PDSes are granular enough to serve individual users, and you can (theoretically) pull from a relay and index only posts from users you're interested in for your appview, if you're hardware-limited. Relays are fungible and pretty lightweight themselves, so you're not depending too much on any central server.

But people don't want to run an always-online server to send their stuff to peers, so they host it on the main bsky servers. The problem with p2p is UX; people don't want to DIY their server.

So I agree with you that they should work like email -- but I've always said that Mastodon is better because it is like email; aka the power is in the nodes.

What do you think is wrong about Mastodon? Genuinely curious because I also am super skeptical that ATProto brings anything that we really need.

> What do you think is wrong about Mastodon?

The same problems as always. Allow federation and you get...

- federation wars and moderators conducting these wars using their own users as hostages - I left Mastodon years ago when some particularly dumb morons decided to do bitchfights regarding Israel / Palestine. No I'm not interested in your pointless squabble, but I do care when I suddenly don't see posts from a bunch of users without even getting a notification...

- Mastodon-specific, when you move your account from one instance to another (e.g. as response to above-mentioned BS) your followings and followers migrate - but all your posts and media do not

- spam, trolls and griefers abusing the system, up to and including sending around CSAM material that inevitably gets sucked in by your instance, making you liable in the eyes of the law

- security issues. Mastodon has been full of these, no thanks I don't have the time to be constantly on guard lest I be exploited from above-mentioned griefers.

- other instances not giving a flying fuck about moderation or abuse going out from their instances.

Sounds like you want to run your own private instance. That way you control your own moderation and federation policies

I think the problem is that it's too onerous to run your own instance, but being on anything but the "default" instance means dealing with volunteer moderators imposing their worldview on the available discourse.

Creating a Mastodon account shouldn't mean supporting the particular political affiliation of the moderators, but I think it feels that way for many of the instances.

And then you are also on the hook to be a sysadmin (including all the legal aspects thereof). That's generally a bit much to ask of someone who just wants to chat with their friends online.

> Sounds like you want to run your own private instance.

I'd like to do so, yes, but that exposes me to a (not insignificant) financial cost, (especially in Germany) a significant legal risk from CSAM/DMCA et al., and a significant amount of effort in maintenance.

Sure, there are "Mastodon as a service" providers that take at least the legal risk and maintenance off of me, but again, these cost even more money, and now I have the risk that the hoster is a fly-by-night operation that one day decides to close up shop for whatever reason.

And if anything happens to that private instance (say, the hoster disappears, the machine disappears without a backup, or the hoster undergoes an orderly shutdown), in the best case I still may have enough preparation to migrate the followers, but the old content is lost in any case. And that is bad.

In contrast with Bluesky and to a lesser degree Twitter, I can at least be reasonably sure that the provider does not vanish over night.

ActivityPub supports a less compelling user experience for many people: you only have a partial view of the network (you won’t see all the replies to the posts of people you follow on other servers), no global search, etc

Technically the internet also doesn't have "global search" but people are able to get along just fine most of the time.

This is how offline social networks work, and it might be fundamentally the only way social networks end up working. If each instance can't filter what it receives, then spam is too easy. If every message is globally flooded, the system scales as O(N^2) and is easily vulnerable to DoS.

AT solves these problems. Even if AT turns out to be a bust, they have an excellent architecture.

AT works by the use of global relays which see everything.

Sure, but it shows global replies, it provides global search, it's not O(n^2), it's not easily DOSed, and it's highly amenable to spam filtering, which are the issues you raised.

It's true that this solution doesn't work for private posts and DMs, but the n in O(n^2) is much smaller there, so I don't think it's as much of an issue for personal data servers to communicate directly in those cases.

The problem with centralized social media is that the admins have power over you. They can ban your account with no recourse, censor some of your posts (or some posts you want to read), or even post something from your own account that you don't approve of.

Mastodon doesn't change this, it just changes who the admins are. It lets a person under the jurisdiction of admin A interact with a person under the jurisdiction of admin B, which is better than fully-centralized X, but it doesn't solve the fundamental problem. Your instance admin can still ban you with no recourse (account migration is incomplete, requires cooperation on both sides, and mostly exists to shut up Activitypub opponents who point these problems out). They're still just as (if not vulnerable) to government pressure as centralized social media, and considering that a single lawsuit could probably bankrupt most instances, I suspect they'd fold very very quickly. They can (and very often do) defederate from instances that post "too much nazi content", and if you disagree with the decision, there's again no recourse (you can migrate, but you won't get your lost relationships back).

A lot of us are our own instance admins, with our own accounts being the only accounts associated with our domains. I don't self-host though; I pay a dedicated hosting provider to handle this for me. This means I end up having a very similar relationship to my Mastodon provider as to my email- and cloud storage providers.

> They can (and very often do) defederate from instances that post "too much nazi content", and if you disagree with the decision, there's again no recourse (you can migrate, but you won't get your lost relationships back).

Worse, they defederate instances that don't also defederate instances that they dislike badly enough so you can't even have neutral instances where you can communicate with everyone.

Yes, very good point sure. I (as a Black not-right-wing person) have huge problems with the whole "The Bad Place" thing (long story short, Black folks that I generally agree with politically are absolutely horribly ban-heavy and way too power-trippy on moderation.)

Is there any other way to deal with spammers that can't be applied to non-spammers by a malicious admin?

Email is the prime example of federated communication. From protocol inception to painful expansion and aging protocol all until corporate apropriaton. But I still think federation is the way forward, absolute centralisation is bad I'll let you figure why, but absolute decentralization is also bad, limitations due to its nature, unusual working for most users... Meanwhile federation is right in the middle, and users already use it with email without even noticing!

People often mention email as an example of federated communication, but the way email works in practice doesn't entirely live up to that ideal. Good luck getting your own self-hosted email server to send emails that actually reach anyone using a major email provider; they'll just be blocked as spam.

In practice, email is much less federated than it seems. A significant proportion of people are just using gmail. You probably don't have to include that many providers to cover a majority of people in the US.

I think federation has promise, but federation in itself is not a solution. Technical approaches do not address the more fundamental issue that, regardless of the mechanics of the system, big players will have more influence on its operation and evolution. Thus we will always need sociopolitical mechanisms to restrict big players.

Federation does at least give you the choice of providers, even a little bit of competition goes a long way to improving a company's behavior.

But in practice in doesn't always give you a choice, because the biggest providers will embrace and extend and start providing things other providers don't. Or they'll just make it difficult to export your data, etc.

Email is by far the least secure form of communication in common use right now. It's trivial to impersonate others over email, and every MTA that processes your email has access to the full contents, because they are never encrypted except in flight (and except by a few tiny disparate groups using PGP, and even these groups can't authenticate one another). And not for lack of trying, I should add.

Comes across as an ad hominem. Email is insecure due to being dated, having a massive amount of inertia, and being essentially impossible to upgrade in the necessary ways without breaking backwards compatibility. None of that has anything to do with federation vs p2p vs centralization.

If you want a fair comparison for reasoning about security related challenges and tradeoffs you should probably go with matrix.

I don't agree with this at all. There are fundamental tradeoffs, and the reason no one has added e2e encryption to email, while we did add it to the web, is not because of backwards compatibility, it's because there was no compelling solution to some of these trade-offs.

Matrix simply doesn't solve some of the problems that email solves, or at least not in an e2e encrypted manner. For example, I can't send a document to a public institution's Matrix account, not in a manner that either (a) isn't e2e encrypted with no realistic risk of a MITM, or (b) doesn't require an out-of-band pre-approval, such as someone from the institution adding my account to some encrypted room.

Also, even if Matrix did find a way to make it easy to send e2e encrypted data to someone else without out-of-band communication, it would then suffer from the problem of spam. Every client would have to filter all incoming messages for spam, instead of being able to centralize this work at the server level.

Doesn't the spam filtering complaint apply in equal measure to _any_ E2EE messaging solution? Signal can't implement content based filtering either.

Out of band confirmation is similarly universal unless you're okay with either TOFU or delegation. (Delegation being recursively subject to the same choice.) TLS on the web goes with delegation and a root certificate store obviously.

My point being that none of this is specific to either email or federation more generally.

None of this is fundamental to the federated model. It's only because email is older than modern security practices.

It very much is.

Even the web suffers from problems of trust to some extent, with the PKI being a huge vulnerability and relying on the collective action of all browser vendors to act as a check on any CA trying to break the agreed guarantees. But in a world where you would have a hundred, or even 20, different popular browsers, with different geopolitical assignments, it would be far harder to punish a CA that decided to sign certificates improperly, e.g. to allow some government or criminal enterprise to MITM communication.

Establishing identity in a non-centralized manner, and without requiring a second, already secure, communication method than the one you're trying to authenticate, such as an in-person key exchange, is in fact impossible, not just hard. There are partial solutions, with different trade-offs, such as the PKI for the web, the TOFU with optional verification options used in Matrix or SSH, or the web-of-trust model of PGP.

We don't need large scale social networks in the first place. The Discord model of small communities is the way forward. Keep groups small enough for natural human social rules to apply. Slows down global dissemination of information for sure, but that's what the news is for, and anything important will eventually travel between communities anyway.

Yep. Once a system gets too large, its starts to break down and everything you do to make work ends up centralizing the process just like in real life. If you want things to work you keep it small and distributed.

I don't understand how you can seriously pose Discord as an alternative in this conversation as it's entirely centralized and full of all sorts of toxic behavior and failure modes.

Like at least suggest old school forums, IRC, or usenet.

Discord is technically centralised but in a way that mostly doesn't matter at the point of use, and its design avoids many of the failure modes of old school forums, IRC, or usenet where moderator cabals take control of any community and bully lowly users.

how does it avoid that? i have experienced just as many power tripping mods on discord as i have on irc. the only difference to me is that i have never seen an irc channel with over 20 million users

By making it very easy for every user to start their own server, rather than the multiple tiers of ircops/server admins/etc. where some users genuinely do have more power (and/or a level of technical ability that becomes a difference in power) than others.

The GP didn't say Discord itself, but the Discord-like model of small communities. Ironically it's also the old web forum model.

Almost. The key difference is I can log in to Discord once and post in unlimited communities. The auth UX is excellent. Joining communities is very cheap.

We need an open protocol of this concept.

I can't help but laugh at the irony of posting this in an ATProto thread.

That's essentially exactly what they're trying to solve for although focused on the Twitter use case rather than Discord. And also one of the key advantages of ATProto over ActivityPub.

I don't disagree, but I'm baffled that, with P2P as your preferred outcome, your orientation toward federated infrastructure is one of opposition rather than support. It feels philosophically confused to me; they're your natural allies, they're a step in your preferred direction and they have an instance of real world success (well, to a degree) which is important. Whatever theory of change motivates this form of criticism of federated services can't be one that's, say, intentional or strategic about outcomes. It feels more first principles.

One might also ask why P2P thesis statements only ever show up deep in the weeds in comment sections in response to the fediverse when logically speaking they would make just as much sense if not more in response to, say, any post about Facebook as a company or social media writ large, or business news about acquisitions, consolidation of web infrastructure into fewer hands, enshittification, or escalations of control over platforms.

Again, I'm fully on board with the dream of P2P but it feels like Buzz Aldrin criticizing Neil Armstrong for not doing enough to bring humanity into the space age.

This is a flawed argument. If someone wants X they don't need to also support Y just because it's closer to X than other alternatives.

It's not a logical syllogism. And I would hope you have more to say about the coherence of a position than that it's merely not forbidden by logic, which is something less than an affirmative defense of its coherence and its motivations. It's about the perfect being the enemy of the good. "Well it's not forbidden by logic" is about as pathetically empty handed as it gets, in terms of accounting for which battles you're picking.

I think supporters of P2P as "the one true way" perhaps don't realize that federation is just as peer to peer if your user count is 1.

The fundamental distinction between a communication network that is p2p and one that is federated is the storage mechanism.

For p2p the network itself is the storage, and as a participating node you connect and retrieve what is addressed to you while the amorphous data blob that contains said messages remains to float in the network. While for a federated network, the receiving node needs to be present on the network at all times to be able to access/receive the messages addressed to itself, after which the messages are absent from the network (to some degree or another).

Personally the overhead of having the network having to bear the weight of all its nodes data is too large to make it viable.

Unfortunately, the swarm is 99.99999% advertisements for penis enlargement pills. How can a P2P system filter them out? A federated system relies on each admin to filter them out. A centralised system does even better, relying on a single dictator to filter them out. A P2P system requires every user to filter every spam message, together consuming far more effort than the spammer needed to send it.

This isn't, and has never been a hard problem. Just pay for people's attention. People you follow don't have to pay, and make that transitive. Penalize people in your network who propagate spam by increasing the cost to get your attention.

This is one of the most interesting properties of peer-to-peer networks.

You can run your own ingestion algorithms, and one of the things you can do is set up inbound rules that incorporate micro transactions.

We have to build a lot of infrastructure to make this work, but it seems ideal for a world full of agents and autonomous systems acting on our behalf.

Do the outbound rules of other participants include microtransactions?

And who besides a spammer would pay more than $0 to have their message read by you? If I wrote a blog post about vulnerabilities of blockchains, or how I ran Doom on a pregnancy test, and you don't read it because I'm not paying you, you're losing value, not me. You guarantee an inbox of only spam — but at least you get paid for it.

If you've got great content, I would just follow you. Or someone I follow would follow you, and through the network it would lead to discovery. I want your content, so unless you charge for it, nobody's paying anyone.

If someone wants me to ingest something novel from far outside my network, one way to gain reputation might be to pay a microtransaction fee. I'd be free to choose to set that up as a part of my ingestion algorithm. Or maybe my peers do it, and if they "upvote" the content, I see it.

If my peers start acting poorly and sending spam, I can flag disinterest and my algorithm can naturally start deboosting that part of the network.

With such systems-level control, we should be able to build really excellent tooling, optimization, and statistical monitoring.

Also, since all publications are digitally signed, your content wouldn't have to be routed to me through your node at all. You could in fact never connect to the swarm and I could still read your content if you publish it to a peer that has distribution.

I still think that any content anyone is paying for you to see is necessarily spam.

I don't agree. I think the chief problem with advertising is that it is extremely repetitive. I'm not, in principle, opposed to being informed about new things relevant to my interests existing. In a world that is completely oversaturated with content, it is hard to gain traction on something new with word-of-mouth alone, even if it is of very high quality. There is a point to being informed about something existing for the first time (maybe I'll use it), and there is a reason why people would have to pay to make use of that informational system (the barrier to entry is necessary to make the new thing stand out in the ocean of garbage).

Advertising is never going to inform you - it is by definition about persuasion, not information. An advertisement is always designed to try to convince you to buy a different product than you would rationally choose yourself. Even a seller in a physical market telling you their tomatoes are very sweet and juicy is simply trying to get you to buy: they have no idea, and don't care, if their tomatoes really are sweet and juicy (and definitely not sweeter and juicier than all the others tomatoes in the market), they just think you're more likely to buy from them if you hear that.

> An advertisement is always designed to try to convince you to buy a different product than you would rationally choose yourself.

Perhaps you could consider toning down the absolutism. This is true in many or most cases, but certainly not all cases. Let's take, for example, video games. I can afford to purchase any game that interests me, and do. However, I often go several months between new game purchases, because I am not aware of any games that interest me that I do not already own. An advertisement for a game does not need to convince me to purchase it over an alternative product, it simply needs to make me aware of its existence and broadly convey what the game is about so that I will know whether it matches my specific game interests closely enough to investigate further.

Particularly in the modern world of hyper-specialised interests, it's quite easy to get into a niche of a hobby where you have found and already purchased all of the things you are aware of. As another example, there are hyper-specific novel genres where there are at most a couple of dozen entries in that genre and you are able to read every single entry in it. You are still interested in that genre, and will likely purchase anything else in it, should you become aware of it. Enter the benevolent advertisement, which makes you aware of its existence in a mutually beneficial way wherein you get more of the content you are interested in consuming and the creator gets money.

> An advertisement for a game does not need to convince me to purchase it over an alternative product, it simply needs to make me aware of its existence and broadly convey what the game is about so that I will know whether it matches my specific game interests closely enough to investigate further.

I agree that it does not need to do more than inform you - but that doesn't mean it won't do more. Please show me a single advertisement for a game that doesn't use bombastic language, show highly selective graphics, or appeal to a sense of nostalgia. I for one haven't seen one, even ones for the niche indie games I respect the most. Sure, not all commercials are equally deceitful, but they are all meant to be persuasive more than informative.

I don't exactly go around saving advertisements, but plainly informational ones do exist here and there. Off of memory, an example of an indie game trailer I think is well-made is that of Wargroove[1]. It's a simple and clear clip reel of gameplay showing off a variety of content and features, and if I recall correctly, advertisements for it were simply smaller slices of the trailer. I think there's nothing offensive about advertisements like this existing (although, that said, the number of times I wish to see such an advertisement is still exactly once).

[1] https://www.youtube.com/watch?v=62nqJxq3E-4

I will grant you that this type of advertisement is indeed benign (though if I were really really really nitpicky, I could claim that the pace of gameplay shown in the trailer is probably not indicative of how you'd play the actual game, and I'm not sure if the music is part of the game soundtrack).

Still, I think this is such a tiny minority of real advertisment that it's barely worth mentioning. For example, here is a trailer for the original The Binding of Isaac, which (while being an interesting piece of art in itself, which many ads are) is stil clearly not just meant to inform consumers about the game, but instead is meant to sell a certain image of the game that it may or may not invoke in you:

https://m.youtube.com/watch?v=iDFnMfJnI7s

I'd also note that advertisments for artistic products such as games are some of the most ambiguous about the line between informative and persuasive, as the "feel" (atmosphere, tone, persuasive storytelling etc) of the final product is an intrinsic part of its value in a way that is not relevant for, say, produce, or consumer goods. It could be argued, for example, that the Story trailer for Elden Ring captures a real and important part of the appeal of that game, despite it including 0 details about the gameplay, and despite it being entirely original footage and dialog that is not in any way part of the game itself. The same ambiguity doesn't exist about an ad showing the glamorous lifestyle of someone who gets a mobile phone plan from company X, in contrast.

> If someone wants me to ingest something novel from far outside my network, one way to gain reputation might be to pay a microtransaction fee.

Nice in theory. In practice spammers will plant malware to steal microtransaction money from random people and push paid content down your throat for almost nothing. When you propose a novel model that will fix all the current problems, the first thing you need to think is how a bad actor would exploit it.

If a scammer, advertiser, or some other form of spammer can get a payout just 1% of the time, they will be willing to pay much more than the average person posting the average tweet.

If you make everything explicitly transactional, you will be left with only people trying to make a profit.

You have the graph of everything you follow, the graph of what they like, second order graphs ...

There are so many heuristics and models you can use to filter.

Penis enlargement spam is worth like $0.00000001 per message. Any number higher than that makes them lose money. The real problem is that nobody will post on a social media network where you have to pay to post.

Micropayments are actually a huge problem, which is a big reason why no one has ever successfully implemented what you're suggesting on any large scale. Email spam is a major problem, and has been almost since its inception, yet the only effective solutions have been the ones that increased centralization and made it harder and harder to run your own email server. And even with all of these modern solutions, a LOT of compute is burned by every single MTA to filter out the spam that goes through for their users based on content filtering.

And this disregards the simple fact that the only people willing to pay to have their words seen are people who are getting more money out of this - i.e. spammers (and yes, advertising in general, including "influencers", is spam in my book).

Should I create 1 million accounts with bots that scroll endlessly to harvest microtransactions?

Ah yes, the sybil attack. This is why establishing an identity is useful, and worthwhile. An identity with no proof is likely not a real person, and therefore has little value in being advertised to.

If you're a real person, then yes, it is valuable to show you things.

Want to know how I'm right? Because fingerprinting browsers and tracking people is how we establish that they are real in the current advertising world. Advertisers pay for that. Thus it has value.

Twitter is thronging with blue-check spambots. This idea has been comprehensively disproven. People will pay to spam you.

In fact, judging by the Exodus of non-scammers, only scammers will pay to send you their messages—which makes sense, since they're the ones who expect to turn a profit.

You did not understand what my original post suggested. I'm not suggesting people pay to be certified. If a spammer wants to pay me $20 to see their message, I am happy to see it.

> If a spammer wants to pay me $20 to see their message, I am happy to see it.

Yeah, but I'm not. It's spam. And the people whose messages I do want to see are overwhelmingly not going to pay $20 to show it to me.

This is a system that selects exclusively for advertisements. Nobody would want this.

Would you be willing to see an ad for $1000? A million? Sure no one would pay it, but you can set whatever limit you want.

No one would want this? Again I don't think you understand what I am proposing.

It isn't a a system that selects exclusively for ads. It selects for people you know, then people they know, and so on, and fades out how often posts show up the further away you get. If someone pays more, then more people will see their message in their network as it compensates people for their attention, starting with the people who value their attention the least.

No one would want this? You think people don't want to get paid for their attention? This is essentially what a job is.

You can centralize spam lists while still having the base communication protocol decentralized - that way people have the option on making their own decisions on whether "advertisements for penis enlargement pills" are really a problem - and let's be honest that's far from the only thing that gets moderated.

Reply on news.ycombinator.com