Hacker News Reader

Comment by darknavi | Hacker News Reader

Federated networks like Mastodon and Lemmy are going to get people well-acquainted with websites shutting down. It's hard work (time, money, etc.) to run these things for people, and people start to really lean on them.

It's almost novel now days getting sucked into something that shuts down. killedbygoogle.com is a meme partly I think because websites shutting down is just so uncommon in areas that we get personally invested in.

I run my own Lemmy instance just for my self and even that can be trying sometimes. I enjoy using it instead of reddit, but one day I will probably shut it down and be sad.

View on HN

I'm starting to think nostr was barking up the right tree after all. Put as much complexity into the client as possible and make the servers dumb and completely uncoordinated, utterly interchangeable. Spam your broadcasts to any relay that will listen. No idea if it actually works (I've read a lot about nostr and AT proto but never used either of them) but I think it's very obvious that any system that deeply relies on some company that everyone becomes extremely reliant on (including AT proto/Bluesky) is only a couple steps away from the same sort of problems as centralization.

Of course, the real gold standard would be P2P, if only it could work. But... mobile phones can't burn battery running P2P clients in the background, everyone's under a NAT these days, and some types of software (like microblogging networks) would be horrifically intractable as a P2P system.

Oh well. At the very least, I really love the concept of Decentralized Identifiers (DIDs). I'd like to see more stuff like that.

The atproto team came from the p2p space. We had a lot of experience running client-side computation. There are challenges you can try to solve in that design -- key sync, key recovery, reliable hosting, reliable discovery, etc -- but what you can't get around is the need to run queries against aggregations of data. Even scales that we consider "mid" start to strain on the client-driven model. Federated queries might be able to solve it, but it's extremely challenging to get reliable performance out of that. The design we landed on was replaceable big nodes.

The common comparable people raise is email/gmail, but we used the DID system and account portability to try to get a better outcome on provider migration. It's hopefully more like web/google -- which still has the centralizing pressures of scale benefits, but hopefully injects enough fluidity in the system to move the situation forward. Sometimes, you pick the design that moves the ball down the field, not the one that guarantees a touchdown. If somebody wanted to improve on what we've done, I'd tell them to focus on the federated queries problem.

In theory AT proto doesn't seem like a bad design. I've read a fair bit of the docs, although mostly skimming. (I've been meaning to read the paper on Merkle Search Trees so I can figure out what exactly is going on with PDSes.)

On the other hand, in practice it seems like the AT proto infrastructure is still very centralized for now. DIDs are excellent in theory, but everyone is using PLC DIDs, which depends on the centralized plc.directory. You can run your own PDSes, but there's only one relay for now (that I am aware of.) I also don't think there is more than one instance of the Bluesky AppView, and the official instance is locked into the Bluesky Moderation Service, which seems to limit the usefulness of some of the censorship resistance of the protocol.

I'm not sure how much of that is social problems rather than technical, but I worry that since Bluesky and AT proto are gaining massive popularity with this status quo (millions of users!) it'll have a problem akin to Matrix.org, where in practice almost everyone is using the same infrastructure anyways.

It's still relatively early days, but millions of people is definitely enough to where you start to hit problems with having everyone under one roof. I really hope we get to see how things play out when more of the network is operated independently.

fwiw roughly 50% of Matrix is on the matrix.org instance currently. we consider this a bug, but also prioritise ease of onboarding over decentralisation purity ideology.

I hate to be a downer but there's a lot of things Matrix could prioritize over decentralization. That said, the decentralization works pretty badly. Large federated joins are somewhere between comically slow and horrifically slow. Status does not seem to work well across federation either.

I'm also a bit miffed that Dendrite was positioned as a "next generation" Matrix server but now it feels nearly orphaned with missing support for newer features, issues with various appservice bridges, few updates at a very slow pace, and no migration path out in sight. I know it came with a fair number of disclaimers, but that still bums me out as it seemed like it would be okay for a small non-critical homeserver, and now it seems likely I'll have to engineer my own path out when clients finally stop working with Dendrite. (It already happened once...)

You have no idea how bad I want to love Matrix, but frankly if it was due to a focus on usability that decentralization "purity" has suffered, it simply does not show in the resulting usability improvements over years of time. Sorry to be harsh.

if you feel miffed, imagine how the Dendrite team feels, given the lack of funding which means it is on best-effort dev currently.

if you’re interested in progress on Matrix, https://matrix.org/blog/2024/10/29/matrix-2.0-is-here/ is where it’s at.

I did see the Matrix 2.0 announcement, though for obvious reasons I can't actually use any of the features listed in it. Obviously, improvements to the basic chat functions of Matrix would be great. For now though, I am stuck with the reality that joining a large federated channel sometimes takes more than 6 hours. I wish I were exaggerating.

edit: I guess though that faster room joins weren't a part of Matrix 2.0. Actually, I don't really have a huge problem with the sync taking too long personally. So maybe Matrix 2.0 wouldn't bring that big of an improvement for me anyway.

I share your feelings. I had switched to Dendrite 1 year ago and its flaws made me open my eyes to the major problems I had with the design of the protocol.

I started to worry that it would not improve and I reexamined XMPP.

Finally I switched to XMPP that I had abandoned more than 5 years ago because I think there is a better chance that the community will finally offer clients with the important features, on all platforms, in the coming year and that it will last over time.

The necessary future is more providers, more relays, a move of PLC to an independent body, and more DID methods.

I will also say - there are ~100 self-hosting PDSes in the network, about 25 relay consumers, 3 alternative appviews that I know of (smoke signals, frontpage.fyi, and whitewind), the firehose & backfill are fully available, the specs are getting fairly complete, and the software is open source. This is a priority for us.

I, too, ran my own instance. I enjoyed it for some time but I've now moved to the omg.lol ecosystem. I feel that by paying a little money for it that I have a higher chance of the server not shutting down.

+1 for omg.lol, it’s a great community.

ActivityPub seems to require a lot of hardware resources in order to run properly, which is unfortunate. It’s not something I would ever want to run myself, especially to the public.

I wonder how much of that is endemic to ActivityPub or how much is about the software stack itself

I don’t mean to be glib, just wondering if things can be “done better”

I'm guessing the syncing process between instances is really brutal on resources. Imagine constantly syncing external databases for your service to function properly.

I thought AP was push-based though? Might be wrong of course, and I think stuff like media is its own thing

AP is push based (which actually causes the "every instance has its own set of comments" problem). You can run pullers on a small instance to get a better experience if the remote sides support listing posts, but the standard sync process is no more than receiving HTTPS calls and storing the JSON contents in the right place.

There's some additional overhead (doing HTTPS calls for verifying signatures, for instance) but that information can be cached pretty effectively.

Pushing contents is no more than posting HTTPS calls to every server in your follow lists, and possibly exposing said content in a GET API for pullers, though that's entirely optional.

Mastodon is heavy because of the way the backend is written (I blame Ruby on Rails for tha one), but there are fully featured ActivityPub servers out there that are orders of magnitude more efficient. Mastodon devs prefer the ease of development over performance but that's a choice, not an inherent problem of ActivityPub.

Please be wary to conflate ActivityPub with the code on top of it, like Mastodon for instance; which is, on both front and back-end, proven to be resource intensive and difficult/costly to scale especially over time. (the older the dbs get, the more inactive users pile up, etc.)

Versus something like Pleroma; which I've used since it's inception, being incredibly janky and lightweight, prone to breaking, but later versions have mostly ironed out most of those catastrophic bugs. It has it's own challenges as well, but it does historically scale better, is more flexible, and less intensive per instance

One demands a lot of money and time, where the other demands a lot of time and not so much on the money side. I'm not going to spend the time to give you a history of pleroma/mastodon instances, as it's a controversial history at best and there's a lot of people who know little, yet who will believe themselves an oracle. (ofc that could also be me so take it with a grain of salt and all) Though if you are willing to read through a bunch of highly opinionated accounts, and you pay close attention to what actually happened, the answer is pretty clear.

ActivityPub is intensive, but not the main culprit.

> ActivityPub seems to require a lot of hardware resources in order to run properly

Correction: "The Mastodon software requires a lot of hardware resources in order to run properly"

Alternatives like GotoSocial, Akkoma, even Honk are much less resource intensive.

Besides the monetary costs of operating small fora there are also significant competence hurdles. Site owners who manage to hit a couple thousand users have to figure out spam handling, automated content moderation (including photoDNA and the required reporting if they host images), registering a DMCA agent with the copyright office, setting up an LLC, assessing their needs for COPPA, GDPR and CCPA, their site's tax situation and anything they may want to do involving employing others (such as T&S) without getting burnt out. The median size for a forum getting its first subpoena is 4,300 users.

Managing all of that is easily learnable in a couple months if they have time, disposable income and few distractions but surprisingly few people who have site management thrust upon them know about these things in advance. To most people who think about running an internet anything the above are unknown unknowns. You can't go looking for things you don't know exist, so burnout is high.

Citation highly needed. I'm close to quite a few people who run larger instances than that, including my own, and none of them have ever told me about getting subpoenaed. That's exactly the kind of war story we'd tell each other, too.

I've seen no evidence that running a fediverse server is nearly so legally fraught.

I'll go looking for the citation and it may take a couple hours but I'll point out that FWIW my info is drawn from some academic surveying vBulletin and Xenforo site owners in the 2010s so I wouldn't be surprised if they aren't applicable anymore.

Update: My apologies. I've totally confabulated old blog posts from Antone Johnson with TSPA articles and some Steven Myers papers.

The actual figure is a median of 1 subpoena per annum per 430k users if the majority of the users are under 30.

No problem. I thought that seemed on the high side but didn’t have any stats to counter with. Which paper does 430k come from? I’d like to squirrel that away for later.

Note that getting a subpoena isn't "legally fraught". You're being called on to assist in an investigation of someone else, not in trouble yourself.

https://wiki.archiveteam.org/index.php/Deathwatch

Eh, it's not new to federated sites. Many of the web sites I frequented a decade a go are dead. Or enshittified useless so I visit them rarely anymore.

It was sad to see say, the "user friendly" webcomic go away. But I enjoyd it in the time. Just live the moment. Don't expect even the big websites and apps of today to last - not at least in the form you enjoy.

> Federated networks like Mastodon and Lemmy are going to get people well-acquainted with websites shutting down. It's hard work (time, money, etc.) to run these things for people, and people start to really lean on them.

Mastodon is filled with such utterly basic UX issues. You move instances because the old one announces a shutdown? No old posts visible, no import possible. You want to see the history of an old account on another instance? The oldest toot you'll see is the first one that your instance picked up from that account. You have to switch to their instance to see old toots - there's a helpful link at the end of the feed, but it's still annoying. "Trending" topics only carry stuff happening on your server, and most of it is days old garbage. Search is horribly broken and inconsistent.

> You move instances because the old one announces a shutdown? No old posts visible, no import possible.

But this isn't "utterly basic" to solve on the backend due to how ActivityPub (currently) works. First you have to allow backdated posts[0] (not supported in the spec) which requires a mechanism to stop them being sent out (else they'll appear in current timelines[0]) but also you need a mechanism to send them out (to update the old URLs except how does the new instance know where the old instance sent the status? And how do you prove that you have the right to even request the change?)

These are probably not insurmountable but they do require a lot of thinking about!

[0] I ran into these importing an old Twitter bot into my Akkoma instance. I had to modify the server code and it was not a fun time.

> But this isn't "utterly basic" to solve on the backend due to how ActivityPub (currently) works.

They're basic for the user. I know a few people who left Mastodon for good after the second or third time they had to shift servers. That kind of stuff should have been thought of from the beginning...

> to update the old URLs except how does the new instance know where the old instance sent the status? And how do you prove that you have the right to even request the change?

The same way an account move is currently reported to the instances where followers reside and handled there - the account-move operation would only need to do a full re-scan of the old profile. That's a ton of traffic for people with followers from many instances, I agree, but the source instance could trigger the creation of something like a data dump that destination instances can download without hitting the API.

> something like a data dump that destination instances can download without hitting the API

That gets you the old statuses, great. How do you then insert them into your existing instance? You can't just repost them because they'll appear with new timestamps (bad). You can't just repost them with old timestamps because servers and clients assume "just arrived == now" (bad). If you're using sequential IDs on your status table, good luck with that because I'm pretty sure someone has taken the shortcut of using that instead of the timestamp. Assuming you can work all those out, now you need to update the old URLs in the follower timelines to point to the new URLs (unless we punt on this and just let the old timeline sit around as it.) Except you don't know who got those statuses when they were posted - the old instance would need to have kept all the queue records for every post and be willing to supply them to the new instance. Or you can "eh" that and send them out to the new followers (except we don't want to do that because it confuses current instances and clients to get old timestamps at a new time) but that doesn't mean everyone will be updated. Or you can try and persuade the old instance to redirect each old status to its new URL once you've updated the software and protocol and clients to allow for status redirection, obviously, and worked out how to verify that server X is actually allowed to redirect A@Y's old statuses and isn't some hijacker / spammer / whatever and ...

I've not even given this much thought - I'm sure people who actually dwell in ActivityPub and security worlds can give a much better explanation of why it's not at all easy to implement.

> That kind of stuff should have been thought of from the beginning...

Yep, can't disagree that a whole heck of a lot more thought should have been put into the AP protocol from the start.

Trending topics were a waste to add, anyway.

Reply on news.ycombinator.com