Comment by lxgr | Hacker News Reader

Comment by lxgr | Hacker News Reader

> In the pull-based system, more work in the end is required (when should Alice query Bob? Also Bob needs to respond to the query, though thats super easy as it is static responses), but the work is better distributed, lowering the maximum amount of work someone has to do (in this case, Bob).

I don't see how that follows. Yes, work is better distributed temporally (since consumers hopefully poll the feed in a randomized way independent of new posts appearing), but the baseline load of these polls will in the end be larger than that of having to do the push fanout per post – at least for people posting less frequently than the average poll rate.

Generally, the push-vs-pull discussion seems like a red herring: For every pull system, we'll want some push mechanism for efficiency reasons in the end anyway; for every push system, we'll need a pull way to catch up with posts potentially missed initially.

To me, the practically relevant differences between Mastodon (push) and e.g. Bluesky (pull-ish, with aggregators) seem to revolve around the actual ease of self-hosting: As the author notes, setting up a Mastodon server seems roughly as complicated as self-hosting email (i.e. possible but practically almost nobody will do it), but I don't see this as a limitation of the protocol (Activitypub), but rather its implementation.

Decoupling identity resolution from hosting an entire server would also be a smart move: Webfinger is way too complicated for this; DNS TXT records would be ideal.

yes it is a federated system by design. doesn't mean it is not a limitation though.

to give you an example: in case of email, though it is not that difficult to host your own server (and even if you are a small startup you'll most probably do so without much effort), in the end basically Google decides on "who is an accepted participant in the network". If Google deems you spam, you are spam. If Google deems your authentication emails "promotion", for most intents and purposes, you are "promotion" and your users will miss your emails.

that, I feel, is an inherent limitation of any federated system, specifically one whose design is really inspired by email.

> I don't see how that follows.

as you've mentioned right after.

> work is better distributed temporally (since consumers hopefully poll the feed in a randomized way independent of new posts appearing).

Still, what's the benefit of that? If peak load is a concern, a push-based system can stagger out individual post deliveries just as well, and push gives the producer much more control over load management. If that's not enough, several posts can be combined too.

In a pull system, you are at the mercy of your consumers' refresh rate setting, and for infrequent producers, you'll have lots of wasted cycles fetching nothing new on top of that.

I do agree that pull is simpler to implement (since subscription management is handled entirely on the consumer side, requiring no network protocol and server-side state for it), but in terms of network calls, it's strictly worse.

the system doesn't need to be completely pull-based though. I don't even think most modern RSS readers are fully pull-based, don't they support WebSub?

the main point is to separate publishing and distribution, making publishing far more accessible and decentralised. for that to happen, I guess, from a publisher's point of view, the system should be pull based. of course we can have hubs and relays to add pull-based mechanisms to ease the load of the system.

p.s. even in that case, strictly speaking, yes a pull-based system, or even a hybrid one, will always require more work than a fully push-based system.

> the main point is to separate publishing and distribution, making publishing far more accessible and decentralised. for that to happen, I guess, from a publisher's point of view, the system should be pull based.

Oh, I completely agree with that assertion: Static content hosting is much easier than stateful subscription management, knowing which aggregators to post to etc.

I just think that this does inherently put more work on the subscribers, and there's no real way to do it efficiently in a relatively flat architecture (with subscribers directly polling publishers). WebSub helps with pure distribution, but not with aggregation (e.g. to allow keyword/hashtag search), for example.

That doesn't mean it's not worth still designing a system like that (in my view, the benefits are significant!), but I wouldn't call it a performance win.

agreed. it won't be a performance gain at all.

but I suspect aggregating / indexing / etc. still is going to be the most resource consuming part of a push-based system if you want your content discovery to not be limited to two-way interactions, which means the gains of a push-based system, in terms of performance, shouldn't be that much (I suspect the main gains will be in realtimeyness instead).

Reply on news.ycombinator.com