RSS to ActivityPub Converter | Hacker News Reader

RSS to ActivityPub Converter

github.com

92 pointssohkamyung34 comments

For those here who enjoy how ActivityPub allows you to push content and then receive comments back but don't have the mental bandwidth to implement such system for their blog, let me point you to:

  https://brid.gy/

It uses microformats and webmentions to find content in your blog and then pushes it to silos and other services. If someone comment about it somewhere that is not your blog, it captures that comment and send it to your blog as a webmention.

All this is part of the (somewhat complex but beautiful when it works) standards and services that people from the IndieWeb movement are shipping. You can see more info about it at:

  https://indieweb.org/

The Bridgy service mentioned early can push your blog content to mastodon and pick replies, boosts, likes and send them back as webmention as well.

I am still not super sold on the ActivityPub format as being better than RSS, any dissenters want to change my mind?

Push tends to be more efficient, and more responsive, for content that updates less frequently than the polling interval.

Also without "pushes" every actor should "follow" any other existing actor just to check if there are messages for them. Like my email server having to poll every other email server in the world to discover if I have new messages.

>Like my email server having to poll every other email server in the world to discover if I have new messages.

In AP you don't need to follow anyone to have messages for you delivered directly to your inbox.

You have to check other servers if you're interested in messages to others but I guess there is no analog to that for email.

Yes that's what I wrote too. Because AP provides a mechanism to "push" messages more or less like emails, one does not need to follow/subscribe to anything in order to receive messages unlike RSS where one does have to subscribe.

That's a pure efficiency perspective. When discussing content sources, however, privacy is also an important concern. By necessity, a push-based protocol needs to know who to push to. A pull-based protocol only needs to respond to requests, which may be anonymized. The operator of an RSS feed gets a list of source IP addresses of feed requests - and that's it.

But it also increases load on small yet popular sources. On the other hand proxies, conditional request headers and HTTP 304 make for a great yet simple distributed caching infrastructure.

Of course if you’re thinking in terms of “broadcast internet” with a Facebook at its center you may be correct

Mail is also push based, seems to work fine without a caching infrastructure.

Mmmm, RSS over e-mail. Is it bad that I want this?

Isn't that just a mailing list?

I guess I need to clarify. I don't want email summaries of blogs - that would be awful. I want websites to offer the ability to send raw RSS feeds over email as a push service.

So basically WebSub but you don't need to run a WebSub server - you could just parse feeds using, for example, a procmail script. Email is already a well-supported protocol; WebSub is not.

Check out https://rssmailer.app

There's quite a few services that will convert new items into emails for you (IFTTT, and others).

A lot of email clients also have support inbuilt. (Thunderbird, Outlook, and others).

I have been using https://github.com/rss2email/rss2email with a dedicated email address for several years now, tracking around a hundred RSS feeds.

The system works quite well, and I can read my RSS both via a client-side app (thunderbird in my case) and on the go via the webmail. I even made a small modification to rss2email to automatically "tag" the mails so I can use mail filters to automatically sorts the feeds into different folders.

I have a server that fetches the RSS content a few times a day, but I guess it could also work "locally", for example by doing it once each time I turn on my computer (if you are okay with "missing out" sometimes on very active feeds).

I decided to do the switch when I noted that the interfaces of my email client and of my RSS reader were basically indistinguishable.

EDIT: for the anecdote, this project was initially created by the very same Aaron Swartz that, among other things, contributed to the creation of the RSS format http://www.aaronsw.com/weblog/001148

To add to the other suggestions, 'blogtrottr' is a convenient free service for receiving RSS as email, with settings for frequency and so on.

What do you think an MTA, MDA, and mail stores are?

They're not really caches, they're storage infrastructure.

There is still a difference between cache and what's on hard disk.

It would surely be less load than each of those push-destinations attempting to pull the source multiple times to query for updates

Personally, I like WebSub a lot more than ActivityPub for this use case. ActivityPub seems really complicated (there's inboxes and actors and objects and activities and like twenty different message types), while WebSub is pretty much just a way to broadcast "hey, my feed updated, go download it if you want":

https://indieweb.org/How_to_publish_and_consume_WebSub

How do you even compare the two? With RSS you aggregate information from different XML sources; with AP there are "actors" sending/receiving JSON(LD) messages to each other.

I agree with you. When I saw this project I thought "the bulk of this should be about 50 lines of XSLT and then a simple script to manage push".

I am probably underestimating the complexity of the problem, but I have a feeling that this is a "I hacked this together in an hour and it works a lot better than expected" type of problem.

It's like the difference between a static website and a JavaScript web application. RSS is straightforward, ubiquitous, and thus maybe superior for one-way publishing. But if your content needs a more stateful interaction with its consumers, you'll end up needing the richer set of actions provided by ActivityPub.

My blog's ActivityPub system is both pushing content out (similar to, but not quite the same as, RSS) and accepts responses as comments (nothing like RSS).

(My blog's currently down due to too much unrelated tinkering).

So in summary: RSS is simple, mature and works.

ActivityPub is still experimental, and you may not even need it.

I don't understand how your conclusions were reached from my post.

My blog's content is natively shown inline with other content on Pleroma. To a Pleroma user, it and responses seem like all other native posts and interactions. They can click its ID to go through to my original blog. There, everyone sees my blog and its style, as well as the same responses as native comments. Back on Pleroma, those users can boost it and share it with others (their followers). Virality achieved. No further engineering needed from me to enable that.

That's federated social networking. Imagine writing a YouTube comment from Twitter's interface and your Twitter account. In this case, my blog's content -- not just links -- natively pops up in followers' feeds when I publish something. And since ActivityPub is built on and compatible with RDF, this could be extended to all sorts of domains and interactions.

> (My blog's currently down due to too much unrelated tinkering).

Oh good, I'm not the only one with a broken blog:) I discovered recently that while my blog is up, I didn't have source files that would generate the site as it exists on the server. Nice to know other people have similar things going on:)

There's also https://fed.brid.gy/ : if your site has microformats2, it'll turn the site itself to activitypub.

I also built https://github.com/edhelas/atomtopubsub/ a while ago :)

Because XMPP Pubsub is basically a container for anything an https://xmpp.org/extensions/xep-0277.html already fully standardize the presence of Atom 1.0 in XMPP it's super easy to publish standard RSS/Atom feeds to XMPP and enjoy fully real-time, decentralized news system.

There is no need to "fake" an ActivityPub account or edit your articles content, the feeds (called Communities) can be managed with roles, subscriptions and many other features and the articles are basically standard Atom elements (like you can find on your average HTTP Atom feed, the container changes, the content stays the same).

All those things are already standardized for years (Pubsub is there since 2002, Microblog since 2008) and implemented in the main XMPP servers.

If you want to check things client side, I'm working on Movim that implements all those things already https://movim.eu/.

Here is the XKCD feed on XMPP Pubsub https://nl.movim.eu/?node/comics.movim.eu/XKCD and our Movim Blog https://nl.movim.eu/?node/pubsub.movim.eu/Movim (notice the likes, the comments, the edited content, the rich content)…

Visiting https://nl.movim.eu/?node/pubsub.movim.eu/Movim seems to try to load some images via port 5280.

What's the reason for using non-standard ports?

Indeed, our image uploading system through XMPP used to use the 5280 port. This was changed recently to proxy things through the standard 443 port :)

Unfortunately we cannot change the existing URLs that were spread across the whole network so we're keeping an exception for the existing ones.

But the new files are now uploaded on this kind of urls https://upload.movim.eu/files/9d94237298995552fa13436420195f...

Moderately tangental - is a node development/runtime environment really so common that it need not be called out as a dependency (such as how this one specifies beanstalkd as a dependency)?

Node.js is the just "hippest" thing and if you're "cool" you already have it installed.

At least that's what I'm told, I'm one of those ancient beings who still remembers the old Javascript, back when it was the opposite of cool. I'm happy it's matured greatly since then.

Node is the first thing listed in requirements in the readme

Reply on news.ycombinator.com