Show HN: MergeFeed – Paste a bunch of links to create a multifeed | Hacker News Reader

Show HN: MergeFeed – Paste a bunch of links to create a multifeed

mergefeed.net

48 pointsamadeuspagel32 comments

If you're looking for feeds, have a look at https://nightfall.city

This is interesting ,

Thnx for the link

I shall give my usual plea (for the OP and feed authors alike): except for podcasts, please deal in Atom rather than RSS if you have a choice. Let RSS die. Outside of podcasting software, everything supports both, and Atom is the technically superior format, including resolving various unspecified behaviour that does lead to inconsistency across clients, such as with respect to HTML markup in titles (which RSS has no answer for, so that one way or the other all clients will mangle some titles that contain <) and in content (where the main but far from ubiquitously used RSS answer is to leave a blank <description> or <summary> or whatever it is, and use the <encoded> element from the http://purl.org/rss/1.0/modules/content/ namespace).

For this project, that would mean emitting Atom rather than RSS. And possibly the autodiscovery needs tweaking too, as I observe it not finding the feed on my site https://chrismorgan.info/, which is properly specified; my first guess is that it’s not coping with the type=application/atom+xml attribute.

In related news, MergeFeed is currently vulnerable to injection attacks because it mishandles Atom titles that are specified as text. If you specify a title as HTML or XHTML, it filters out a <script> element properly (good! and while retaining inline formatting too!), but if you specify a title as text, it doesn’t escape the HTML (which is what it should do) or filter it (which would have been wrong but would at least have closed the security vulnerability). Proof of concept: https://mergefeed.net/-?urls=https://temp.chrismorgan.info/x...

> And possibly the autodiscovery needs tweaking too, as I observe it not finding the feed on my site https://chrismorgan.info/, which is properly specified; my first guess is that it’s not coping with the type=application/atom+xml attribute.

I fixed that: https://mergefeed.net/Untitled?urls=https://chrismorgan.info...

> In related news, MergeFeed is currently vulnerable to injection attacks because it mishandles Atom titles that are specified as text. If you specify a title as HTML or XHTML, it filters out a <script> element properly (good! and while retaining inline formatting too!), but if you specify a title as text, it doesn’t escape the HTML (which is what it should do) or filter it (which would have been wrong but would at least have closed the security vulnerability). Proof of concept: https://mergefeed.net/-?urls=https://temp.chrismorgan.info/x...

I fixed that too. But for now I wasn't able to fix it without also removing formatting. I might be able to figure out how to keep formatting without allowing XSS attacks later, but I'm not sure that's even a good idea. It's kind of a link aggregator and link aggregators generally don't allow formatting. You don't want people to try to gain attention that way, leading to an arms race.

I think that autodiscovery is only finding the first feed. Since you’re aggregating feeds, I’d expect it to find all feeds on all pages. Sample page with multiple feeds: https://chrismorgan.info/blog/tags/fun/ has the site-wide feed and then the tag-specific feed. I say only “I think” at the start of this comment because the entries in the second feed are a subset of the entries in the first, though it has a different feed ID and title, and I can imagine it possibly merging them.

Losing formatting makes me a little bit sad. The widespread idea that titles are necessarily plain unformatted text is just bad. Italics especially are very regularly useful.

> I think that autodiscovery is only finding the first feed.

Yes.

> Since you’re aggregating feeds, I’d expect it to find all feeds on all pages.

No, I don't think that would be useful. For example, a lot of blogs have autodiscovery for a feed of all posts and for a feed of all comments on their homepage. But if I want to create a multifeed of several blogs, I only want the posts.

Generally, the way I expect autodiscovery to work is that a link to the most specific applicable feed comes first. For example on the page for the tag "fun", the feed for the tag fun should come first, because that's most likely what people are looking for on that page. If they wanted the general feed, they'd go to the homepage.

> I'm not sure that's even a good idea. It's kind of a link aggregator and link aggregators generally don't allow formatting. You don't want people to try to gain attention that way, leading to an arms race.

Wait till you hear about Unicode bold and italics: https://yaytext.com/bold-italic/.

(I hold that most tools being weak doesn’t mean that my tools should be weak, and I view text-only titles as weakness, one that can be solved without introducing problems, so long as you don’t get this encoding problem.)

> …Atom is the technically superior format, including resolving various unspecified behaviour that does lead to inconsistency across clients, such as with respect to HTML markup in titles (which RSS has no answer for…

The answer seems unambiguous in the RSS spec, which is that <title> is plain text (not HTML). The reason I know it's not unspecified is that entity-encoded HTML is documented as supported in other elements (e.g. <item><description>).

Your evangelism for Atom is interesting because, of the hundreds of feeds in my newsreader, I don't believe any are Atom. So my impression was that Atom was "at least as dead as RSS" for user-facing applications, which makes me curious what the popular use cases are for it these days.

The problem of RSS is that it’s not governed by any spec. Yes, the spec says <description> can contain HTML and doesn’t say that about <title>, but RSS is truly defined by usage rather than spec, much more than most formats.

As a weak example showing this philosophy early on, from <https://www.rssboard.org/rss-0-9-2>, emphasis mine:

> Further, 0.92 allows entity-encoded HTML in the <description> of an item, to reflect actual practice by bloggers, who are often proficient HTML coders.

And that’s RSS in a nutshell, a format specified only in reaction to prevalent usage, where non-conformant documents are a normal part of life, and parsers are lax in a variety of unspecified ways because they’re reacting to usage. And what I have observed is various RSS feeds with titles containing characters significant to HTML being mangled by some feed readers, and various RSS feeds with descriptions that are plain text (possibly with characters like <, & and >, possibly not) so that some feed readers try to guess whether the <description> is in fact plain text or HTML.

The very existence and popularity of <content:encoded> is an acknowledgement that <description> is broken.

Atom is good in this regard because it provided a sane definition that people are vastly more likely to actually conform to; you can take the guesswork out of things.

I still wish Atom supported providing both text and html values; e.g. in https://chrismorgan.info/blog/make-and-git-diff-test-harness..., my HTML title is “Using <code>make</code> and <code>git diff</code> for a simple and powerful test harness”, while my text title is “Using `make` and `git diff` for a simple and powerful test harness”. On principle, I use the HTML version, but alas, most feed readers will strip the formatting and leave just “Using make and git diff for a simple and powerful test harness”.

I took a look through some of the feeds I subscribe to; RSS is definitely the significantly more popular format among them, but Atom is common too.

A bit more text describing the app on the website would be great - I wasn't thinking about RSS first...

I wrote a little Python service which combines multiple rss/atom feeds into a single feed, in case anyone wants to host their own API for doing that:

https://github.com/cristoper/feedmixer

So this could be interesting.

I get asked fairly frequently at work what blogs / newsletters / sites I read to keep on topic X.

Could I use this to create a feed for topic X sourced from what I read and then share the merged feed with others ?

Yes.

I like this. Text is too large when viewing on my phone though.

Just made it smaller. (You won't immidiately see the result due to caching.)

So it extracts RSS feeds from websites?

https://mergefeed.net/WFH?urls=https://wfhjobs.us/

Yes, it uses RSS autodiscovery.

A readability nightmare

Sorry to sound like an idiot, but what would I want this for?

- If you want to stay up to date on a topic, make a multifeed for it. Just google "topic blogs", click on a result that looks good and paste it into mergefeed.

- If you come across a blogroll (or any list of blogs) you're interested in, you can turn it into a multifeed.

- If you have a bookmarks folder with blogs or podcasts, you can paste it into mergefeed to turn it into a multifeed.

Mmm ... Thanks for the explanation, but it just feels like half of a feed reader (not removing articles read etc). And the format gives me a headache.

Stick it in your favourite reader (eg phone app)

That said Feeder didn’t like the URL in URL format and barfed on this because it has Twitter in it. I think OP needs to URL encode the params.

https://mergefeed.net/Cyber_Security_Blogs?urls=http://krebs...

I've thought about encoding the params, but that would make it harder for humans to read and edit.

Why not just put those feed in your RSS reader? What does the extra step add?

The only reason I can think it’s a handy way to share an aggregation

It's not intended to replace your feed reader if you already have one, but even then it's still useful to discover and share feeds.

Some multifeeds based on topics:

- Cyber Security: https://mergefeed.net/Cyber_Security_Blogs?urls=http://krebs...

- Economics: https://mergefeed.net/Economics_Blogs?urls=http://econbrowse...

I'm pretty confused about the concept. Is it supposed to look like this? https://imgur.com/a/4A52Wer

How is such a busy overwhelming feed useful?

I guess that's how it looks on mobile. I've thought about making it look different on mobile, but on the other hand maybe on mobile it's especially useful to get a dense list of 20 links? I don't use my phone much so I find it hard to judge.

Time, indents, linewrap, and padding between lines may help.

https://techmeme.com/river and https://hckrnews.com/ are good clean inspiration.

I feel compelled to mention FraidyCat[0], which is offered as a desktop app. It groups posts by feed/individual, with a mini activity bar next to each. It also groups feeds by the specified intensity of your follow (real-time to occasional).

[0] https://fraidyc.at

Some multifeeds based on blogrolls:

- Marginal Revolution: https://mergefeed.net/Marginal_Revolution_Blogroll?urls=http...

- Crooked Timber: https://mergefeed.net/Crooked_Timber_Blogroll?urls=http://11...

- Naked Capitalism: https://mergefeed.net/Naked_Capitalism_Blogroll?urls=http://...

Reply on news.ycombinator.com