Feedmaker: URL + CSS selectors = RSS feed | Hacker News Reader

Feedmaker: URL + CSS selectors = RSS feed

feedmaker.fly.dev

173 pointsmustaphah30 comments

That is a good idea.

59 requirements, including Django, seems pretty heavy though?

For my own RSS feed, I use this 48 line Python file with no dependencies outside the standard library:

https://github.com/no-gravity/atomfeed.py

It takes an array with the entries as input, not a web page. But I guess the HTML parsing should take no more than another few lines? For HTML parsing, I have good experiences with the lxml module which is in the Debian repos. It is fast and works pretty well.

I recently added the python-feedgen module for creating feeds in my blog generator: https://github.com/oxalorg/genox/commit/3a73013ffe82930b1a7e...

I always love removing dependencies and simplifying software. I will try and switch to a simpler implementation like yours, thanks for sharing!

Glad you’re find the tool interesting! A short blog post behind it: https://kschaul.com/post/2023/04/16/feedmaker-quickly-genera...

And the GitHub url (hopefully easy to host your own instance): https://github.com/kevinschaul/feedmaker

Looks like you're hosting this on fly.io - PAYG model. You could probably host this for free on Cloudflare Workers; 100k requests/day on the free tier; static content (the homepage) is free & unlimited.

Edit: The catch is the 10ms CPU cap per request - you'd need a super lean implementation. Django's too heavy for that.

Python alone is many milliseconds to start. Unless they give you some allowances for interpreter overhead.

Well, someone already did with JS: https://github.com/ProfessorManhattan/rss-worker

https://github.com/RSS-Bridge/rss-bridge is what I've been using for the same purpose.

The good news: made it to the front page.

The bad news: so did the 503 page.

In some ways a good thing, no? Shows you've got work to do on optimisation for large audiences. A free stress test (unless you're on a host that charges per hit or bandwidth excess), as you will.

Did load eventually for me, thought it was broken as no styles but looks like it's intentional.

Seems to be hosted using fly.io

Not the same but this gives me an idea… what if there was a map reduce for doms as a web primitive. Like imagine if I could make a dom (or feed) that was some selection and transformation of another dom

You have just re-invented XLST.

*XSLT

Related discussion to remove XSLT from the web platform: https://github.com/whatwg/html/issues/11523

https://www.w3schools.com/xml/tryxslt.asp?xmlfile=cdcatalog&... give it a whirl!

In the days of XHTML, XSLT would be exactly what. The modern web equivalent would require XSLT but for HTML/SGML (SGSLT?).

I wrote a similar thing in go (using chromedriver, so it could handle things that need JS).

Handled most things nicely, but I found a few sites where I wanted multiple selections to be combined into one document.

I emailed the result to myself, turning any images into attachments; this meant my “feed reader” had read/unread tracking that synced across devices, some html support, folders, offline viewing, etc.

You can just use an XSLT stylesheet like this: https://wwwcip.cs.fau.de/~oc45ujef/misc/src/atom.xsl xsltproc includes a handy --html flag that lets you just process the source file directly.

Can you also generate+use the XSLT stylesheet dynamically from a form input so that you can use a single meta-stylesheet for multiple sites?

Oh, and is you brother coming to the party?

I made a CGI program that ran CSS selectors against URLs and returned the output. I debated making it public and then realized I probably didn't want to run an open proxy. I'm curious how long this will last.

You could just have made it public and self-hosted.

I love this.

Has anyone tested to see if it works with Blogtrottr which will email you whenever there's a new item in an RSS feed?

Just since this doesn't seem like it even includes a date field in the RSS? And of course no guid. So I'm wondering how compatible it winds up being.

Dates shouldn't matter. The feed has ID elements which is what identify entries. Atom has no guid element. So I would expect this to work with any reader.

I wish they had concrete, accurate id and created_at. IIRC these attributes are fixed in AT.

But is this producing ID elements? And if so, based on what, since they don't seem to be coming from any CSS selectors? That's my question.

It seems to use the link as the ID based on clicking a few examples on the site. An ok option for this type of thing.

Should be able to achieve this without selectors with HTML to Markdownish (something like Firefox's Reader mode).

Oh, so this is what Reader mode does.

The few times I actually tried it, it worked badly, with huge chunks of text content missing from the page. Makes me wonder if with modern web the task has became so difficult even a browser couldn't pull it off, or if they just wasn't trying to do a good job with the feature.

Same can be done wirh freshrss

If you're serious about RSS curation and reading, FreshRSS really is the Swiss Army Knife. It does so much, including this“any site/page to RSS”. My favorite feature is that it makes refreshes in your feed reader client pretty much instant, which is such a huge quality of life improvement.

Reply on news.ycombinator.com