How RSS beat Microsoft
buttondown.com
buttondown.com
Case in point: I saw someone had unsubscribed from one of my email newsletters, and when I went to go read the "reason why" field, they'd filled out: "subscribed to the RSS feed instead."
That's right, my email newsletter has an RSS feed (thanks Buttondown!), and they prefer to receive the newsletter that way rather than via email. And can I blame them? Absolutely not! I love RSS. Is it better for my vanity to have their email address in my database instead, rather than some nebulous XML file going out to who-knows? Of course. But again, this format keeps on winning year after year because it's one of the best consumer-first features of the open web.
I thought the comment says "it was an absolute dream for consumers" but actually it says "it's". Sorry to burst your bubble, if you ask any normal person who does not spend 10 hours on HN per week, chances are that they have never heard of the term RSS in their life.
I still use RSS but I am completely with you on this. Most people, even those who work in tech related fields very rarely know what RSS is and when you explain it to them, they mostly don't care.
I have had a very nice example where I explained what RSS/Feedly was to an interested graphic artist/designer (pretty competent with computers, spends all day on it). Once she got it, she told ME it was basically the same as email newsletter or subscribing to a Facebook page, Twitter feed or things like that. Which of course it is.
So, she couldn't be bothered to try the thing and set it up. I completly get it. For those who are used to it, it makes sense. If you already have your subscription stuff managed elsewhere, it's just more work for little benefits for most people.
What's more RSS actually inverse the responsibility in the relation, and you become the one who has to manage long term changes. Some feeds change URLs and I don't even notice it and you need a specific software to read them, which might be paid and might change overtime. And unlike emails, you can't even archive it easily, you need another software or a reader that has the functionality.
It's just a hassle and the reason most people don't care is because they can get the same benefits without paying for anything or having to manage/learn new stuff.
RSS is just a technology that matches an idealism about the world, very prevalent in the optimist technophile sphere. But the reality is that people don't want to make that much effort for some dubious "advantage" and they are right.
- Some major platform still provide RSS, which makes me use them (I do not use twitter, because it does not provide RSS
- If not for RSS I would not be using Reddit
- the moment platform drops RSS, I drop the platform
Links:
[0] https://github.com/rumca-js/Django-link-archive - my own RSS reader
Viewing the source of a subreddit on old.reddit.com shows an RSS link; viewing it on the new domain does not.
But I also have a child & a job, so in general my available time to goof around online is less.
I can imagine an alternate timeline where Google Reader turned into a sort of Twitter (or FB or IG) feed.
That's a silly thing to say. Of course you can put ads in it since it allows linking to things. What you mean probably, is that it's not as easy as embedding some google ads markup in your sidebar.
Advertisers love to burn money, but they draw the line at not being able to verify that the spend did what was promised.
With email, you normally use unique image and link URLs for each recipient, so you generally know who's opened the email and what they've clicked and can map that to their email address and whatever other information you have about them.
With RSS, you generally don't have any information about who's accessing the feed other than an IP address. It is possible to require users to log in and receive a unique RSS URL, which is what podcasts often do to give paid subscribers access to paywalled episodes, but that's not common for web RSS.
Even the injected ads idea was tried with companies like FeedBurner, later acquired by google.
If you're generating feeds on the fly with tracking metadata based on the requester, you can identify aggregators, and treat them equivalently to social media platforms where users circulate normal web links. You still get click-throughs to the underlying content from the end users, and you'll know the aggregator was the referrer.
Many podcasts use direct ad injection using metadata from the request to the enclosure link -- that seems to work well enough, and seems like the sort of thing that could be used for other content than just audio and video.
Let's not. Please.
If RSS has been more common, I imagine the bigger RSS readers (bearing in mind one of them was from Google!) would also have standardised on other ways of tracking clicks and ad views and all the rest. There just weren't enough people interested in RSS to make any of that worthwhile.
I say this as a user of RSS and someone who publishes a (very sporadic) RSS feed. It's a niche, because most people don't want to curate their own feeds.
Podcasts inject ads into the content: from RSS you get the link and description of the episode, and inside the episode are ads.
I guess that's why RSS is still a thing for podcasts? :-)
Yes, but mostly because of a lost opportunity.
I was working on my own web based reader when Google made a significant upgrade to their reader. It was similar to what I had made, so I thought it would be foolish to compete with Google and stopped working on it.
I wonder where RSS would be now if Google had not discouraged potential competitors.
Polling rate also has nothing to do with frequency of updates if you care to receive those updates in a timely manner. I haven't seen a reader default to 30 minutes or less.
Probably in both cases you just notice the bad implementations more because they make more requests.
And Atom supports pagination so you can limit the main feed url to be just one entry while still allowing for clients to retrieve older ones.
https://www.earth.org.uk/note-on-site-technicals-99.html#202...
I'm a small enough deal that I can call out specific readers at the top of a new post. A couple weeks ago I put up a post that started with the aside "hey one of you has a Turkish IP and you're pulling the whole feed every 10 minutes, please stop doing that."
(The podcast where I capture primarily audio material such as voice or sonfification that might as well also be presented in this alternative channel form...)
I'd be interested to listen to your podcast if you were to ping me its URL via one of the contact methods in my bio...
Readers don't know how to reply to the author in a standard way (like an email)
Paying for content is a conscious action, it has a higher activation threshold than just clicking mindlessly on something that looks fun.
Then, transactions are expensive; micropayments are not a thing.
Subscriptions alleviate that a bit. Large middlemen alleviate that even more: Apple and Google can make micropayments like $0.50 viable within their ecosystems, so apps or in-app purchases can be tiny, and allow to remove ads for paying users. Attempts to do something similar for websites never took off, sadly.
I host freshRSS and it's been amazing for me.
Well, RSS won the battle, but lost the war.
Is it a popular main stream thing? No. Does every since site offer feeds for every reasonable thing you could want to subscribe to? But does it still work quite well for those that want to use it? Yes.
For apple ecosystem best client is https://reederapp.com/classic/
Are there good tools to RSSify sites that don’t have one?
You can link it to your reader so you just click the button and it adds the feed into it.
Any half-decent feed reader app will do it for you after just pasting the website’s address.
> Are there good tools to RSSify sites that don’t have one?
https://createfeed.fivefilters.org
And for newsletters:
For the first question, I should clarify that I'm hoping to just ingest these RSS feeds myself in various scripts. But yeah, makes sense that most of the good feed readers mostly take care of that.
Browsers used to detect this and show an RSS icon near the address bar if the website you were viewing had a feed - and you could click the icon to see more details and subscribe.
I use this Firefox addon which replicates that functionality: https://addons.mozilla.org/en-GB/firefox/addon/feed-preview/
FreshRSS is a good self-hosted RSS feed reader, and you can configure it to scrape non-RSS webpages for updates too: https://danq.me/2022/09/27/freshrss-xpath/
Discord, WhatsApp, and iMessage are all messaging applications that aren't directly related to the use case for RSS.
That leaves Twitter and Instagram as the two major sites for which RSS would be applicable, but which don't natively offer RSS feeds. And a cursory web search reveals the large number of solutions people have come up with for subscribing to content from Twitter and Instagram via RSS, indicating that there's significant demand for it.
It's also worth noting that with Twitter in decline, the main competitors gaining traction, BlueSky and Mastodon, do both natively offer RSS feeds.
On top of that, the entire podcasting ecosystem is fundamentally based on RSS, and it's still the primary mechanism for syndicating blog content.
So RSS is not just alive and well, it's thriving.
Many times this sort of meta information reveals much more than expected
The last time someone tried to convince me this was a good idea was just after the iPhone was announced, and before everyone and their monkey had a super computer in their pocket. It seemed like a good idea at the time, so we almost started - but my advice to the punter then was "lets see what the mobile phone industry looks like next year" .. well that put a pin in it.
Nowadays, I'm not so sure I'd be so willing to do this - again, because it requires the user do the printing - but if you were to, say, make this into a vending machine product, which users can walk up to in the street and walk away with a custom 'zine full of their own interests, you might be onto something.
Here in Europe we have a lot of old telephone booths converted into mini neighborhood free libraries. I've often wondered whether it would make sense to put a public printer in those libraries and let people print things .. seems like this would be a revolutionary new product to make, with printable broadsheets based on a custom RSS, an obvious killer app .. assuming someone can be found to maintain the printers.
(Off to find thermal paper for my ClockworkPi, which I always wanted to turn into a custom RSS printer in the toilet...)
My feeds are pretty unpredictable - sometimes I have 40 new articles in a day, sometimes just a few. The cheapness of digital consumption and interface makes it viable for me to skim titles and read, defer, or dismiss at my judgement. I don't want the entire feed printed out - not viable.
But if some SaaS is curating my feeds for me, I fear it'll turn into another algorithmized something optimizing for what exactly? At least the first-pass filter is explicitly set by me - feeds I subscribe to.
Curious to hear your thoughts on it, and wishing you luck.
I am sure people use RSS in many different ways though, it just doesn't seem useful to me.
I suspect maybe it's easier now to nail the layout if ai can read content before it goes to print.
AI is indeed a crucial part in solving the two most difficult challenges -- typesetting and curation, although we'll probably do things that don't scale for a little while before fully automating.
Recently I’ve been living in a cottage town and thought of this idea again… rather than be reading on phones or tablets people could read printed books with their favourite articles or blogs. But I think the actual distribution system would be the killer, unless it’s at a big resort the transportation will kill the idea.
People confused Betamax with Betacam, Sony’s professional grade recording medium, which is absolutely better quality.
People conflated VHS’ ability to slow the tape for even longer play at the expense of quality. That of course made the recording terrible. Betamax did not initially have this capability.
People listened to Sony’s own marketing. When they couldn’t compete on features, they banked on their reputation.
That was it
It's (a part of) what killed MiniDisc too. It was _the_ coolest format ever invented, but Sony gatekeeped it so hard nobody used it.
Nothing more Cyberdeck-y than using a MD to transfer data :)
"When Betamax was introduced in Japan and the United States in 1975, its Beta I speed of 1.57 inches per second (ips) offered a higher horizontal resolution (approximately 250 lines vs 240 lines horizontal NTSC), lower video noise, and less luma/chroma crosstalk than VHS, and was later marketed as providing pictures superior to VHS's playback. However, the introduction of Beta II speed, 0.79 ips (two-hour mode), to compete with VHS's two-hour Standard Play mode (1.31 ips) reduced Betamax's horizontal resolution to 240 lines.[7]" https://en.wikipedia.org/wiki/Videotape_format_war#Picture_q...
If you want to collect obsolete formats and you have a TV with analog inputs VHS is probably your best thing to get into. This place
https://mastodon.social/@UP8/114286077399818803
sells VHS decks for $12 and you can get pretty good movies for $2. Contrast that to compact cassette decks which start at twice that and have a good chance of being non-functional. That place has the complete works of Barbara Streisand but if you want music that anybody would want on cassettes the sky is the limit for collectables.
My impression is that the quality of VHS isn't terrible. The video is worse than DVD of course but a lot of DVDs have NERFed soundtracks because they mixed them assuming you're going to play their 5.1 mix on a 2-channel system. Any deck you get now is going to support VHS Hi-Fi and if you have a 5.1 system with some kind of Dolby Pro Logic the soundtrack of a good VHS can be better than the soundtrack of an average DVD. (Blu-Ray often has better sound not because the technology is better but because the 5.1 soundtrack is more likely to really be a 5.1 soundtrack)
Beyond this, is when they bake a 16:9 movie into the 4:3 format losing significant fidelity. Batman Begins was nearly unwatchable.
This of course doesn't get into the sound quality/mixing issues you mention... I wish they had something closer to h.265 at that time, as I don't mind a blurry background nearly as much as blotchy/blocky artifacts for similar sizes or smaller. A 2gb h.265 movie from blueray looks dramatically better than a 4+gb DVD movie.
> Any deck you get now is going to support VHS Hi-Fi
When you say "VHS deck", do you mean something other than a VCR?
As far as I can tell, it's become the "de-facto" for Anthropic related RSS feeds.
You'd think RSS was dead, but I release this earlier this year and it's at 100 start.
2 – It feels like RSS is one of those topics where the same old observations and opinions are raised, but nothing new is ever tread for or against it.
I’m not sure whether these works address my interest for the former, but I think they’re cool.
I think, as someone that has a RSS feed on my blog, that RSS is a total mess and Atom was probably the better choice.
Maybe even some modern JSON based format would be OK, but maybe that’s what ActivityPub is?
Anyway, after dealing with the mess of images and inline HTML with CDATA in RSS, I have complete fatigue of the whole endeavour.
RSS works. Atom splitting the standard into two probably did more harm than good. In the end it doesn't matter since every reader supports both and both do the job well.
That’s what JSON Feed is. It’s supported by several RSS readers.
> but maybe that’s what ActivityPub is?
No, that’s for social networking.
Beyond this, maybe a framework to show a single header ad on the reader giving the revenue credit and money to the original content site.
The reason for newline separated json, is simply that you can do a partial content download in the reader... the most recent 100kb or 2mb or whatever... you the most recent is on top, and allows a site to publish more than just the most recent, but you don't have to grab that. Or maybe just standardize a since=(iso-style-datetime) or last=## (number of articles).
Just a couple loose thoughts on this.
People found the web more boring, because it became more boring.
They found the algorithm more interesting, because it allowed them to see what was going on with people they barely knew (from former school mates they'd lost touch with to celebrities without press filtering), and that was compelling.
But there's a next phase available to us, which is to make the web more interesting, entertaining and compelling again.
I love that b3ta.com still exists. I love that metafilter.com is moving on. I think it's great that web comics I love still publish to RSS.
I just think more of us need to provide more demand, and more people will wake up to supply, and the flywheel will start to turn.
RSS beat ICE, and it can beat Meta and X if people want it too, albeit for different reasons.
RSS did not weather Twitter. Social media is huge compared to RSS. It turned out that singular recommendation feeds are able to push URLs around better than needing every site to build in feeds themselves and then still requiring someone to turn those feeds into a singular feed for the user.
They still are in most cases.
Then I remembered that Twitter was once referred to as "micro blogging," so I put those folks in my blog list on Feedbin, and was happy again.
I do miss the glory days of Twitter, tbh.
First, RSS has a bit more friction. Smashing the follow button on Twitter et al is faster than adding the feed to your RSS reader of choice unless your OS has support for default RSS app.
Second, discoverability. Just like with any distributed system vs monolithic platform, you need to find what to read yourself. For some niches this works well. If you are a software developer/hacker, you are more familiar with blogs in your area of interest. But if you have a wide range of interests you’d need to find the blogs yourself and hope their RSS feed is well formatted.
Third, the algorithm. A monolithic platform can do more to try to mix in new content based on your interests and intelligently mix up the content from sources you follow. This is of course controversial because feed algorithms can also try to cram bullshit into your feed or hide important stuff from you or create an echo chamber. But in the best case scenario they can also expose you to new sources of content you wouldn’t have found otherwise. An RSS reader would mean it is up to you to do this discovery which is more friction.
And ultimately content creators realized that they get more eyeballs on their stuff by using platforms like Facebook, Medium, Instagram, Twitter, than on blogs especially since blogs tend to be then repackaged by blog spam bots, Google’s AMP, and now LLMs.
So IMO RSS is just too manual and requires too much work. And of course since you can’t effectively advertise through it there is less incentive for creators and platforms to support it.
(In contrast, ICE did not weather RSS.)
Meanwhile, RSS is barely relevant today. For decades (Youtube turned 20 this year), people have had access to feeds curated by "the algorithm" operated by a commercial interest (hoping to maximize the amount of ads you look at); and most people seem to prefer it that way, if they're even aware of alternatives.
Ironically, my microsoft feeds are pretty active, and xkcd is still there, The Daily WTF is still going strong.... but a lot of my feeds are just dead.
Bluesky is basically RSS on JSON.
A dedicated extension is needed to have that feature back. Chrome needs one as well, so does Edge; only Vivaldi and Opera come with build-in feed readers. There are of course standalone applications but that seems to be a niche nowadays.
I've found an old rssowl opml file from 2014 last week and I decided to see what's still up. I've found some RSS readers in flathub but sadly, majority of what I was visiting back then died.
I still use rss daily for keeping up with we bsites, but now it is for 99% of internet users something tech embedded in an application, or just not heard of.
So in that sense, the poster child of "Web 2.0" was taken out back and kneecapped.
To consume an RSS feed you poll it. There are two polling speeds: too fast and too slow, and it's possible to be both at the same time.
Note the struggles of this Karen to turn RSS from a simple stateful protocol to a complex stated protocol, and she'll ban you if you ever reset your cache and rerun your fetcher because your cache got corrupted or you suspect it might have been corrupted.
http://rachelbythebay.com/w/2022/03/07/get/
You really want to have a stream of feed items and to be able to: (1) replay the whole stream all the way from or to the beginning and (2) query "what got added after time t?" and just get that. ActivityPub accomplishes this but people don't really like it. For Dave Winer it is all blub but even if he doesn't believe in the Fedi, he's on it.
I really like
because it does all the polling for you and hits your webhook whenever a new feed item appears. My webhook is about 15 lines of Python running as a Lambda function that posts items to an SQS queue and my YOShInOn RSS reader just drains the queue at its convenience. The pricing at 10 cents/feed/month is a bargain for high volume feeds like MDPI, arXiv, and The Guardian [1] but unfortunately I can't really afford to subscribe to 2000 little blogs that post maybe once a week at that rate. I wish there were more Planets.
https://en.wikipedia.org/wiki/Planet_(software)
[1] AWS costs would be trivial in comparison even if it got out of the free tier
https://datatracker.ietf.org/doc/html/rfc5005
Http might be better in 2025 than it used to be but historically the cache is as much a problem as it is a solution. That is, when you have heisenbugs the answers are frequently "clear the cache" [1] and "don't use the cache" [2] and it's often a better strategy to "rip and archive the whole site right now and come back in six months and make a fresh rip" if you're working on a large scale and can tolerate being up to six months out of date.
In general database-backed sites struggle to implement if-modified-since since a fully correct implementation has to traverse the graph of all database objects that are looked up in the request which costs about as much as… the request. Your cache needs a cache and you can make it work at the system level if you have a modified date cache and always invalidate it properly. If you are doing that you might as well materialize a static web site fronting your site the way the Wordpress supercache works —- then you’ll find your site crashing a lot less!
I’ll admit ActivityPub is complex but http caching is complex in a particularly poisonous way in that there are so many alternate paths. An ActivityPub system (client+server) could be 100% correct but an http-based system might not have a clear line around it and might never be quite right. A stateless system could run for 10 years without trouble, it might take you 10 years to realize a stateful system was feeding you corrupted data.
[1] fix it... for now
[2] ... and they're no longer part of your life
That's only true if your implementation favors editing performance over GET request performance ... in which case you get what you ordered. If your feed requires any database connections then that's a choice you made.
I disagree that HTTP caching is complex. The basic primitive are:
- An indication until when the returned document can be used without checking for updates.
- A way to check if a cached document is still valid (based on either modification time or id).
This provides everything you need to implement an efficient feed reader.
If anything characterizes the RSS community it is a lack of imagination and rejection of the last 25 years of work in relevance, ML and UX. RSS readers are still failing with the same failing interfaces that failed 25 years ago.
That said, the people who are still using it feel satisfied and other than Rachel that aren’t a lot of people who will try to stop you if you do try something ambitious. You just gotta poll poll poll and poll poll poll some more or pay somebody to poll poll poll for you.
Correct me if I’m wrong, but is Winer’s somewhat recent effort with FeedLandⁱ any different from Planet?
(2) I really am mad at Rachel for this and that's from someone who's been writing webcrawlers [1] since 1998, been an RSS innovator, and been responsible for complex systems when they fail.
(3) Maybe I am missing it but I don't see an actual feed for that URL, I see only an OPML file. Dave is really gay [2] for OPML files but I'm not because I still have to work to fetch all the items. Yet, visually the OPML file and blogroll look like a planet and you're not the first person who's pointed Dave's blogroll as a solution as opposed to the problem that I see it is.
(4) Looking at the head of the list I think "Daily Kos" and "404 Media" suck but that I already subscribe to many of them like "Arstechica" -- looking at the tail of the list I see there are gems that I'm not getting. If those things were getting aggregated topically it would please both me and Rachel.
[1] that don't crash your server
[2] in a good sense!
I think I catch what you’re getting at and it reminds me of what seems to be a primary gripe with RSS—compromising an otherwise decent spec as far as I can tell—discoverability and curation.
I can’t grasp the more technical issues surrounding polling just yet so I can only intuitively get how it corresponds with the other issues. In spite of this, and leaning again on my intuition, I didn’t get a good feeling from Rachel’s blog post when I first came across it either.
And yeah I’m not sure how FeedLand works under the hood, I just matched Wikipedia's description of a planet with what I knew FeedLand looks like on the frontend.
I also know there’s been some “folksonomic" efforts for curating lesser-know feeds, but to be honest I’m not fond of what these networks of folks appear to be into. Compare your distaste for Daily Kos and 404 Media (which I share) to the vibe I’m trying to avoid on that front.
Don't know what you're so angry with Rachel about. I provide feeds for readers and many of them are way too greedy and frequently do hammer websites, often unneccessarily. Mix in all the rampant AI bots and you got a recipe for an extremely expensive server bill.
Getting angry at someone who's trying to keep their server costs down to provide you something for free is kind of weird.
- I've looked at the abyss of bankruptcy from server bills. I actually think it's a hell of a lot worse than Rachel does, I thought that 15 years ago, and I've suffered worse
- The whole discussion around RSS has been naive in every respect from the very beginning, for instance Dave Winer thinking we care about him publishing a list of the tunes he listens to back when you couldn't actually listen to them (I'll grant that in the age of Apple Music, Spotify, YouTube in such things may have caught up with him.) There are never any systems thinkers just people who see it all through a keyhole and can't imagine at all that anyone else sees it another way.
- To use a Star Wars analogy, Google is the evil empire that blew up up our planet with a Deathstar 10 years ago and now we're living on the wreckage in an asteroid belt. I see the AI upstarts as The Rebel Alliance that at the very least reset the enshittification cycle back to the beginning and create some badly needed competition. Opponents to A.I. crawlers are brainwashed/living in the matrix and defacto defending Google's monopoly and slamming the door to prevent the exits from enshittification that A.I. agents offer (e.g. they look at the content and drop out the ads!) They think they're the rebel alliance but are really the stormtroopers suppressing the resistance until the Deathstar charges up for the next shot.
- Rachel's supposedly some high powered systems programmer or sysadmin or something but is just as naive as Winer. We're supposed to "just" use a cache. Funny, the reason why the web won out of every other communication protocol is that you can "just" use curl. curl by it's nature can't support cacheing because it's a simple program that just runs in one process and if you wanted to have a cache you'd have to have a complex locked data structure which would force a bunch of trade offs... like I am downloading something over my slow ADSL connection that takes two hours but I also need to clear my cache so do I force the download to abort or hang up the script that needs the cache cleared for two hours? curl is "cheap and cheerful" because you just don't have to deal with all the wacky problems for which "clear the cache" or "disable the cache" clears up like a can of Ubik. But in the age of "Vibe Coding" solutions that almost work are all the rage, except when you finally realize they didn't actually work you clear your cache and rerun and BAM you got banned by Rachel's blog because you hit it twice in 24 hours.
Web browsers are some of the most complicated software in existence, ahead of compilers (they contain compilers) and system runtimes (they are a system runtime), right up there with operating systems (less having to know things that aren't in the datasheet to write device drivers at least.) For the first 15 years you could not trust the cache if you were writing web applications, somewhere around 2010's I finally realized you could take it for granted the cache works right. I guess implementations and the standards improved over time but all this complexity is some of the reason why it is just Google's world and we all live in it and there are just two big techs that can make a browser engine and one unaccountable and out-of-touch foundation.
So I wish Rachel would just find an answer to the cost problems (Cloudflare R2?) or give up on publishing RSS or advocate activitypub rather than assume we care what she says enough to follow her rules for her blog without seriously confronting what a system-wide solution for the problems RSS tries to solve and the problems it poses.