Hacker News Reader

Comment by streamofdigits | Hacker News Reader

How can we reverse the trend that a wonderful piece of tech such as RSS is obliterated without any regard to implications? If you have been using RSS readers you'll have noticed that over time even quality sites (not ironic) slowly remove their RSS feeds and leave only the usual social media links.

This means that they are providing their audience with no option but to have a social media account (where their interests can be tracked and cross-referenced, data mined etc), not to mention that they endorse and promote particular for-profit private companies (which is in general not done lightly, unless there is a partnership or other disclosed interest)

New open source tools like this engine, especially if they integrate more with this other wonderful piece of tech, the email client could create a more healthy information retrieval environment. The time to think anew about how to evolve a positive digital life is now and the pieces of the puzzle are all around us.

View on HN

> How can we reverse the trend that a wonderful piece of tech such as RSS is obliterated without any regard to implications?

RSS being killed is a part of the commodification/privatization of knowledge. RSS simply gives users too much freedom.

"What if we thought of some of the most lucrative tech companies as essentially tax collectors, but privately-run (and thus not democratically accountable)? Economists call this rent-seeking, and what we’re seeing with a lot of tech companies is that their telos is little more than “rent-seeking as a service”. It’s basically baked in to their business model. Once you’ve fully developed the technology underpinning your service - be it coordinating food delivery, or processing payments, or displaying intrusive ads to people who just want to read a goddamn page on the Internet without being entreated to buy stuff - then your whole schtick then becomes collecting taxes on a whole ecosystem of economic activity."

https://dellsystem.me/posts/fragments-86

You're seeing this in the podcast space, where content is normally distributed via RSS. You have big players like Spotify buying up podcast productions and making them exclusive behind their app. Luckily, right now open RSS distribution is still the norm for the overwhelming majority of podcasts, but who knows how long that'll last.

That quote is absolutely right with the MO of a lot of tech companies these days. They don't actually innovate on tech itself---that's too risky and expensive. Instead, they innovate on business models to make themselves toll operators on everyday life.

>Luckily, right now open RSS distribution is still the norm for the overwhelming majority of podcasts, but who knows how long that'll last.

Presumably it will probably last so long as advertising pays the bills for those for whom podcasts are a directly commercial endeavor. At which point they'll go behind paywalls and/or die.

Paywalls and RSS aren’t mutually exclusive. Ars Technica offers full-text RSS feeds for subscribers. Substack sites have RSS for public posts but also some subscriber-only posts. Podcasts could work similarly; I’d be surprised there weren’t some already.

Fair enough. Although I suspect that a bunch of new subscription services is not what most people campaigning for RSS are looking for.

Patreon provides RSS feeds for creators, that's how paywalls are done for most podcasts these days.

"They don't actually innovate on tech itself---that's too risky and expensive. Instead, they innovate on business models "

So true. When you look at most of the unicorns most of their tech could have been done 10 or 20 years ago.

Considering their size the innovation output of giants like MS, Google or Apple is really low.

One of the great contradictions of the Californian Ideology:

On one hand, freedom is sacred, especially freedom of speech, information, etc.

On the other hand, private property is sacred, whether it is physical or intellectual.

Yet intellectual property laws directly impede the free flow of speech/information. So what'll it be, private property or freedom?

What makes this specific to California instead of the US as a whole?

they most militantly 'defend' their intellectual property system and have also monopolized many communications networks

See here: https://en.wikipedia.org/wiki/The_Californian_Ideology

yes and this: https://www.metamute.org/editorial/articles/californian-ideo...

"There is an emerging global orthodoxy concerning the relation between society, technology and politics. We have called this orthodoxy `the Californian Ideology' in honour of the state where it originated. By naturalising and giving a technological proof to a libertarian political philosophy, and therefore foreclosing on alternative futures, the Californian Ideologues are able to assert that social and political debates about the future have now become meaningless.

The California Ideology is a mix of cybernetics, free market economics, and counter-culture libertarianism and is promulgated by magazines such as WIRED and MONDO 2000 and preached in the books of Stewart Brand, Kevin Kelly and others. The new faith has been embraced by computer nerds, slacker students, 30-something capitalists, hip academics, futurist bureaucrats and even the President of the USA himself. As usual, Europeans have not been slow to copy the latest fashion from America. While a recent EU report recommended adopting the Californian free enterprise model to build the 'infobahn', cutting-edge artists and academics have been championing the 'post-human' philosophy developed by the West Coast's Extropian cult. With no obvious opponents, the global dominance of the Californian ideology appears to be complete."

That's libertarianism, not specific to California. In principle everybody has a lot of freedom but in practice the powerful accumulate more and more power and freedom for themselves at the expense of others.

Using the state to enforce intellectual property is not really libertarian. By the way any libertarians defending the use of the state to maintain artificial monopolies and enforce regulations that advantages big organizations is a corporatist à la Federalist Society.

“Using the state to enforce intellectual property is not really libertarian.”

As far as I understand, in a libertarian world the state would not enforce property rights but you would have to go a court and sue. I think the result would be the same as we have now. The party who can afford more lawyers or lobbyists will most likely win.

I always blame chrome for demise of RSS. When chrome came, all other browser (firefox, opera, even internet explorer) had native RSS view support. To this date, chrome opens RSS as XML garbage e.g. open this in Chrome https://quakkels.com/index.xml

Chrome came, all other browsers lost, then finally Google killed Google Reader and we had nowhere to go. That's how I believe it happened.

What's still worse is that Chrome DOES NOT SUPPORT RSS natively.

Chrome doesn't care because either Google doesn't care, or because Google is actively hostile to open web technologies.

Google killed their very nice (at the time) RSS product, Google Reader, so they could push people to Google+, their solution to compete with Facebook.

I can see the rationale behind Google's (mis)judgment. Social media was steamrolling RSS in popular adoption, and seemed like the future. Sometimes, new technology does replace old technology. But there were a lot of issues with planning and execution.

>> But there were a lot of issues with planning and execution.

That's a very generous understatement:

https://www.theverge.com/2018/12/10/18134541/google-plus-pri...

https://www.zdnet.com/article/thanks-for-nothing-jerkface/

https://mashable.com/archive/google-plus-history

> I always blame chrome for demise of RSS. When chrome came, all other browser (firefox, opera, even internet explorer) had native RSS view support. To this date, chrome opens RSS as XML garbage e.g. open this in Chrome https://quakkels.com/index.xml

I tried opening this link in Chrome and Firefox, and it looks exactly the same in both.

They do both look bad now (raw XML). However at least in Firefox, there used to be some basic styling.

Remember the little blurb in the header about how "this RSS feed doesn't provide its own styling so we've added our own"?

It seems to have gone away now too.

I absolutely despise Chrome. It nags me into Google's ecosystem and it just feels wrong to use a window to the internet owned and controlled by the biggest bully on the internet - Google.

Chrome is the reason for many failures of the web experience. Support Firefox, it is equally as good IMO.

I'd like us to not see a day where we get "Only supported on Chrome" warnings.

> open this in Chrome https://quakkels.com/index.xml

Chrome just displays xml. Edge does the same. And Firefox, displays it as if it was mangled html, so no xml tags are shown.

It's too bad too. Firefox used to support RSS till recently I believe. I wonder if Pocket had anything to do with it.

Is Mozilla still even a non-profit at this point?

I never thought that I would say this, but Mozilla's failure to evolve empowering user-centric clients from the great initial success of firefox and thunderbird is beginning to look suspect.

Increasingly they look like a tired alibi for a dystopic status-quo

A non-profit that gets the lion's share of its contributions from Google.

I'm not sure if Chrome is exclusively to blame. I used to have an RSS feed button on my Firefox toolbar for news sites, and it was cleaner than having to go to the site itself for updates. This would be around 2007 or so.

I stopped using RSS altogether because at some point, every RSS button I clicked started taking me to a page with XML markup. Previously, clicking the button would add the site to my feed. I didn't understand how RSS worked, so over time I just stopped clicking the button.

The one-word answer to “why this keeps happening?” would be “advertising”. RSS makes it much harder to control ad inventory. Obviously the bigger problem here is how content creators/publishers are paid for their work.

Both Google and FB are often blamed for the current state of things, but the ways in which they impact open standards such as RSS differs.

Problem is that RSS is not in the interest of the publishers, but mostly serves the interest of readers:

* Hard to keep users on your site - in an RSS reader you just open the next interesting article, most likely from another site. So it reduces page views.

* Harder to serve ads - RSS readers show the text of the articles, not all the other stuff on the page of the publisher

* Easier to steal content - other sites have an easy way to take your content and republish it on another website.

* Harder to track your users - most rss readers just show the content, not all the javascript nonsense required to track the user.

* Harder to monetize - for profit sites like to keep their readers behind a paywall, but how likely is it that you'll pay for a new site, if your RSS feed shows content from 20+ news sites? You can't pay for all of them, so you'll most likely pay for none.

If you want RSS to succeed, it needs to bring value to the publishers.

> “If you want RSS to succeed, it needs to bring value to the publishers.”

rss does bring value to publishers: viewers/listeners. that publishers want more than that is a separate issue.

RSS is definitely a problem for publishers that depend on ad revenue and "engagement". In RSS it's difficult to develop dark patterns to attract more attention.

Viewers/listeners are not valuable by themselves. They are only valuable if the publisher gets paid for them, and they get paid via advertising (or a subscription fee). RSS does not provide advertising revenue for publishers. It could work with a subscription model, I suppose, but I am not sure there is the demand to sustain it.

you're looking at the situation too narrowly. publishers & content creators also want esteem and influence, which they can bank to get paid later (perhaps through a related but separate effort, like product 'reviews').

the short-termist, transactional view of society (you are only a good as your last payment, no long-term relation) is not natural, not required and undermines many other adjacent social contracts

publishers should support a lively rss ecosystem (actually evolve next-gen tech based on its principles) and draw revenue on the basis of visibility (and as you say, esteem and influence) this provides

people have always been paying for content. we have normalized the exception and abberation that is the "pay-with-your-private-data" business model

yes, it’s a lowest common denominator view of society that collapses all the complexity into simple transactions. it’s a lifeless way of perceiving the world.

Podcasts have been distributed via RSS forever, and it did not stop publishers from growing followers, advertise or charge for content. I call BS that RSS doesn’t work for publishers, it was killed by social media because “the feed” turned into the main way to consume content on the web.

Podcasts are different, no JavaScript based ads or tracking, no links to keep you engaged on the site. RSS does bring value to podcasts and has been successful for podcast especially since the downsides mentioned for normal websites don’t apply to podcasts.

Podcasts basically proof the point: there’s nothing wrong with rss, and publishers will use it, if it is in their interest.

I think the point is, we’ve been sold by Google and FB the idea internet advertising is “ads”, cookies and JavaScript, but it absolutely doesn’t depend on that. Publishers advertising products directly as part of content has never been a problem, it doesn’t depend on invasive technologies.

Podcasts are a very special case. They are self-contained content and have not much demand for anything around them. Any ad can be delivered inside the podcast or between streaming them. Videos are similar. Just look at Spotify, YouTube, Twitch, how many ads do they have outside the audio & video-streams?

And podcasts are even more special, because their main purpose is to be consumed without any interface, often even without any internet-connection. So the whole setup is already against classical advertisement.

FOSS front ends like Nitter provide a RSS feed for Twitter feeds. Invidious for YouTube and Teddit for Reddit also work, though the original sites in this case still provide their own RSS feeds.

Newpipe for Android is also good and available in the f-droid market.

How long until momma bird locks out whatever api nitter is using?

https://github.com/RSS-Bridge/rss-bridge

Not possible as long as the pages are not walled behind a login.

The point of Nitter is that it uses the API their official web client (which is a JS SPA) is using. They can't lock out Nitter without also locking their own frontend out.

Twitter can alter that API at any point, since they can update their front-end at the same time.

They seem to be pretty stable when it comes to their APIs. Over 5 years ago I built a small service to provide your personal Twitter main timeline as an RSS feed. Never touched it since. Still works.

https://twissr.herokuapp.com

The biggest issue with RSS in my eyes is that it tries to replace the Web instead of just being a better way of viewing the Web. The info RSS provides should be extracted out of the HTML itself by your Web browser, not a separate document provided by the content creator. It should be like a cross between Bookmarks and ReaderView on steroids. Leaving it up to the content creator just makes adoption much harder than it needs to be and is a large part of the reason why the semantic web never really went anywhere.

But as long as browser manufacturers don't really care, I don't see much chance of anything changing. Bookmarks haven't changed one bit in 25 years, despite offering so much potential for improvement. And it's not just Google's fault either, even Firefox removed that little bit of RSS integration that they had some years ago, when they really should have done the opposite and made it more useful and flexible.

I agree about the "replacing the web" observation, however I think the nostalgia is warranted in that it's people splitting hairs between the lesser of two evils.

While RSS replaced web sites (particularly aesthetically), they are less nefarious than social media companies and search companies whose only goal is to use the content of others to sell ads and their own products.

One of the indieweb approaches to feeds is to just structure the HTML sufficiently, and not have separate feed files. This works pretty well, and some feed readers work with it. Some info is at https://indieweb.org/h-entry

> How can we reverse the trend that a wonderful piece of tech such as RSS is obliterated without any regard to implications?

I'm not sure this is doable without government regulations.

I'm not sure it is doable with government regulations, as to do it properly would require politicians who're both clueful, and not captured by vested interests.

well governments (and the massive number of associated government funded sites) could start by always having RSS and not sending their citizens towards the social media platforms with gratuitous links and endorsements.

that doesn't need much regulation, just an elementary ethical / moral code

but you are right about the limited role of governments in resolving this. this is not a complex / high risk / long term project where you need them. actually just the people in this thread could probably solve this from a technical perspective.

the elephant in the room is the publishing industry. one could excuse an initial decade of them being dazed and confused, but its 2021 and they should wake up and smell the coffee.

> well governments could start by always having RSS [...] that doesn't need much regulation

But it does require them to be clueful.

Does anyone think that Joe Biden or Boris Johnson could give a coherent explanation of RSS (or any other piece of computing technology)?

you are giving the massive number of people below the figureheads at the very top an easy pass... government IT is a major, major, segment and technology, protocol etc choices they make can have huge influence.

No-one knows everything, and every national leader has to employ experts on different subject areas.

Do you think that Joe Biden or Boris Johnson know enough about computer technology -- or any technology -- to employ staff who're competent? Do you think they can tell the difference between people who really know what they're talking about, and plausible-sounding bullshitters?

Particularly when the plausible-sounding bullshitters are executives from Big Tech companies who're trying to persuade a government to do something which is in the interests of Big Tech but not in the interest of the general public?

I don't, and I think many world leaders could easily be bamboozled by bullshit. Their thought processes would be something like: "This person is from Google/Microsoft/Facebook/Intel/etc, and that company provably knows a lot about technology. They know what they're talking about, and if I say they're wrong, I risk looking stupid (as I fully admit I don't understand the technology). So my best bet is to go along with them."

Social media sites offer a simple way to share links with their circle which drives back more traffic than just one user consuming the content through RSS.

Driving more traffic means more money. It’s a self-made problem by the content creators.

thats a very good point, the ease of "propagating" the news of an RSS update surely plays a role in decisions publishers make

but its more a client-side issue, the degree to which the social graph of a person (e.g their list of email or phone contacts) is easily accessible / allows forwarding with comments etc. maybe the problem is that client app functionality has remained stagnant over decades?

also consider that a lot of that easy social media virality is actually part of the problem

Easy peasy.

Make RSS provides as much data as publisher/provider/broker want, enforce a common set of metadata when (re)sharing content so stats can made and funnels be monetized and add a mandatory opt-in et voilà.

What ?

> mandatory opt-in

What does this mean? To me, "mandatory" implies compulsory, while "opt-in" implies optional, i.e. not compulsory.

Mea culpa. I meant mandatory as in "there MUST be an opt-in mechanism for users to agree to sharing their metadata". So the privacy conscious crowd can still read RSS items and share them without automatically sharing metadata.

Without a way for authors/producers/adnetwork to extract values from the RSS format there's no incentives for them to use it. So give them what they want if we want.

Big words for what is just UTM added to the specs.

I am saying this tongue in cheek because I doubt people would opt-in if there's nothing for them in it. They opt-in for facebook and google because they get gmail/googlesearch/facebookfeed but with RSS they already have everything they want.

RSS is hard to monetize without putting it behind a paywall.

I don't think the trend can be reversed. I don't blame social media, I think RSS is incapable on the supply side as well. Quality content producers are not going to use RSS because

- HTML/CSS/JS allow for much more sophisticated, expressive presentations than simple markup, using video/images/canvases and whatnot.

- News is much more frequently updated now than in 2005. With RSS, the update cycle is not in control of the content creator.

- RSS is largely incompatible with paywalled subscriptions.

> - RSS is largely incompatible with paywalled subscriptions.

But it doesn't have to be. It really comes down to the clients having so many different ways to authenticate and therefore a complex UI. Consumers and producers also struggle with how to manage so many subscriptions.

Full disclosure: just started working for a company trying to streamline paid podcasts.

RSS works great with HTTP Basic Auth. This is the approach I'm using with Haven[1] to expose private personal blog content via RSS. In this manner, each user gets a dedicated RSS link of the form: https://name:token@example.com/rss.xml

You can even use the same tools to prevent login sharing such as checking how many IPs the URL is fetched from etc.

[1]: https://havenweb.org

I've also seen some private/paid RSS feeds just using tokens in query parameters (I think Patreon and Ars Technica do this), eg 'https://www.patreon.com/rss/foo?auth=...'

I'm not entirely sure what the benefit of one over the other is, unless some RSS readers/podcast apps have issues with Basic Auth or maybe it's just easier to fit into existing code server-side.

On the first point, you only need to include a summary of the resource for notification purposes. The idea is that the user still visits the site to consume the actual content

I am not sure what you mean with the second point, the timing of updates is under the full control of the content creator no?

The third point is quite relevant. Some sites ask that you subscribe to get updates (via email). Obviously they want to have better visibility of their audience rather than have a large set of passive (lurkers). That is a legitimate choice.

You’re missing two big ones:

1. RSS readership can’t be effectively tracked, and regular readers may transition to RSS, so RSS content becomes a black hole of readership.

2. RSS can’t easily be monetized via ads or a paywall (although some sites have managed it, like Ars Technica).

It sucks, but most sites have to make money somehow, and RSS just isn’t very conducive to that.

Reply on news.ycombinator.com