RSSHub: Everything Is RSSible
github.com
github.com
A plug for my own answer to this:
I recently launched https://aboutfeeds.com -- a one pager "getting started" guide aimed at first-time users of web feeds/RSS. The goal is to have it linked next to every "RSS subscribe" button out there.
We both opted for "web feeds" over "RSS", which makes me feel better about that choice.
Do you have any sense of whether you've been able to reach new users? It's going to take a lot of effort from us and people like us, I think.
My belief is that, in order for RSS / ATOM to really catch on, we need to completely abstract those concepts for the end-user.
In my opinion, getting people to care about RSS / ATOM is like getting them to care about HTTP. They do not care about the actual mechanisms - only the end result. That being said, I think all we're missing is GREAT product design. Too many pipes are still exposed and it scares away the average user.
Make webfeeds as intuitive as social media giants and we have a revolution on our hands :p
PS: would love to build this with someone if there's any takers :)
I think basically you want reddit's early marketing, "the front page of the web". Except it's your front page. It's a feed where you can go to see the latest stuff from everywhere you're interested in, rather than having to check each site individually. Like how one's reddit home page shows posts from all the different subreddits to which one is subscribed.
I think the best entry point is probably the browser, like Firefox's "top sites" on about:home. The browser is uniquely positioned to know what sites you frequent. If it also knows where to find those sites' rss feeds, it could automatically suggest adding them (also on the home page or similar). Once you've got that -- people actually using it -- you're over the hump, and it's easy enough to for users to transition to curating their feeds, using a different application, etc
The current UX, at least on my iPhone, is that I go to a site that looks like it _probably_ has a feed, and then I use the share sheet to push that URL to NetNewsWire. Then RSS auto-discovery takes over.
But there are a growing number of sites that look like they _should_ have feeds (posts on the front page arranged chronologically) but don’t. And the user experience is simply terrible when you attempt to subscribe with auto-discovery and it silently fails.
The problem is getting that in-browser glowing button on mobile. Browsers are too locked down.
Yes, but just having that button isn't enough. Consider: I'm someone who understands the benefits of RSS, and despite seeing that button for years, I rarely clicked it, and mostly wasn't sure what to do after I did click it. Because that button showed you the feed, but the browser didn't really have a centralized reader built-in (that I remember, anyway, and if I couldn't find it then you bet the average user can't, either). There's a big difference between just showing the feed and having an "Add stories from {{newspaper}} to my home page" button.
On top of that, a big problem with they are walled gardens as well. Some support exporting easy enough, but we've seen firsthand what happens when big readers (google reader, etc) go down.
That being said, I think our best hope is to create an open-source, web-based reader.
raw feeds are one of the last bastions of freedom on the internet, and we can't afford to keep building them on bad foundations.
If I had to imagine it from the ground up, I'm picturing a desktop-esque environment running straight from the browser. almost like google's environment, to be honest. There could be full-blown search, email, news, etc; but they are all intertwined by the ability to 'subscribe' to any of these results, and have them piped right into your homepage.
Sort of like smichel said above, a true front page of the internet, but your front page.
unfortunately, I think browsers will continue to be locked down for the foreseeable future. given the locked-down nature of the app store as well, I think our best bet is building an open source, web-based RSS feed.
What’s unusually about the feed ecosystem is that it was once popular, and now it’s present but nobody really understands it. Like, Wordpress sites are 30% of the web and they have RSS by default... but are the site owners really aware? It would be a hard push to get any of them to change.
And, for most technologies, the existing users are the best word of mouth evangelists. But here, the existing users have their own favourite apps, and if you change anything too much then they won’t evangelise.
So (imo) growing the ecosystem means finding ways that respect that situation.
it never caught on because it was a protocol. not a product. and yet stubborn users for years insisted otherwise.
it's like if the internet did not have a browser; only a textfield to input urls. who would use that?
Still the UX has to be excellent. As you wouldn't abstract the fact of a sharp edge off a knife. Still the grip has to be very different from the sharp edge. It's 2 things that have to balance to make sense. The worse limits the result.
No user ever is bothered with Atom spec technicalities, is it?
You are 100% correct with this. But not only that is the problem. There are too many sites that don't offer obvious RSS feeds. Like YouTube. I don't have a google account, but I still want to follow some channels. What would be the best way? Right RSS, but YouTube doesn't offer that really obviously. So here is how you follow a youtube channel with RSS:
1) Klick on a YouTube Video from a channel you want to follow
2) From there click on the youtube channel (Doesn't work if you directly click on the channel from the serach)
3) Then you have the url https://www.youtube.com/channel/<channel-ID-here> (for example colinfurze's channel ID would be UCp68_FLety0O-n9QU6phsgw
4) Now the last thing left is combine that channel ID with the following: https://www.youtube.com/feeds/videos.xml?channel_id=<channel... and add it to your RSS feed! (For example the full colinfurze url would be: https://www.youtube.com/feeds/videos.xml?channel_id=UCp68_FL... ) Easy, isn't it? :)
I myself just wrote a little script to turn youtube channels, playlists and searches into video feeds that I can subscribe to in my podcast player. I love rss.
Podcast: I’m not close enough to this world to know. Has podcasting answered the discovery question by centralising directories, or are podcasts still discovered “in the wild”?
1. What is a feed? (a.k.a. RSS)
ATOM, in many, but not all cases, is superior, since it allows for easy extensibility. Should people be made aware of Feed technology, I would love them not learn to associate it with RSS, but understand, that there is also ATOM, which even has derivates, namely OPL and ODPS. There also exists a PubSub type API for ATOM.
Thank you.
A lot of the software I use and contribute to has gained many contributors from China, Singapore and Japan through the last 1-2 years.
It really warms my heart to see that FOSS knows less and less borders :)
But I also immediately feel concern. I hope this Shanghai developer is not targeted, bullied, or persecuted in any way like having their money seized.
Damn. Just donated to their PayPal because I feel so encouraged by it.
Is there a case where we hate the people ? Are people as a collective harmless.l ?
You don't get a "get outta jail free" card for saying "I love CHinese people" and "oorah for China devs" and then heaping shit all over their country's achievements the rest of the time.
Do people who say this think they understand China or Chinese people even a little bit?
I didn't say that anywhere. China's achieved a lot, in amazing ways. My comment said nothing about that.
> Do people who say this think they understand China or Chinese people even a little bit?
- Have you been to China, invited by their government in an official delegation and seen first-hand the mob-like lack of distinction between business and government, the nepotism, and the opaque vista of corruption, like I have?
- Have you had more than one Chinese national as a romantic partner, like I have?
- Have you worked and studied extra-curricularly with Chinese people for much of of your life, like I have?
- Have you learned Mandarin at age 18, like I have?
I think I "understand China or Chinese people" more than the average non-Chinese person.
But even that I think doesn't count for much. Chinese people don't share their real cultural opinions with people in the West much, including my ex-partners.
I still have lots more to learn about the very contrasting popular opinions there and slowly understand some of it. I respectfully listen to pro-government Chinese people who occasionally appear on HN, and engage with them with an open mind, though with no concession granted in regards to expectation of intellectual honesty. Recently in my comment history, there's one such conversation. I gave a very unforgiving rebuttal to their comment, but I respected them by not just downvoting, but engaging. We really should talk more.
It appears you're a non-Chinese person who's been triggered by my comment based on other experiences. I really invite you to not 'other' individual people like me as faceless "people" like you just did. In contrast, it would have been great to hear from a thoughtful Chinese person.
You're racist because a racist said so. We're all equal.
It sounds like you've confused "identity" with opinion. I wasn't saying, "you are bullshit", and I'm sorry that you seem to have taken it like that. I was saying "I think that (view you write) is bullshit". I wasn't trying to offend you.
But also, to assume that had I had the experiences you list, I would arrive at your conclusion is incredibly narrow minded (and, to be honest, arrogant), and makes me trust even less that you would be able to see beyond the nose of your existing biases and consider others with different views... Because it's a big world out there, plenty of room for differences of opinion and experience.
The false dichotomy between "the Chinese state/government and the Chinese people" is a common Western propaganda tactic I've noticed to, exactly as I said, seem like you're not being racist when you are. And, unfortunately for you, god bless you, you've done exactly as I predicted, "I'm not racist because look at all the connections I have"
You didn't go on to criticize anything beside saying you don't "trust" the government, but I'm great at reading between the lines, and reading people and I know the type of argument you're hinting at and where it goes.
It reminds me that people don't hold these anti-China views from ignorance, simply because they've carried a Western-propaganda mindset into their own affairs. It seems like in your jaunts you forgot to check your pre-existing biases at the door/luggage-check counter.
Which, so far, has meant, not that you don't have enough experiences to think about and understand China, but only that you've so far failed to properly think about them, being blinded by your biases. So far, you've wasted those experiences you have by not yet thinking clearly about them. I'm not sure where you're from (and I don't care here because it doesn't matter, like it doesn't matter where I'm from or who I am, what matters is the quality of my thought and my ability to think clearly, critically and for myself, as I hope you can too), but here's a video of a white dude saying something that is mostly correct. I don't know him nor anything else about him but this is about right:
So, contrary to the popular, but incorrect anti-China notion that you can't trust the Chinese government, I think you can trust them more than any other place on Earth right now to deliver as they say, and to deliver results for their citizens. Simple as that, really. And I think most Chinese are proud and happy to have such a system.
This shrieking Western hysteria smacks (to me, anyway) of bitter nostalgia for imperial glory-days where we could pull off stunts like "the treaty of Nanking" and "the Opium War", the "United East India Company" and wiping out native populations of N and S American, and Australian aborigines.
I'm not anti-West. I'm just balanced between both places. And speaking up against the river of fake and negative opinion, which, in my view, only serves to hasten the West's demise by blinding them to what they could learn, and giving them the fake pay off of feeling good by (doing that old colonial thing) of putting other people down and pretending they're better.
But to bring it back to you finally... if you want to make it about identity, then sure, I'm just more cosmopolitan than you are. In your future jaunts, I hope you're able to see beyond the veil and check those implanted biases with other stuff you don't need. So you can finally see clearly. Best of luck! :P ;) xx
It sucks that I can't even say 'I love Asians' without being tagged as racist. It sucks that I can't have prejudice against or for some people. We are not all equal, and if you pretend we are, YOU are racist. I recommend the sketch 'I don't see race' by collegehumor: https://youtu.be/5qArvBdHkJA
I repeat what I said countless times the past few years: A country is not its people. Even if you pretend it's a democracy.
That said, I hate America for ruining the world's economy and environment. I love most Americans though. Hang in there.
I used Seafile and it's great but it has a pretty hard paywall.
Gitea is just great. In conjunction with Drone it's the most elegant CI/CD platform I know.
I wonder what platform they used before but I don't know any of them personally. I only read their Github bios.
I mean, I’m a Japanese but how I notice a developer is a good fellow countrymen is usually through noticing telltale signs and chasing them down, not because there’s a filename in the language I speak at the root of a repository.
I always love to find out more about (development) culture in other parts of the world. Can you elaborate a bit on how it's 'bold' to use your own language in a README or docs? Do you find it annoying?
Edit: Actually just noticed your comment was about anime girl avatars, and not the project quality. I have no opinions on that matter.
It does support atom. You need .atom at the end. By default, it's rss V2.
Atom is interoperable and extensible and great care was taken during design and implementation testing to achieve these goals.
RSS, OTOH, is garbage on fire ruined by one man's ego.
https://web.archive.org/web/2004/http://diveintomark.org/arc...
———
¹ Except podcasts. Have I mentioned them? They’re still frozen in 2004.
———
¹ And seriously the whole podcast RSS scene is full of “well RSS is bad, so let’s add our own elements to fix it” namespaces, so that you’re writing the same thing four or five times, where Atom would have obviated at least some of those.
RSSHub conforms to RSS 2.0 and Atom Standard, simply append
.rss .atom to the end of the feed address to obtain the feed
in corresponding format, default to RSS 2.0.With the Feed Creator feed URL I'm not able to see the content within Feedly (it just says "no content"), but it does relay new posts to me, which I can open directly, and it was amazingly quick to get a functional feed URL, so I'm very happy nonetheless.
A pragmatic approach. Trying to build a generalized framework handling each and every edge case promises endless complexity.
> Your catchword is "share", but you don't want us to share. You want to keep us within your walled gardens. That's why you've been removing RSS links from webpages, hiding them deep on your website, or removed feeds entirely, replacing it with crippled or demented proprietary API. FUCK YOU.
> You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or Atom feeds.
> We want to share with friends, using open protocols: RSS, Atom, XMPP, whatever. Because no one wants to have your service with your applications using your API force-feeding them. Friends must be free to choose whatever software and service they want.
> We are rebuilding bridges you have wilfully destroyed. Get your shit together: Put RSS/Atom back in.
From that readme. Nice project!
I wish someone would make a social rss reader. It would be pretty neat. Although, activitypub might be better protocol for that.
- I regularly recieve feed updates that mark all articles as "new" (i.e. the bridge fails to recognize that it already exported some articles before) - I also often receive "error 521" from rss-bridge (i.e. the bridge cannot create a feed), but on manual investigation, the feed can be created just fine. I assume this is due to a lack of request throttling somewhere in the process, but can't be bothered to investigate.
Next I guess I have to install this thing to RSS-ify other channels I like.
This is kind of exciting.
I’m trying to create a personal clone of mailbrew to send myself a newsletter of all my favorite sources.
By the way, I've been using this for a while and it is superb.
This[1] gives you a JSON of all the routes.
It’s probably more Instagrams fault than rsshub’s that it was breaking. Just be aware of these limitations.
My email client is actually much more powerful than most readers in letting me capture read status and labels / sorting.
I've been planning on transitioning from my current TUI RSS reader (Newsboat) to a maildir-based CLI with notmuch filters for a while; I might give it a shot next weekend and post some of my notmuch filters for RSS.
It surprises me that people here fount it out so late. The project has been quite popular for years.