Cloudflare and RSS
tedunangst.com
tedunangst.com
And they don’t even bother to implement it properly—for example if your site tries to follow best practices and uses a separate domain for your static assets, you will just get errors on your static assets, resulting in a page with broken styling and no images. That despite pissing off your users by having them go through the Google hosted captcha (which also breaks all the time btw[2][3]).
One of the websites that was horribly broken by this was Stack Overflow. As anyone trying to stay safe on public WiFi by using VPN can attest.
Coincidentally, Cloudflare has lost Stack Overflow as their customer recently: https://meta.stackoverflow.com/questions/323537/cloudflare-i...
They’re now behind fastly.
_
[1] https://ipfs.pics/QmTZo6oPKHwUgWB7p7LfZwZsVQJV1n7k9qNQNZBCEu...
[2] https://ipfs.pics/QmeuJjgV621NV9aNKyNAUoEHdWZYtzCrwkLHoHneg3...
[3] https://ipfs.pics/QmRWcCkBdaG214GKttkGFcadncUJ6YvfMTSE8jiAxA...
bonus picture: https://ipfs.pics/QmPkncvs2R9EkhZQuzPzWYs4z7UUdKqQzg1k8mc7y5...
I'm very optimistic about the direction the internet is going in right now.
As a quick primer, it's a hash that points towards a directory of content. Everything's deduplicated. Based on Torrents, GIT, and self-referential filesystems.
an IPFS hash is immutable. The hash points at the hash, no matter what. Indestructible. Publish stuff by
ipfs add -r folder
An IPNS hash points towards an IPFS hash. It's a pointer you publish every 12 hours. It IS mutable. Do this by
ipfs name publish <ipfs key of resource>
The browser plugins with IPFS running allow you to pull the DNS text record of the IPNS hash, and you never touch the website!
Example, PageNodes :
http://ipfs.io/ipns/QmVjH4F65fnqy1GkBBYiuAkdazKzYsw3LbMVANGF... POINTS AT ---> http://ipfs.io/ipfs/QmbLPfyehFnViKZpU237P6a6DpjCfWFSoDBMQFGU...
Tl;Dr. DDoS makes no sense regarding IPFS. Everybody makes the network faster.
Having said that, thanks for the interesting digression. You've made me want to try out ipfs.
CloudFlare is a anti-DDoS and CDN network. IPFS is a CDN protocol that anyone can join or put files into. It doesn't quite hide the endpoints, but anyone can inject data.
It does what CloudFlare does, but better. And as more people/nodes get online, free and ubiquitous.
Which is what exactly? The default being to have a uniform service across all asset types seems sensible enough to me. It's either that or everything is off by default and you manually include paths/expressions that are included. The latter seems more of a hassle for most people.
> Isn't "this stuff is hard, let us figure it out for you" CloudFlare's entire value proposition?
I'm not sure what their angle is. As far as I can tell it's to re-centralize the internet so they can be the single tap point where SSL is added and removed[1]. Once they've got a sizeable chunk of the world funneling through them, they could make some decent coin giving direct access to the feds.
Until then, it's just free SSL and DDOS protection. Can't really complain against free either. Heck just don't use it.
[1]: http://www.newyorker.com/wp-content/uploads/2013/11/nsa-smil...
That really is not a viable option when you want to access content which happens to (now) be behind Cloudflare.
Sure it does. Vote with your bytes and move along.
Odds are the nntp client in question should customize the user-agent, and a request made to Cloudflare to improve the ranking for that agent... that's about the best that can be expected here.
Cloudflare could make this easier if it detected that they had an RSS feed and offered a suggestion, via email or the browser (or both) that was like "Create a security exception for your RSS feed"
If they publish something that offends someone with a botnet, and have an exception for the RSS feed, the offended someone will just attack the feed to take down the site.
The real solution is for the feed reader to contain better error handling.
Given how many sites use Cloudflare, I'd welcome they switch to a more reliable captcha service until Google repairs their scripts.
Oh and other captcha services work in Tor Browser Bundle.
Cloudflare is a hosted reverse proxy service that handles DDOS protection. It enforcing rules on RSS pages is no different from me putting in a captcha extension on nginx or Apache and having it run on all pages by mistake.
And this isn't even a default configuration issue. This is simply a mis-configured service, no different from misconfigured haproxy / nginx / Apache / Cloudfront.
Torrent traffic, file transfers, VOIP and all the other non-HTTP type traffic that Cloudflare just breaks by default make up a good chunk of the traffic you see on the web. That Cloudflare pitches itself as a one and done solution with minimal configuration just nakes this worse, since website owners generally won't bother to set up custom rules for RSS feeds and the like. Additionally, if even 1% of the RSS feeds that were broken by cloudflare were emailed to you, your inbox would be flooded.
Huh? We don't handle non-HTTP traffic. How can we break it?
I'm known to use console/text clients (w3m, lynx, links, elinks[2]) from time to time. Cloudflare definitely interferes with these.
Not sure about the other examples given.
And, to hijack: I wanted to say thanks for the work on a Tor-friendly anonymised reputation system. I've commented on that in the past, and need to take a closer look / see others' thoughts, but definitely appreciate the effort.
Google does the same thing on Youtube when you're browsing from a "bad neighbourhood" (OVH). Incredibly annoying and, unlike Cloudflare, redirects you to youtube.com instead of the video you wanted to watch. The problem keeps reoccurring despite being logged into a Google account.
Ironically, youtube-dl from the same IP works just fine. So I don't what they're protecting. Are they trying to prevent automated comment-reading?
In fact, it's possible that the person he was hitting was in "I'm under attack" mode or similar, which would try to reduce bot hits to the web server by any means necessary to prevent a layer 7 attack from taking the site offline.
In a security context, automatically poking a hole through for RSS is automatically giving attackers an easy-to-use door straight through to the underlying site to DDoS them.
You might want to say "Oh, well, then, let's just set some bandwidth rules", which will certainly work for specific sites, but it's going to be difficult for CloudFlare to correctly guess them generically. (Not necessarily impossible, but it is impossible if you measure it from the POV of them never being wrong. It would only be a heuristic guess.)
And if your web application allows queries that produce RSS feeds that could still result in a really bad L7 attack if you simply were to ignore all feeds. No caching + randomized queries on a small site would knock it offline in no time.
https://support.cloudflare.com/hc/en-us/articles/200168306-I...
It's pretty much dead simple though, basically you put in a path like "/feed.rss", you can also use wildcards like "/feeds/*" and then set up a configuration for it including parameters like cache time and security level. In the case of a feed, high caching and minimum security are probably reasonable for most sites. Free users are limited to only a few rules (5 I think), but paying even for the cheapest plan removes the limit.
In my opinion, anyone who puts this sort of stuff in front of their blog is overly paranoid (unless they are some sort of high-profile victim, like Brian Krebs). Just avoid reading their blog.
The only reason Cloudflare captchas exist is because of utterly incompetent engineers, nothing else. You can stop attacks without breaking the internet for people without US residential IPs.
I would much rather than Cloudflare added a header when in "under attack" mode, and delivered cached responses to get requests.
https://en.wikipedia.org/wiki/Defense_in_depth_(computing)
Really, the longer a website is online, the more it attracts bots. Spam bots, brute-force login attempting bots, information gathering bots, vulnerability testing bots, and many more. This isn't even counting targeted attacks.
I've seen many small websites recieve much more bot traffic than user traffic. Sometimes to a crippling level.
My point is, it's not paranoia if they really are after you - even if it's a mindless mass of bots.
Like all security though, sometimes you affect legitimate users. This is why it's great to have multiple ways they can provide feedback.
This was mitigated by implementing a few layers of a decent caching strategy, as well as some db improvements, and moving search queries to a separate database server (mongodb, then elasticsearch) altogether.
In the end, there are lot of things you can do to help mitigate these things... It really just depends on what you are trying to accomplish with a given site.
Maybe have working default configs instead? Users are not going to change the configs unless you make them.
You have a firewall, and then complain to mongo because you can't access your database from another computer.
Also keep in mind that just because it's "RSS" doesn't mean there is a quick and easy way to exclude it from security. On your average Wordpress blog the URL is /somethingradom/feed/. So either Cloudflare assumes every URL of /feed/ should be exempt, or it should read the contents of every page to check for exclusions?
Also do keep in mind that if you're a Cloudflare customer you can easily exclude specific URLS from this type of security scrutiny. So perhaps the blog owner is incompetent? Perhaps this person posting this is on a network with a computer thats infected with a botnet.
Who knows. This "article" is shit.
And for that matter, it's entirely possible that they consider this correct behaviour. They may have a poorly written dynamic RSS feed that doesn't cache for instance, and want it protected.
Moreover, Cloudflare provides plenty of ways for the website owner to avoid this behaviour - the global security level can be adjusted, settings for individual endpoints can be adjusted, even settings for individual IP ranges can be adjusted.
As a user, you have no idea what settings the website owner has selected; it seems rash to blame Cloudflare in that context.
I don't even know why this is a discussion. Not blaming customers and users for anything is a common sense.
Personally I stopped using "traditional" RSS readers years ago, and have stopped using more "modern" RSS readers like Flipboard or Feedly last couple of years, since they're just a subset of what I can find on Twitter.
Most people who own a blog have a Twitter and they share all their posts on their feed anyway, so I just follow them on Twitter and use Twitter as "RSS reader".
Before you open web standards advocates throw rocks at me, I am not an open standards hater either, I have built a couple of RSS related apps as projects in the past and still believe that's the way to go in the long term, but in 2016 I can't really find a reason to still use a RSS reader.
In fact the scenarios like what OP mentioned in the article is exactly why I would rather use Twitter. Why deprive of yourself of an opportunity to read content from someone you like, when you have a totally free option?
This quickly breaks down. I subscribe to 1096 sites according to Newsblur. I would 100% miss some content from those sites if I only followed them on twitter.
And for some of those sites, that's probably fine; a fair amount are just 'entertainment'. But there are updates I would hate to miss, even if they tweeted links to them at 3am or right before a huge tweet-storm from someone else.
That is exactly the problem RSS was designed to solve.
That's not possible, because you can get your Twitter timeline as an RSS feed.
Also remember that every podcast app is an RSS reader.
This year, 21% US adults ages 12 or older have listened to a podcast in the past month.