Requests dropped when using Cloudflare’s free tier for a commercial project
pawelurbanek.com
pawelurbanek.com
you can (and should) put monitors on the outside of your infrastructure as well as inside. I've multiples hitting the login page and robots file where I work, and never got an unaccounted 503
one has to start somewhere reasonable and realistic first.
I’m working on such an app, and am hoping to use Cloudflare + Backblaze B2, because of the bandwidth alliance. https://www.cloudflare.com/en-gb/bandwidth-alliance/
However, the Terms of Service say:
> 2.8 Limitation on Serving Non-HTML Content
> The Services are offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as part of a Paid Service purchased by you, you agree to use the Services solely for the purpose of (i) serving web pages as viewed through a web browser or other functionally equivalent applications, including rendering Hypertext Markup Language (HTML) or other functional equivalents, and (ii) serving web APIs subject to the restrictions set forth in this Section 2.8. Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately as part of a Paid Service or expressly allowed under our Supplemental Terms for a specific Service. If we determine you have breached this Section 2.8, we may immediately suspend or restrict your use of the Services, or limit End User access to certain of your resources through the Services.
https://www.cloudflare.com/en-gb/terms/
And I am wondering what kind of subscription would be appropriate for my use-case if any, that would allow me to host the files on Backblaze B2, and serve these files through Cloudflare.
(Cloudflare would also be hosting our API, so it’s not just video files that we want to serve via Cloudflare.)
However, for a CDN for video content YMMV. Perhaps you should try contacting them directly to have a written estimate.
You'll have to get an Enterprise account for that (it's no longer starting at $5K/mo, you can get custom accounts setup to match your needs).
Meaning that it might be possible to get what we need at a suitable price? $5K/mo is way over our budget until we have actual users and revenue.
If so then that is great. Thanks for the tip :)
https://community.cloudflare.com/t/cloudflare-workers-live-v...
TLDR; He's retracted his claims
If the problem is on cloudflare's side then the cloudflare user doesn't even know about it if it's an intermittent error.
I'm a free plan user because my application is not monetized and obviously would switch to paid plan once it starts generating money. CF free plan has been indispensable for many such free projects.
UptimeRobot(Free) doesn't show anything fishy, except for CF error 521 occasionally. Again UptimeRobot requires paid version to check for HTTP status message, guess I'll have to do it or setup a self hosted(recommendations?) solution to monitor HTTP status of CF Free application.
If CF does perform rate limiting for such low rpm, I would prefer to be intimated as even third rate shared hosting providers do it and I hope CF wouldn't put their reputation on the line for something like this.
But can we at least get an update if this is the intended behaviour of the CF free plan accounts (so people know to upgrade) or is it an one-off incident?
1. Cloudflare doesn’t send 503 when they’re deliberately rate limiting; they send it when there’s an unexpected problem. It’s fairly rare, and it usually results in something being posted on cloudflarestatus.com if it persists.
2. Sometimes error rates are low enough or the conditions are obscure enough that you may have to contact support to point out that it’s happening. This is even rarer.
3. When you switch between plans, the IP addresses assigned to you may change, and users may hit different edges. If the issue is regional (as is usually the case), this may resolve it.
4. I have been sending tens of requests per second to a free plan over the course of several years and have never measured an increase in 503 responses on Cloudflare’s end compared to a paid plan.
5. Cloudflare’s response mentions specific 503 errors that are fairly rare; they’re notable because they look like a standard Nginx 503 page, rather than a nice Cloudflare error page. The only difference is that they will say “cloudflare” at the bottom where the Nginx version would normally be. You probably frequent sites that use Cloudflare free plans; how often do you see this error message?
6. What the hell is this n=1 correlation? It reads like a conspiracy theory. It is a conspiracy theory.
There are plenty of reasons to criticize Cloudflare, but this isn’t one of them.
It turned out we got throttled because we were serving some (a minority, something like 10% of the traffic) video files from the domain and they wanted us to upgrade to enterprise (from business, I think). We just stopped serving video through them.
The annoying part was that we weren't notified and, obviously, had some downtime (no way you can call a site loading in 5 minutes up). They said they have released an update that does notify customers when this happens, though.
Free tier is awesome for small websites, keeping domains and many other things. But if you run business who earns, then just pay for that god damn thing, and stop ranting. :-)
ETA: * The CTO of Cloudflare
What if he accuses them without justification? Which is probably a case here.
He's already doing that. Even if it turns out that he happens to be right, it's coincidence.
I guess HN users might have caught up to the rest of the internet on how to get clicks.
It's not a recent phenomenon.
Bug resolved. Reason: not enough coffee.
Seriously these systems are complicated and it's easy to confuse correlation for causality.
Summary of things that we've had issues with lately:
- Caching of a specific route just stopped working. I log a support ticket (nightmare to find in the dashboard) and miraculously it starts working again after the ticket gets responsded to.
- Some of our staff got locked out of part of our app because we exceeded the 5 free users of the "Access" plan. Upgraded to the 50 user plan in the new Teams part and it still didn't work. Contacted support, fixed again but no explanation. (multiple day turnaround on tickets)
We're invested in their tech a lot and I love workers - they really take some big tasks off us. If Azure come up with something compelling we'll probably switch though (assuming I can make sense of Azure's billing and product naming strategy)
I'm running around 40 domains on Cloudflare Free plan. About 20 of these domains have some traffic.
How do you know for sure that you didn't? One of the main takeaways from the original article is that these errors are invisible from server side (your own stack) because it's the traffic is being throttled before it gets to your own servers. Unless you have very good and consistent client side availability metrics you probably wouldn't notice the 2%-3% drop.
Special kudos for allowing the -- very cheap, only $5 -- worker addon without forcing a paid plan.
Anyway i did a quick check, my 503 error is currently at 3.23k the past 7 days , that's out of 1 million request the past 7 days.
I was about to upgrade to ARGO and pay for it since i was optimising the bandwidth the past month until I saw this article which really gave me a shock.... because I heavily invested in Cloudflare stock market...... Which so far has good returns and i still believe in it but this article is critical ... If what this person say is true, I might exit Cloudflare earlier, from the stock market i mean.
Also why is author giving trust in a free service for his paid service? CF is not that expensive.
I am careful about what technologies I add to my stack because I am a sole developer, but Digital Ocean and Cloudflare have been big wins for me.
There is no such thing as a free lunch [0] for people to depend a commercial service on. If your business model depends on another company providing you a free service perhaps you should reconsider.
This seems to be a person being cheap and jumping to conclusions, claiming broad assumptions and conjecture as fact.
Yes, cloudflare's free tier is deliberately dropping your requests to foil your freeloading commercial company - raise pitchforks!
Assuming this is all true (which I don't), I don't feel sympathy for the author for not purchasing a paid plan.
[0] https://en.wikipedia.org/wiki/There_ain%27t_no_such_thing_as...
To avoid this you need to turn the firewall and security feats to "Essentially Off" at least for asset requests (you can do this partial blocking via a page rule).
That being said this doesn't seem to be an issue with Free vs Paid, just a general problem with their blocking.
I'm curious, do you know how it was fixed? Is it not serving a captcha for the second request on a connection or something? Or does it somehow figure out that asset requests are "safe"?
I'd expect it to be based on the Accept request header (and possibly the Content-Type response header to prevent bypassing it). Or perhaps even the Content-Type of past requests to that url.
But I don't really understand what the purpose of these captcha checks is in the first place. Handling a captcha challenge is more expensive for the server than most GET requests. Perhaps it's done in anticipation of later POST requests (which can't be blocked transparently without breaking the functionality of the website).
However today and few days ago too it seems requests going through Cloudflare are just timing out. So we finally move to paid plan
We're monitoring both endpoints (CF and the origin)
So that's another reason to subscribe since it's 20$ month. (AWS and other SaaS cost us an order of magnitude more)
Any case, what I meant to say, is that free plan works fairly well even at higher volumes.
If you expect more reliability, should definitely subscribe.
the advanced cache statistics (will help us reduce traffic sent to the S3 origin, in one case, and reduce bandwidth cost)
Traffic is served from the closest location to the user (instead of the bigger central locations, where they can serve traffic cheaper)
Plus paid suppport, for sure
Admittedly the cloudflare free plan does say it's for things "that aren’t business-critical", but what does that actually mean in terms of resource quota and expected uptime?
Isn't that quite reasonable? Especially if you're scared if your site goes down.
Same applies for AWS 'Free tier' database usage. Nothing free about it practically.
I would not want to use a CDN with some hidden limits. Especially not if I don't even get informed when rate limited.
I can imagine them rate limiting me in a way that I will never notice. Like only limiting requests from some other country or continent.
What is a good CDN for a site that has about a million visitors per month? And how much would one have to expect to pay for it?
Also, if the infrastructure is in AWS, CloudFront would cost cents if he really does have "4 requests per minute" with full integration and logs.
I have the feeling that with the advent of cloud solutions, very big companies depend on these really small and very useful tools that disregard almost entirely a number of best practices and standards. The Solarwinds incident is going to happen a lot more, that's for sure.
Regarding Cloudflare: I'm fine with the free product not being great, but hiding the logs is not understandable. I bet a lot more people would upgrade if they knew.
[1] https://www.cloudflare.com/learning/ddos/glossary/web-applic...
It does seem to be more Marketing if you want to lock in customers before they get too big to switch.
1. Marketing...
2. Data collection (after all if you are not paying you are the product) There is monetary value in collection, observation, and tracking of a vast amount of internet traffic.
3. Education. If you get people just starting out in their careers using your product or service for they hobby, personal project, etc then when they have a business need for some like your product or service they will just naturally choose you. Adobe and MS has been doing this for decades with low or no cost education licensing
Then use another service or pay for it.
This is highly unlikely to be as simple as the author conjects, we're talking about a service that processes an enormous amount of traffic and if what the author suggests is true, would someone else not have noticed by now? It's certainly possible that different infrastructure is used by free/paid plans, and perhaps this specific site was hosted on an unhealthy instance. But we don't have any external data points here to analyse - only those reported by Cloudflare, along with anecdotal reports.
Edit: I see the title has now been changed, at least we're reducing clickbait.
https://i.imgur.com/UOlW836.png
edit: changed bull crap to the more appropriate horseshit in response to this unsubstantiated-bordering-conspiracy blog post, because if the former is already enough to triggers the community downvoting brigades, no point in showing any restraint