Not downplaying the immense work of infra / engineering at this scale but my neighborhood local grocery market shouldn’t be down
That is true. it is also the problem. It means the biggest providers do not even need to bother to be reliable because everyone will use them anyway.
But this is not really the case. When Azure/AWS were down, same as this one with Cloudflare: significant amount of web was down but most of it was not. It just makes more obvious which provider you use.
So why be on Cloudflare to start with? Well, if you have a more reliable way then there's no reason. If you have a less reliable way, then you're on average better off with Cloudflare.
As for websites which don't need Cloudflare, in my experience almost every website will be DdoS attacked from time to time.
And why I should overthink my architecture now? If I had to manage redundant systems and keep track of circular dependencies I just could keep managing my infra the old way, no?
I'm being sarcastic here, obviously, but really one of the selling point for cloud back in the day it was "you don't have to care about those details". You just need to care about other details, now.
The place I work at has been online since 1996, not even a DoS yet, let alone a DDoS. Though we now use CF to filter all that bot traffic.
I run a small video game forum with posts going back to 2008. We got absolutely smashed by bots scraping for training data for LLMs.
So I put it behind Cloudflare and now it's down. Ho hum.
https://i.ibb.co/qHCJyY7/image.png
I wrote the below to explain to our users what was happening, so apologies if the language is too simple for a HN reader.
- 0630, we switched our DNS to proxy through CF, starting the collection of data, and implemented basic bot protections
- Unfortunately whatever anti-bot magic they have isn't quite having the effect, even after two hours.
- 0830, I sign in and take a look at the analytics. It seems like <SITE NAME> is very popular in Vietnam, Brazil, and Indonesia.
- 0845, I make it so users from those countries have to pass a CF "challenge". This is similar to a CAPTCHA, but CF try to make it so there's no "choosing all the cars in an image" if they can help it.
- So far 0% of our Asian audience have passed a challenge.
I will say one very appealing use of Anubis I'd love to try is using it as a Traefik middleware to protect services running in docker containers.
Edit: To answer my own question, yes: http://www.arijuels.com/wp-content/uploads/2013/09/JB99.pdf
Edit 2: Maybe TLS would be another reasonable place for it?
As a side note, what does your site do that it's possible to use up all server resources? Computers are stupid fast these days. I find it's really difficult to build something that doesn't scale to at least multiple hundreds of requests per second.
The other part is just how convenient it is with CF. Easy to configure, plenty of power and cheap compared to the other big ones. If they made their dashboard and permission-system better (no easy way to tell what a token can do last I checked), I'd be even more of a fan.
If Germany's Telekom was forced to peer on DE-CIX, I'd always use CF. Since they aren't and CF doesn't pay for peering, it's a hard choice for Germany but an easy one everywhere else.
Hetzner has the WEAKEST DDoS protection out of ANYTHING out there - Arbor sucks.
Send me your website url and I'll keep it down for DAYS and whenever you cry to hetzner I'll just fry it again, it's that easy and that's why they're the cheapest - because everyone ran away from them back then.
And yet my website is still up today, and has not been down for years.
The first time we switched to Cloudflare which saved us. Even with Cloudflare, the DDoS attempts are still damaging (the site goes down, we use Cloudflare to block the endpoints they're targeting, they change endpoints, etc.) but manageable. Without Cloudflare or something like it, I think it's possible that we'd be out of business.
How?
The VPS I use will nuke your instance if you run a game server. Not due to resource usage, but because it attracts DDoS like nothing else. Ban a teen for being an asshole and expect your service to be down for a week. And there isn't really Cloudflare for independent game servers. There's Steam Networking but it requires the developer to support it and of course Steam.
Valve's GDC talk about DDoS mitigation for games: https://youtu.be/2CQ1sxPppV4
And yet game servers still work fine. Which answers this subthread's question ("how likely is it to get DDoSed if you don't have Cloudflare"), answer: not very likely, it happens once in a while at most.
The issue is DNS since DNS propagation takes time. Does anyone have any ideas here?
Only if you're doing very basic proxy stuff. If you stack multiple features and maybe even start using workers, there may be no 1:1 alternatives to switch to. And definitely not trivially.
There are other cloudflare products for which there are not many alternative(durable objects, workflows etc), but at least for us we don't use them in the critical path. We deliberately avoided them in the critical path because we knew we'll have to setup multi cloud for 99.999% uptime(we run a POS system so any downtime results in angry calls and long lines for our merchants)
You think we have a say in this?
If the internet was always a nice place we wouldn't need Cloudflare and similar :(