Hacker News Reader

Comment by phillipseamore | Hacker News Reader

It's time to start discussing a fail-open option for us CF users. Most of my sites are using CF for global performance rather than DDoS protection and security. I'd be fine with them changing DNS to point to the origin (or any other user defined IPs) in case of issues (even if it would take hours to return to normal).

This is also important for countries with limited connectivity to the Internet, if the PoP in that country looses it's connection back to CF it shuts everything down, so even if the origin is in the next rack over from the PoP, it's un-reachable.

View on HN

You can implement your own DNS server that CNAMEs to Cloudflare and falls back to origin IP when there is a problem with Cloudflare. I think a downstream Cloudflare provider could provide such services if they desire.

Outages occur to the best of CDNs. But the likelihood of two CDNs suffering an outage at the same time is close to zero. Check out www.cdnreserve.com.

Last time I checked that was limited to the enterprise plan.

Of course it is... Typical Cloudflare.

Imagine having to pay for a service

Cloudflare's pricing has issues.

My company was paying $20 a month. We were heavily depended on CF, we'd have been happy to pay more.

But... the one feature we wanted was for our accounts team to have their own login so the ops team didn't have to download invoices every month. Nope, that one feature required an enterprise plan which they quoted $4,000 a month for.

Oh dear your poor Ops team, they had to download a few invoices to save $4k and every month! My heart bleeds for them.

Companies where you have to log in and download invoices are the worst. If there's a viable alternative to their products I switch immediately. You make it seem like it's not a big deal, but a reasonably sized startup has dozens of service providers. Should we pay every little service $4k/mo just to save the communications and context switching overhead?

You jest, but imagine how time consuming it would be if every app we used was setup like CloudFlare, where only the one super admin can deal with billing.

Also in these days of remote work, it's a problem if the credit card details need updating - either you have to give the company card details over a slack call, or you need to give a card holder your root password.

I've been impressed with Cloudflare's (non enterprise) value thus far, what Bandwidth & Users did the $4k quote cover?

Is there no api for that? It would even save the manual download effort.

Just write a script?

Imagine having to pay for a service... at enterprise prices.

This is not really an extra or a nice-to-have, it might be more a hostage situation.

Imagine already paying for a service and then having someone snark at you for wanting things for free.

I tried to exercise some restraint this time, but screw it. Here's another rant:

Beware of Cloudflare's tactic of luring people in to their CDN product with "free" bandwidth, and then locking useful features arbitrarily behind what I can only imagine is a thousands of dollars per month enterprise plan. Just look at their cache-purging page for a super obvious example of this (there are plenty more, way too many to list), everything other than basic purge by URL is enterprise only: https://developers.cloudflare.com/cache/how-to/purge-cache/

These days Cloudflare is literally my last choice for a CDN for my new projects. My new go-to is bunny.net, who charges a reasonable usage-based fee for bandwidth and gives you unfettered access to all the features they've built (and doesn't route your users to farther/closer nodes based on how much you pay: https://cloudflare-test.judge.sh/). Though I'd even reach for Cloudfront with their expensive bandwidth costs these days, because at least their pricing is transparent and scales smoothly with usage, and they don't arbitrarily cut you off from useful features that you might not know you need yet.

Even their bandwidth might not really be "free", since I've heard if you actually use any significant amount, the sales people will come knocking on your door to coerce you to get on the same enterprise plan or have your site taken down.

Can I ask out of interest (most of my projects are high perf/low traffic) what kind of traffic you are dealing with at the point you decide you need a CDN?

I don't really use a CDN to manage high traffic volumes. It's more to provide a better, lower-latency experience for my users regardless of where they access my apps from.

You’d need to have TLS certs on origin ready to go for this scenario to work. Additionally, you’d need to make sure to test it and ensure that there’s nothing wrong in this event.

On top of that, depending on your scale, can you take all the traffic on origin that Cloudflare currently offloads?

No issue for me. This is obviously a power-user option. It's kind-of implemented for Enterprise users were you don't have to let CF have full control over the domain.

> I'd be fine with them changing

This would be best implemented by you. If the point is to avoid CF as a PoF, why would you rely on their infra to fail safe when something breaks?

Wouldn't this expose the origin IPs to attack?

Yes, but he says in his second sentence that he doesn’t mind and mainly uses CF for performance.

Plenty of commenters also seem to miss the most important word in the first sentence: "option".

I'm talking about this as an option for users like me, that don't have an attack surface, but need the global performance gains of CF.

Probably not many users who need the performance and can handle unexpected failover. There would also be the issue of setting the policy defaults effectively. Most users wouldn’t benefit from this footgun.

If you’re serious, you could probably automate this right now with your DNS provider and uptime monitoring.

I'm so serious that I already have failover after 1 hour at the registrar level, but those changes are not immediate and can take up to 24h to roll-in and roll-back due to DNS propagation and caching.

We seem to have hit the nesting maximum 1123581321, but to your point "There is no immediate option with DNS changes."

There's a huge difference in changing nameservers for a domain and simply changing host records.

This is a 7th level comment. Synu below[0] has a 9th level comment, and I've seen nested comments go quite a bit further.

[0] https://news.ycombinator.com/item?id=31821497

I think there is no nesting maximum (or if there is, it's much bigger than this). There's a limit which stops you replying to a comment immediately, to prevent super long quick-fire arguments.

Ah, sorry, misunderstood you. You can’t rely on them to change their host records when they’re down.

If you want CDN-independent automatic failover, look into anycast with two providers. If one of them is Cloudflare, use the tier that lets you manage your DNS elsewhere.

There is no immediate option with DNS changes. CF can’t immediately remove their IP from the route. Sounds like you’ve solved your problem in the sense that you have an automatic failover, though, which is good.

Typically CF _is_ the DNS provider though, right?

Only up until the $200/mo tier. This kind of feature would be locked at that level anyway.

As I said to the OP elsewhere, he should be doing something like anycast to multiple CDNs if this is critical.

Yes, but I'd wager that most sites experience cloudflare outages more often than they experience bona fide attacks.

You say that, but there's tons of automated attempts doing the rounds on everything directly connected to the internet; centralized providers like Cloudflare can detect and prevent these patterns, whereas you need to be on the ball yourself if you have a service directly open to the internet. Exploits are exploited quickly, and while I make no assumptions about your particular website / application, a lot cannot push an update on short notice.

You would lose your wager.

DDoS attacks are extremely common at just about any scale, even if you only have a few thousand users.

Cloudflare going down like this? Actually first time I remember it happening. There’s been downtime before but nothing so major.

The origin IP's are already open to attack for most users.

It's trivial to scan the whole IPv4 internet to find out which IP you are hosting your site on.

You need to block 443 for any other IP than Cloudflare. The IP list can be found at https://www.cloudflare.com/ips/.

and/or use Authenticated Origin Pulls with a TLS client certificate.

That would leak your IP nevertheless. People can figure out that you're serving a specific website by inspecting your certificate on handshake without actually connecting.

Reply on news.ycombinator.com