How to bypass Cloudflare bot protection
jychp.medium.com
jychp.medium.com
This article contains several misunderstandings.
If a Cloudflare customer has configured their origin server to respond only to Cloudflare IPs, then they MUST also verify that the "Host" header on any request actually matches their domain name. If they do not verify the Host header, then anyone can sign up for Cloudflare and simply configure their DNS to point to the victim's origin IP address, and requests will be routed there -- but will have the attacker's domain in the Host header. Workers is not needed for such an attack. This attack has always been possible and is common to basically all CDNs. Fundamentally, the CDN has no way of knowing if the origin server that a user has configured really belongs to them -- the CDN can only tell the origin (via the Host header) what customer it thinks it is serving, and expect the origin not to accept requests that were on behalf of a different customer.
Instead of IP-based authentication, we strongly recommend using mTLS-based authenticated origin pulls (with a zone-specific key pair) or Argo Tunnel, as these methods are much more secure.
As long as the origin server is verifying via one of the above means (IP+host header, AOP, or Argo Tunnel) that the request was processed by Cloudflare on behalf of the customer's zone, then the attack described in the article doesn't accomplish anything. If a Worker makes a request to a hostname that is on Cloudflare, then all of the target host's Cloudflare security settings will apply to that request the same as if the request came from an external client.
Additionally, as mentioned in the article, any requests coming from a Worker will have the CF-Worker header identifying the zone which sent the request. If a customer suspects abusive requests coming from Workers, they should report it to us.
~~The article claims that the X-Forwarded-For header can be forged, but this is not true if the target host is itself a Cloudflare customer.~~
EDIT: What I said about X-Forwarded-For seems to be incorrect. CF-Connecting-IP has always been the header we use to identify the client IP, and it cannot be forged. X-Forwarded-For seems to have more complicated and subtle behavior that I'm not familiar with. I'm investigating.
Out of interest do you provide any automatic checks for this? It seems like it would be trivial to add some kind of verification step that simulates a request with a bogus host header and checks the status code of the response.
It would be a nice feature if CloudFlare customers could opt-in for origin server IP verification. Once the origin IP is verified, it would prevent other CloudFlare users from using the IP as their own origin. However, I understand why CloudFlare doesn't verify origin IPs -- it would require a verification process that doesn't rely solely on uploading a verification file to a web accessible directory. It'd be a bit difficult to keep the verification process simple for end-users if they aren't running a web server. Or maybe I'm wrong and non-HTTP use cases don't need to be considered and a verification file uploaded to a web accessible folder would work fine.
I can think of a whole bunch of corner cases to this, but that’s the general jist.
You wouldn't want users who don't own the origin IP to be able to turn on that sort of check and potentially cause a service disruption. It could be the case that the legitimate owner already has a Host header mismatch in their HTTP requests/responses.
The logic is tricky, but in the end origin server ownership verification is what's required to safely support Host header checks.
I'm not arguing for or against ownership verification, but there is opportunity to improve here that does not depend on the question of ownership verification.
You'd have to have implement it in-line with the dns resolution in the server before talking to the origin, right? There's no other time you could reasonably do it (DNS changes), and it would have to be checked on all customers requests, not just the ones that opted into the feature (because those other customers could be the "attacker"!). Of course, you would cache it and use an efficient lookup scheme, but in general cloud services really don't like creating features that have non-localized effects like this.
Not to mention this sounds like a support nightmare foot gun for enterprise customers.
Anyway, thanks for hearing my suggestion!
* The IP address 2a06:98c0:3600::103, mentioned in the article as appearing in the CF-Connecting-IP header, is a special IP address that is used for all cross-zone requests that come from Workers. This is not the actual address of any Cloudflare machine. Workers fundamentally don't have distinct IP addresses. Instead, the CF-Worker header needs to be used to identify which worker sent the request. We intentionally do not identify the original client's IP here because the Worker itself could be working against the client, and could have maliciously modified their request to be something entirely different. Hence, the request can't be considered to have come from the original client.
* Any cloud service that lets you make HTTP requests can, obviously, be used to set up a proxy that hides your IP address. Workers is actually less useful for this compared to many other cloud hosts since we tag all outgoing HTTP requests from a Worker with the sender's domain name (in the CF-Worker header), which makes it relatively easier to track, report, and block abuse.
Where is this documented?
The worker code sends the correct host name.
https://github.com/jychp/cloudflare-bypass/blob/dd7c1cfc2e6b...
The author of the blog post seems to have been confused by the fact that different behavior happens if the target host is on Cloudflare vs. not on Cloudflare.
This could be improved considerably if Cloudflare removed support for insecure TLS options. "Full" SSL bypasses certificate hostname verification on the origin server which could prevent such an attack at the TLS level. Every time I've added a new zone to Cloudflare, Full or Flexible (ie, no SSL) has been the default. The only safe option is Full (Strict) but I'd bet good money that the majority of Cloudflare customers aren't using that.
> X-FORWARDED-FOR: 1.2.3.4 (yes, we can override this header with whatever we want !)
Why aren't you guys disallowing that field from having any external value?
https://support.cloudflare.com/hc/en-us/articles/200170986-H...
For better or worse, this is how X-Forwarded-For works, by convention: each forwarder is supposed to append a value. This means that, even in a correct implementation, an attacker can specify an arbitrary prefix of values. The consumer must be very careful about which part of the chain they trust.
But the report here seems to be suggesting that we're mishandling this appending in some cases. We're investigating and if true will fix ASAP.
Because X-Forwarded-For is surprisingly complicated, we recommend using CF-Connecting-IP instead. As the docs say: "To restore original visitor IP addresses at your origin web server, Cloudflare recommends your logs or applications look at CF-Connecting-IP or True-Client-IP instead of X-Forwarded-For since CF-Connecting-IP and True-Client-IP have a consistent format containing only one IP."
I strongly recommend you play around with Azure CDN, because this is not entirely true.
Ironically, the purported problem in this case is the opposite: Because we don't attribute the request as coming from the original client (in order to avoid the security problem you reported), a client that has bad reputation associated with their IP can proxy through a worker to hide that. The answer is to assign reputation to the Worker itself.
This seems hard. Let's say the request is the very simplest "GET /" request. The worker does nothing except rewrite the hostname, in order to proxy it to a different server. Is it "the same" request?
Well, imagine the new destination host is a company-internal dashboard that authenticates users based on IP address. Now this "GET /" request which was proxied will receive a response that contains secrets. The worker that did the proxying can intercept the response and learn the secrets.
So clearly if the request's host is rewritten, it cannot be "the same" request and it cannot be considered to have come from the original client.
But the only case where we care about any of this is when a worker makes a request to a different domain. If a request is to the same domain, we trust it, because in that case the attacker can only attack themselves anyway. So the only case that even matters here is when the host has been rewritten, and in that case, we've established that the new request definitely cannot be attributed to the original client.
So it seems to me there's nothing interesting we can do here. A request coming out of a worker (going to a different domain) must always be attributed to the worker itself, not to the client that triggered the worker.
We'd love to use Argo Tunnel but unfortunately it breaks with an OPTIONS request for us. We've tried opening tickets with Cloudflare.
What exactly could such a misconfiguration enable? Just that the attacker could configure lower levels of Cloudflare security settings for their domain, to bypass the origin's intended Cloudflare security settings? Would this also bypass I'm Under Attack Mode?
What happens if you receive a report of abusive behavior from a Worker? Would you rate limit the specific Worker, or would there be any impact on the larger Cloudflare account of the Worker's owner?
For instance, if we have a Worker that connects to an arbitrary endpoint based on user-generated parameters, or some vulnerability is found (in the JS code for the Worker) that allows this to happen, and it ends up being used for abusive behavior, would we have recourse to fix the vulnerability, or would action be taken immediately? Would our larger account, or our unrelated Workers, be at risk of being taken offline?
Could you comment on that?
I would speculate that proxying through any cloud service (Workers, AWS, Digital Ocean, whatever) can indeed be a tool to hide bot activity, and that any bots product needs to think about how to combat that. I would also speculate that originating IP address is not the only signal the product uses to determine threat. Moreover, I would speculate that this is all an arms race, there will be a constant stream of new techniques developed by bot authors and new techniques deployed to thwart them.
You may be right that the blog author intended to report a problem specifically in bot management, and I have no idea whether that specific problem is real. However, the blog post is not very clear, and many readers have misinterpreted it as reporting a way to totally bypass all Cloudflare protections, such as WAF, geo firewall, rate limiting, etc. That's the misunderstanding I meant to address. Sorry, I could probably have been clearer about that.
Cloudflare's anti bot tooling is terribly annoying while using Tor. I understand why they do it, though. Using Tor is a good way to show how terrifyingly much control Cloudflare has over the internet.
I do agree that Privacy Pass is in theory a good idea, although to say that CloudFlare "merely" requires captchas to be filled out is a bit disingenuous. You're often required to complete 5-10 captchas, and sometimes even after that, you get denied. Had that happen to me multiple times.
Seemingly I am passing the captchas, they are not giving me any errors. Simply seems to accept my input, reload the page and ask me to do more.
Glad you mentioned it. This also has happened to me many time. It is unfortunate botnet and spam has give IP addresses bad names.
I'll say it again: centralizing the internet around a single company is a terrible idea. Especially so when said company terminates TLS and thus has access to unencrypted traffic.
You make your life simple by saying "Cloudflare makes that decision." Cloudflare says "We are merely offering a service here! You don't have to use it, if you think it's wrong." And ISPs say (or should say): "We are here to sell Internet access, not to watch what everyone on the Internet." and may by law not be allowed to look into the traffic.
In the end the user suffers and is discriminated against and no one wants to be responsible.
I would take the position, that each person, who uses Cloudflare, has to live with being responsible for whatever Cloudflare imposes upon users / visitors. The mass of people using Cloudflare services makes the difference. Each person using it adds a little bit to it. It is a collective responsibility.
Cloudflare doesn't "require" captchas for TOR users. By default, they treat TOR IPs it like any other IP. However since a lot of traffic comes out of a TOR IP, it looks like bot traffic
"Some" because they seem to have internal heuristics for detecting Tor Browser users before they enable that feature for a connection, so it doesn't apply to all connections made through Tor. YMMV I guess but I haven't seen a Cloudflare CAPTCHA over Tor in a long time.
https://blog.cloudflare.com/cloudflare-supports-privacy-pass...
Sounds like it allows you to browse without tracking in a way that more reliably signals you're a human, which seems very useful.
Why would a human browsing the internet with Tor not want to use Privacy Pass?
> And they do offer Privacy Pass as a sort of
> approach to make it a little less annoying.
And I don't even use Tor. But every few weeks I'll have a two or three day spat where every webpage that I visit requires at least two captchas to be solved. Maybe another user from my ISP is scrapping, but I'm a good citizen on the 'net.
Care to elaborate? Why block access to static pages by default? Tor’s aggregate bandwidth is tiny compared to CF’s aggregate bandwidth, so DOS attacks surely aren’t the reason.
Malicious traffic takes many forms. As an attacker trying to attack a website, you are probably not going to come in from a residential IP address, unless it’s one you have access to illegally. It is much more likely you will use Tor.
And because of the nature of Tor, you literally, full stop, cannot ban individual Tor users to any meaningful degree. So if you offer services not requiring sign up, or services where the sign up is unobtrusive, users abusing the service via Tor are difficult to stop.
Of course there are counter points... such as [1]. But let’s be honest, from the perspective of a tired sysadmin trying to stop ongoing vandalism, your options are limited, especially if you don’t want an intrusive sign up option.
As a counterpoint to the counterpoint, if Tor users were treated like any other user malicious parties might use the Tor network even more often than they already do.
Edit: also, though it is not necessarily malicious per-se, there are some types of traffic like scraping that may be undesirable from the PoV of the website. I am not going to make a value judgment regarding the merit of blocking scrapers, but it is nonetheless a goal where you can see the reasoning behind blanket banning Tor.
[1]: https://blog.torproject.org/the-value-of-anonymous-contribut...
X-Forwarded-For should always be treated as forged at least for security purposes (except the hops set by reverse proxies under your control). If you do IP whitelisting based on untrusted X-Forwarded-For you’re doing it terribly wrong.
As this post demonstrates, you need to verify against the cloudflare origin pull CA (see https://support.cloudflare.com/hc/en-us/articles/204899617-A... )
I really want to know more about this scene. If it's not too much work for you, please contact me at Bibliohoarding@tutanota.com
Thank you!
If the X-Forwarded-For value from the request is forwarded by Cloudflare’s proxies unaltered, giving special privileges to Cloudflare workers, that’s a problem, but it’s not clear whether “a site who display your headers” is protected by Cloudflare and displaying an unaltered X-Forwarded-For. I don’t expect Cloudflare to have made this oversight, and the e-mail response from them matches: “All workers subrequests go through Cloudflare again, and therefore you won’t be able to bypass any restrictions directly.”
No, you're not. Which is why Cloudflare were correct to say this isn't a security issue.
I'm not so sure this is true. Not long ago Cloudflare's CEO was making fun of people trying (and failing) to bypass Cloudflare's anti-bot measures. This allows to bypass this because as far as Cloudflare is concerned, the requests are coming from a trusted source. Setting up a proxy with a provider that isn't Cloudflare, means the requests aren't coming from a trusted source any more.
router.route('/proxy/:url', '*', async (event) => {
event.request.headers.delete('cf-connecting-ip');
event.request.headers.delete('x-real-ip');
return await fetch(event.parameters.url,event.request);
Do you happen to have a link?
The limits on workers' sub requests (e.g. requests are done by host name only, can't use IPs with a different host name) make it somewhat cumbersome to set up reverse proxy workers (which was my use case), but the tech itself is just great, works very reliably and the delay, while noticeable, is tiny and never bothered us.
But I agree with CF, they don't inherently have a security problem. They have the normal situation of a proxy again. If a lot of new domains show up to bot proxy then the Baysian for an unknown worker domain is going to become bot..
I don't see why I can't request any site on the internet whatsoever, having legit Cloudflare IP address. Is there some limitation of Cloudflare workers?
2. they have abuse alarms for obvious "spray requests by utilizing Workers" behavior (although we don’t know how well these might work)
2. The Cf-Connecting-Ip header is passed with the correct originating request IP, so if a website at least partially trusts CF (without directly using CF's proxy) they can match based on this header if the REMOTE_ADDR is Cloudflare. They could also block requests with the header Cf-Worker[0].
3. You can't forge the HOST header on CF, so if a website does use CF you can't bypass the actual CF proxy firewall by fetching the IP and setting the HOST header yourself (you can do it within your zone though)[1].
0: https://community.cloudflare.com/t/is-the-cf-worker-header-f...
1: https://developers.cloudflare.com/workers/runtime-apis/reque... (resolveOverride)
There is a way to bypass CF bot protection, which mostly uses basic HTTP features, nothing fancy (and especially not a security issue).
Stumbled on that some days ago. Quite annoying and there doesn’t seem to be a easy hack.