Cloudflare's rationale for a ToS violation is absolutely bonkers. Who cares what the file extension is? By virtue of being accessible on the web via a URL it is by definition web content.
That's exactly what it is, and Cloudflare should be honest about that instead of coming up with an excuse as pathetic as ".txt files ain't web content".
They could do that anyway by putting up the "Checking your browser before accessing" page and redirecting to whatever file was being accessed. There's precisely zero need to outright modify the HTML files being served to inject such checks.
Tell me, how does CF put a 'Checking your browser' page in my `wget https://easylist.to/easylist/easylist.txt`
Bad things are already happening. Random HTML data is no less useful than no data at all.
> Tell me, how does CF put a 'Checking your browser' page in my `wget https://easylist.to/easylist/easylist.txt`
By serving the HTML instead (with JS that does whatever checks and then redirects to the intended resource), and if the client can't cope with that, then tough shit; the absurdity of such arbitrary evaluation of whether or not clients are sufficiently browsery to be worthy of consuming bandwidth aside, ain't "only browsers directly navigating to this will be able to cope with this and gain access" exactly the point of using such JS-based client checks in the first place?
<sigh> and a lot of similar rebuttals to this one.
> In short, it's much harder to protect a text file against DDoS.
It's exactly the same difficulty. Bandwidth is bandwidth, bytes are bytes. The only suggestion I've seen where HTML/CSS/JS might be relevant is using JS for client-side probing, and if Cloudflare is indeed injecting arbitrary JS into HTML pages it serves then that's utterly horrifying and is a problem in and of itself.
> <sigh> and a lot of similar rebuttals to this one.
My point is that a lot of people don't seem to understand the 'why' of this issue and instead appeared to just jump to the conclusion that:
> It's exactly the same difficulty.
When that's not at all the case.
> using JS for client-side probing, and if Cloudflare is indeed injecting arbitrary JS into HTML pages it serves then that's utterly horrifying and is a problem in and of itself.
Well they are. From CF:
>> Cloudflare’s bot products include JavaScript detections via a lightweight, invisible code injection that honors Cloudflare’s strict privacy standards
But even before we consider that, if the request is for html then it's likely coming from a browser. If CF replaces that html with their own then the browser will likely run it allowing them to run all kinds of probes then run the redirect. The same is not true for a .txt file or an image.
We understand the "why" of the actual issue - i.e. the bandwidth consumption - just fine. It's Cloudflare's decision to micromanage data formats consuming that bandwidth (instead of just, you know, evaluating the bandwidth itself and being done with it) and using that as their basis for a ToS violation that's entirely whackadoodle.
> Well they are.
Then that is indeed absolutely horrifying and a problem in and of itself. Privacy concerns (of which there are a multitude) aside, this seems like a really great way to break all sorts of things, and I'd trust the claims of "lightweight", "invisible", and "strict privacy" about as far as I can throw them.
> If CF replaces that html with their own then the browser will likely run it allowing them to run all kinds of probes then run the redirect. The same is not true for a .txt file or an image.
The same is absolutely true for a .txt file or an image. Consider two flows:
1. You click on a link, your browser loads a text file.
2. You click on a link, your browser loads an HTML+JS file, the embedded JS redirects your browser, your browser loads a text file.
Same end result, same client-side probing opportunities, and without needing to rely on swinging JS into the end document like Patrick Bateman swinging an axe into his coworker's face while ranting about "Hip To Be Square".
Or better yet: just don't do this and only care about the raw bandwidth consumed, instead of actively making the World Wide Web a worse place with "clever" tricks like anally probing my browser to see if it's browsery enough for some arbitrary standard of browserness (for the sake of a "protection" that's almost certainly trivial to break with one of the umpteen headless browser solutions anyway).
A bit ugly, but works.