I'm confused about the ToS comment by Cloudflare. The txt is on a website so it is a web content?
So robots.txt is not supported by Cloudflare to cache/proxy it? That would be a weird regulation. And I bet everyone violates the Cloudflare ToS then.
So robots.txt is not supported by Cloudflare to cache/proxy it? That would be a weird regulation. And I bet everyone violates the Cloudflare ToS then.
2.8 Limitation on Serving Non-HTML Content
...Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately...
A huge text/plain artifact, requested often, would seem to fall into that category of "disproportionate percentage" compared to text/html served.
May be EasyList could host them there? That's what we do [1] (and the dashboards show 400TB+ per mo [2], likely rigged by the traffic between Workers and Cloudflare Cache).
[0] https://news.ycombinator.com/item?id=20791660
[1] https://news.ycombinator.com/item?id=30034547
[2] https://nitter.net/rethinkdns/status/1546232186554417152
What else is important to note that the client is being abused and not the client abusing the service. That should be taken into consideration, when deciding if someone is breaking the ToS.
> What else is important to note that the client is being abused and not the client abusing the service. That should be taken into consideration, when deciding if someone is breaking the ToS.
My understanding has this as moot. The issue from Cloudflare's perspective is only that the content is non-HTML and doesn't have anything to do with the rate of traffic (the abuse).
The key is "as viewed through a web browser" imo, this is not really an API and it's not a webpage; it's a datafile and would fall into R2 or similar things.
Cloudflare is balancing on a razer for this TOS technicality.
To oversimplify, they’re saying Cloudflare’s service is to be used for serving websites to browsers.
Serving a static text file that is primarily used by applications is not in line with their terms of service.
Cloudflare provides a significant service to the free and open web by subsidizing the hosting costs of static content for websites. They give that away for free under what appears to be reasonable terms. I’m not sure why you’re trying to “gotcha” through their ToS.
It would be great if Cloudflare would donate resources to EasyList - it would do a lot to help the free and open internet by giving users more power over what gets delivered to their browser. But call that what it is: a donation.
People are doing the opposite, pointing out the hole and asking them to get a better rule. Surely they don't just want the list merely converted into html.
> They give that away for free [...]
So they should specify things that influence cost such as total bytes served, number of files, etc. Currently all you can do it bypass the rule because you don't know how to cooperate.
Imagine you do that and I DDoS the URL. CF will then mitigate this DDoS by, in part, replacing your html with their Browser Integrity Check html.
If you're serving 'web pages and websites' everything continues to work. What would happen if this list suddenly became an actual webpage.
If your site is serving 'a disproportionate percentage' of non-html you decrease the ability of CF to tell good traffic from bad.
<!DOCTYPE html>
<title>a</title>
So it's not that crazy an argument that a plain text file is a html document
They serve websites to browsers for people to view. This file (be it properly formatted .html or .txt) is not a website people go to in their browser - its used internally by an application. This is the key point.
text/plain though is decidedly not text/html and I would expect CloudFlare to potentially do some on-the- fly optimizations that are aware of the structure of an html file that save terabytes a day at their scale.
Some think its very Oracle of Cloudflare to do so. I do not blame them.
But anyway, just rename .txt to .html and you're done.
Cloudflare just seems to be trying to limit the free tier to "caching website html for the purpose of showing it to humans". They have pricing and plans for things other than that.
At least file extension is limited and externally visible (and thus accountable) to third party behavior, which should limit the worst complexity excesses.
Is filesystem metadata actually different (theoretically) from extension? Or just data in a different format?
Extension seems a nice balance between simplicity / brevity and utility, albeit as a hint, not a commandment.
I mean, one could simply wrap the content in a HTML body and change the extension, but that would actually increase the data load for no good reason. So it is complete non-sense to complain about txt files being served.
I assume you probably know this but just wanting to share there are some pricing scales with R2 they're just pretty generous for a lot of things.
Though, adblocking is a big business, many actors there are getting large revenue.
For example, Eyeo's income was 50 million USD per year last time I checked (and I guess most of it is actually profit), so they can find a solution if they really want.
https://img.phantasmagoria.me/img/96XJrjejoHNdrQv7.jpg
Even if you have a private bucket, you can give people a signed link with read access, for up to two weeks, IIRC.
But it'll still cost them money by number of reads
If they're downloading text you can still use the headers, and some tricks around redirects, but overall you have far less data on which to decide.
> Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately as part of a Paid Service or expressly allowed under our Supplemental Terms for a specific Service.
We will never know the reasoning of the support agent who replied to the EasyList maintainers, but I can imagine that it is indeed disproportionate for EasyList.
I really hope that Cloudflare actually sees that they are making a wrong decision here and actually help the EasyList maintainers.
it's cloudflare deciding to protect "web content" and not videos or .iso images or other things that normally are not commonly served while you browse a contemporary website and read HTML.
That's false in two ways: first, text is normally served while you browse a contemporary website; second, so are images, which are explicitly called out as potentially violating this clause. Text is the only data that isn't covered by this clause.
It's all 1s and 0s too
Even more wtf- the file extension determines the file content?