Unintended consequences of blocking IP addresses
blog.cloudflare.com
blog.cloudflare.com
I understand the position of CloudFlare where they basically don't like governments to block their IP's. On a smaller level this is exactly what CloudFlare is doing. Why is that justified?
There was likely botnet activity in the network - on the users' machines connected through the VPN - triggering anti-DDoS protections.
You run privacybadger? To CF purgatory for you!
If you're using CF for DDoS protection, then blocking and captchas are the features you want to use. Just not abuse.
I’ve been planning to use Cloudflare for an upcoming project, under the impression that ‘Essentially Off’ really did mean essentially off, and would only block obvious DDoS traffic. I could understand a legitimate user being blocked if they shared an IP with a host actually engaged in a DDoS attack, but short of that, I don’t want anyone being blocked. It sounds like I should look at other CDN options…
A heading called "Lack of transparency with IP blocking"? Let me get my tiniest violin CloudFlare... This is a joke until they stop serving a stream of captchas for some IPs.
I still mourn the fact industry got conned into DoH and funnelling all DNS traffic on "modern" and "secure" apps thru cloudflare's 1.1.1.1
How so? Do any programs force the use of DoH and not let you configure which server they use?
How is that taking control away from a local sysadmin? It'd be a great thing if they did that, since then malicious network administrators won't be able to disable DoH on computers that aren't theirs anymore. If you're really trying to control devices that are yours, you'll still be able to turn it off via local Firefox policy.
1. People who want to censor and/or surveil other people's devices and traffic. These people hate DoH because the entire point of it is to protect against them doing so, and by running over port 443, it's really difficult for them to block it. This is the group that people who say DoH would be a good thing if only it used a network-provided DoH server, or that they wish DoT would get used more instead of DoH, usually fall into.
2. People who have a workflow that it breaks. In most cases, there's some setting or workaround for that workflow that still lets you use DoH for most Internet queries, though.
3. People who say it's making the Internet more centralized or bad for privacy. These arguments are valid to say "don't everyone use Cloudflare as your DoH provider" but not to say "don't use DoH even with other providers", since there's no reason DoH servers have to be any more centralized than regular DNS servers.
2. Why should their workflow be broken so that the dns info gathered by CloudFlare is more valuable to CloudFlare.
3. The argument is that doh through privately owned servers is bad, so I don't know why you tried to specify that only CloudFlare is bad. DoH is, by definition, more centralized than DNS servers unless all DNS servers implement some form of doh. In which case you're not using doh and you're just updating dns to support encryption. If every DNS server doesn't implement doh then you're just adding a few centralized points which have access to unencrypted DNS data, making that data more valuable to the private entities holding it.
Sure, but "CloudFlare can see my data but my ISP can't" is strictly better from a privacy perspective than "CloudFlare and my ISP can both see my data".
> Why should their workflow be broken
My point is their workflow doesn't actually have to be broken.
> so that the dns info gathered by CloudFlare is more valuable to CloudFlare.
Huh?
> The argument is that doh through privately owned servers is bad
How is it any worse than insecure DNS through privately owned servers, which basically everyone uses today?
> DoH is, by definition, more centralized than DNS servers unless all DNS servers implement some form of doh.
Is IPv6 also by definition more centralized than IPv4, since not all IPv4 servers implement some form of IPv6?
> In which case you're not using doh and you're just updating dns to support encryption.
What are you saying is the difference between those two things? And don't forget there's a huge anti-censorship benefit, even if you don't care about privacy at all.
> making that data more valuable to the private entities holding it.
Wait, are you arguing that reducing the number of entities that can access our data is a bad thing, since then our data will be more valuable to the ones who still can? That seems completely backwards.
2) I don't see why the workflow should be at risk of breaking if there's no good reason to introduce the new tool. Sure it's possible that requiring an animal sacrifice doesn't have to break their workflow, but why are we doing it in the first place?
3) Fully adopted IPV6 is less centralized than IPv4 since the larger address space allows for centralized layers(like nat) to be removed. IPv6 gateways in an ipv4 network would be more centralized since they would require traffic from many sources to be proxied through a single source.
In the same vein, DoH that proxies many connection through a single source would be more centralized than not proxying those connections.
The difference between DoH and updating DNS to support encryption is that the latter doesn't allow for a "CloudFlare" to exist on top of existing DNS infastructure which has exclusive access to unencrypted DNS data.
> Wait, are you arguing that reducing the number of entities that can access our data is a bad thing
It's a bit more nuanced than that. Adding doh proxies on top of existing DNS infasructure increases the number of entities that are required to access your data while decreasing the number that has access to the data to "number thats needed to function + the proxy".
I'm arguing that the number of entities that have access to the data should be "number required to function" or "everyone", "number required to function plus the proxy" only benefits the proxy because they have exclusive access to data. Data is worth money the less people have access to it, so a solution that sends data through a proxy is rife for exploitation and not the best solution.
But that's not really the trade-off here, it's about sharing data with Cloudflare that would not necessarily end up there if you were using services from your local ISP. Whether this is a good idea is more complicated. It depends on how ISPs are regulated and what they actually do with user data. Cloudflare's services, being optional in nature (the website operator or the end user chooses to use them, but not necessarily both at the same time), are likely to be less constrained by law, particularly if you are not a resident of California.
Or put differently, it's far easier to say “you shouldn't have used Cloudflare if you don't agree with their business practices” than “you shouldn't have browsed the public Internet if you don't agree with your ISP's business practices”.
This is me, with the caveat being that it's my damn device. It's only the OEM trying to say it's theirs and that I'm "censoring other people's traffic". If I bought it, I should be able to do as I like. If I tell it the DNS server is local and/or that domain is elsewhere/non-existant, I don't want it deciding otherwise and sneaking traffic out over port 443.
In your specific case, that means that while devices should offer a configuration setting for which DNS server to use, it shouldn't be via blindly listening to the possibly malicious DHCP server.
Because you shouldn't be able to control other people's devices just because they happened to connect to your Wi-Fi. And you don't have to configure them all individually anyway: you can use Group Policy, MDM, etc. to configure that setting on your whole fleet at once.
Why? If it's my network, why should I not have control over all the traffic on it?
Unfortunately, with DoH, this now means that I have to go scorched earth and block all common DNS server IPs at the firewall. You use my gateway to resolve (DNS - 53) or you're out of luck.
I suppose it's only a matter of time before even the cheapest IoT junk just establishes a VPN to its maker's cloud and sends zero unencrypted traffic.
In theory, sure, but in reality they are more centralized.
Anyway, the fourth category is people who want to own their devices on their network and are being fought by the vendors of their Internet of things devices. When I have dns queries I can build a pattern of what it takes some rando device to operate and then lock down and alert on anything else. Can’t do that with DoH.
With DoH, I just have to allow opaque DNS smuggling to the wider Internet and hope that the device hasn’t been compromised. It’s trivial to run bidirectional c&c over DNS and DoH makes that invisible to anyone. It’s a monumental step back in security for the local network to improve the privacy of the individual device.
It’s completely fine for there to be conflicting goals even held by me. I want to not have my traffic interfered with when I’m on someone else’s network but I don’t want the vulnerable internet of shit stuff to be even more opaque on my network.
Here's a couple of thoughts I've had about Cloudflare recently;
One is that Cloudflare are a problem because they are self appointed policemen who do not know who the good guys or bad guys really are. Such well intentioned but naive "helpers" almost always cause more harm in the world then they solve.
Another is that Cloudflare do not understand the nature of Free Speech at a fundamental philosophical level. There are two essential sides to it. The freedom to write/speak must be matched by the freedom to read/listen. Cloudflare's model pf the world recasts this as a "tradeoff" and pits the speakers against the listeners. It does this because there is money to be made from "service providers" but none to be made from ordinary internet "users". It robs Peter to pay Paul.
Solving this in a way that doesn't block tons of regular users (since doing so would cause the site owners to drop Cloudflare) is precisely where nearly all of Cloudflare's $15B market cap comes from.
It's not a binary solution. Cloudflare very much does block tons of regular users, which is where all the hate is coming from in this thread. If the solution is in the domain of "a bag or squishy heuristics" it's going to be somewhat inaccurate, so then the only question is tuning... how many false positives are acceptable, which depending on the area could be anything from "how many can you get away with" to ">0 hurts our bottom line".
To reframe the problem in the latter, consider "The optimal amount of fraud is non-zero" [0]. Where it's understood the cost of inconvenience to customers ultimately will also hurt the business's bottom line. So instead the balance is very much in the favour of the customer, to make sure the wheels stay greased businesses eat the vast majority of the fraud where they could employ stricter but slower methods to verify funds etc.
There is this cost benefit balance in many things. Some things naturally balance themselves, especially when the ultimate bottom line is monetary... others not so much.
I suppose the problem with serving requests is twofold: firstly it's not necessarily a business, and even if it is, an individual visor represent a very tiny peace of the pie over their entire life. Second, bandwidth is paid for twice, by both the visitor and the provider... It could be argued this whole problem wouldn't exist if it weren't for the latter. At most DDoS problem may still exist. Either way the ultimate cost is to fairness, people are discriminated arbitrarily. It also depends on awareness of the site owner, if they care about fairness and know the cost of using cloudflare is potentially unfair to visitors, they may not bother... unfortunately I think most site owners don't realise how many false positives there are, and I'm not even sure Cloudflare does, I mean how would they, you get blocked enough you just give in and close the page, and they think they did a good job it's a negative feedback loop.
[0] https://www.bitsaboutmoney.com/archive/optimal-amount-of-fra...
Nit pick: it's not "actions" impacting others. It's when person A exercises their rights to impede anothers. This kind of argumentation that cloud providers don't provide conduit or extenders for free speech is pretty lost on me, but I'm also of the opinion that all of these services should be highly regulated or government owned so that these silly pedantic, and at times opportunistic, arguments stop. It's become the wedge issue of the internet.
The courts have precedent that would cover this if online speech equals free speech. Part of the reason I want the government to assume responsibility is because it'll either force them to acknowledge they're the same or craft specific laws for online speech. Part of people's frustration is that the rules are all over the place.
> Even if you do, and everyone's in agreement, you're effectively sanctioning off the social media sites to only allow posts from US nationals
That's a silly conclusion. Do US companies only enforce US fraud or sanction laws despite operating in another country? The answer is no. There's regionalization baked into services operated on other regions. I've worked on such services.
Websites, as they globalize, have to do this with regular frequency.
If cloudflare actually only blocked bots, a lot of criticism towards them wouldn't exist. Personally, I have some more abstract concerns about the position they occupy, but the primary reason that I emotionally dislike them is because they like to block me and then pretend that it's my fault ("are you sure you aren't infected with malware?").
1. The internet worked fine before Cloudflare, and there was no mass blocking of IPs before that.
2. Websites are commerce based. Their operating costs are already far covered by their income (ads or a service)
3. Cloudflare is a CDN so customers feel no impact from excessive traffic unless their site - which 99% of the time is pure static content - is poorly designed
4. Cloudflare could just choose to throttle the highest traffic IPs to one particular site (the one being attacked) during an actual attack, which is what DDoS mitigation companies do. Instead they just block every shared IP address forever (or force them to solve captchas, which after 8 years of bad implementation, moved from from wasting tons of the users time to being just barely acceptable)
Don't give me any further lectures on how businesses work until you understand how technology works.
It's almost like all intermediaries are responsible for this damn mess.
Cloudflare's power fundamentally comes from the good job they do protecting website owners. GP may not like it, but many website owners clearly feel they need the protection.
There is no Grand Moral dilemma here, just basic tradeoffs between costs and availability. No different than a shop not shipping to a country with high shipping costs.
The power came from "free" CDN as loss leader (which is clever because the long tail of readerless sites costs essentially nil to cache but will still bump up the NPS).
Cloudflare but also others from small to large services justify blocking IP addresses using the basis of "some IPs being a source of too much trouble"... but this doesn't make sense in an internet age of highly NATed and highly recycled IPs.
One IP != one person, anything based on this assumption today is severely broken.
The issue is not blocking of IP addresses, the issue is one company hosting almost half the internet and having too much power causing such unintended consequences.
On the other hand, we rarely can opt out from the government blocking.
This is a little poetic. :)
Cloudflare is often blocking legitimate access to its customers' Web sites, sometimes triggered by browser privacy&security settings and blocker rulesets, sometimes triggered by using Tor exit nodes.
I wonder whether Cloudflare customers are aware of this (or does it just show up in metrics of "bad" requests/hosts they've blocked), and whether the customers would prefer that those users could access.
From a privacy and security perspective, I wonder whether Cloudflare often blocking a user from accessing a site they want to access encourages people to disable privacy and security measures?
It would be really nice if IPV6 one day actually becomes default so we no longer have to mess around with this stuff.
Cloudflare allows customers to configuring blocking and captchas which is very different.
(I'd noted this the first time I'd read the blog post a couple of weeks ago.)
My favorite experience was building a website for a client, putting cloudflare in front of it, then discovering cloudflare blocked the IP of our client's office from seeing their own website.
Before you step in and go “captchas aren’t blocking!”, consider that the end result is exactly the same with just a different statistical success rate. Cloudflare has systematically discriminated against privacy sensitive users for years based on IP and now they cry foul when it happens to them. Hilarious.
Maybe this will have a decent outcome in provoking more interest in decentralization.
How so? The outcomes could be different depending on how different the statistical success rates are
Right now it's broken again, they just give you a captcha that breaks 90% of the time and fucktarded propaganda points that don't even make any sense to anyone over 30, like "in 2003, botnets were 1000-5000 computers now they are millions".
But this is real rich coming from the #1 blocker of people on the internet. What a lack of self awareness. Yes, IP blocking can be bad, but other forms of blocking that cloudflare engages in are just as bad and not everyone has cloudflare's MITM market position to exploit.
Unfortunately there is little you can do about such collateral damage other than wistfully wonder “what if IPv6 had been implemented widely, and properly, some time ago, so such address sharing wasn't necessary”. VPN users might still end up coming from the same address/range so be indistinguishable from bad actors using the same VPN, but that is their choice to be unidentifiable and they should be aware of the ramifications†).
--
[*] It can impact those with a dynamic address similarly, but more randomly.
[†] I'm not saying not wanting to be identifiable is not a bad thing: just that you need to appreciate that being indistinguishable from the average non-identifiable Internet user makes you, erm, indistinguishable from the average non-identifiable Internet user[‡].
[‡] Who is trash. Or a bot written by trash.
If your IP address is for some reason blacklisted by Cloudflare, you're screwed. With CGNAT, this means that you can have a hard time visiting many web sites if your IP is used by someone else who does nefarious things.
And even if they did agree to block e.g., Austrians from accessing some webpages of some Cloudflare costumers, they would have to base it on some geolocation, which is just one more can of worms.
As for doing legal content blocks based on geo-location, that is not a can of worms. It's table stakes, and I find it hard to believe they wouldn't already have that capability.
Because their customers are breaking the law? I agree that there's an issue with how to handle jurisdiction, but you can hardly ignore a court and then get mad when they take action against you.
Blocking is often used as an easy alternative to prove in courts that laws has been broken.
For example considering the list mentioned in the article (and I do not know which ones affected cloudflare): http://netzsperre.liwest.at/
Why should a US company censor a Russian website (rt.com) because it is on a Austrian list because of a EU regulation? Austrian citizens reading rt.com are not breaking the law.
And I better add, that I am not claiming that the content of those sites are trustworthy, but I do feel insulted that they do not trust me to read if I want to.
In an essay that seems to be a sober technical analysis of the issue, it's noticeable that they didn't even bring up this solution. I guess they don't want that solution either, so don't want to bring it up, not even to explain why they don't want it, it would only confuse things. But, I mean, we're going to think of it anyway...
So the court says "fine, just block Cloudflare's servers at all Austrian ISPs. Perhaps that will get their attention."
See, Cloudflare wants it both ways: They don't want to have to tell rights-holders who their free-tier, movie-thieving customers are, but hey, that doesn't mean you should block their servers. Right?
But yes, they want to have it both ways -- the only way available to you to block is an overreach that harms the internet, and we are not interested in providing you other ways to block.
Maybe if Cloudflare actually paid attention to email sent to their abuse address, they'd have seen some correspondence. They ignore it, and, after avoiding answering questions for ages, finally admitted that they ignore it.
They are so complete hypocritical, since they block IPs all the time, and they couldn't care less about collateral damage, unless the collateral damage happens to be their paying customers.
Curious what the conversion rate is from "outgoing click registered at $price_comparison_website" to "purchase made" before and after enabling cloudflare or a similar vendor, for those who have such stats available to them.
i think TFA summed it up such as:
"It would be hard to imagine, for example, that a court in response to alleged wrongdoing would blindly issue a search warrant or an order based solely on a street address without caring if that address was for a single family home, a six-unit condo building, or a high rise with hundreds of separate units. But those sorts of practices with IP addresses appear to be rampant."
The archives of Off the Hook are full of such examples https://2600.com/offthehook/
i hungaround on astalavista quite a lot when it was the thing.
The problem with this article I have is it says "no IP banning" without, as far as I can see, suggesting an alternative which will work well in practice.
DNS blocking can work, but requires hiring every DNS server, a major undertaking. Laws are no help if one country wants to block access to a server in another country.
If they cannot figure out a way to block malicious actors, they cant just randomly block every second request - though that would also work just as well. Malicious people would just try again, and the collateral often wont. Same result.
The simple solution is for Austria to ban or sanction CloudFlare if they think they are doing something illegal.
If you think my trucking company is smuggling drugs, you don't just ban a few trucks with certain license plates regardless of cargo.
Cloudflare ignores 100% of email sent to abuse@, so take their claim of never being notified with a huge pinch of salt.
I guess it felt like the obvious conclusion is that IPv6 would allow for this sort of individual site blocking but they didn’t want to say that so spent a lot of word count trying to make it sound ridiculous. I hope I simply misunderstood something because it felt very dishonest.
I think part of the reason this feels dishonest is that they make the analogy of a skyscraper and it’s street address - obviously many tenets must share. But that relationship becomes inverted with IPv6, now everyone can have many. Problem solved.
Except they deny this by comparing the size of the DNS namespace to the IPv6 address space and stating because it isn’t 1-1 that it doesn’t work. That argument doesn’t follow. They even point to the early internet as a time when everything had its own IP address and that sort of IP blocking made sense, which it would again under IPv6.
I’m not sure what you are getting at with your example of RAM - if I had 64 bit addresses then I would expect that all of them should be treated as valid, which they are. It is the same here. Not every address in a 128 but space would resolve to something valid, but any of them could.
Is blocking a /64 a fundamental limitation? I dont see how it is relevant otherwise.
I'd say not right for the following reasons: 1) The article never states anything about the demand of IPs exceeding the number of IPs 2) The article never claims such a lack of IP supply is the reason for shared addresses.
These may be your reasons for using a shared IP but that doesn't mean it's everyone's or the only reason for using a shared IP and it certainly doesn't mean that's what the article is written about.
> So in a hypothetical world where we are using IPv6 no one would need to share hence there would no longer need to be any collateral damage.
And so being in this hypothetical world does not actually imply there are no more shared addresses it just implies address demand isn't a reason for shared addresses. Things like convenience and scaling still drive sharing services on a single IPv6 address in the real world.
> Except they deny this by comparing the size of the DNS namespace to the IPv6 address space and stating because it isn’t 1-1 that it doesn’t work.
They never really deny this as they didn't even consider the possibility they were supposed to be confronting that argument in the article. This section is purely an answer to the question they pose immediately prior "Here’s an interesting question: could we, or any content service provider, ensure that every IP address matches to one and only one name? The answer is an unequivocal no, and here too, because of a protocol design -- in this case, DNS.". When read as if the article is making the argument for why shared addresses are used I could then see where you're coming from but the article never talks about the why it just says they are. This section is actually arguing the design of the internet never intended them to have a 1:1 mapping not that the lack of such a mapping is why Cloudflare (and others) use shared IPs.
That may seem an extraordinarily pedantic difference from what you said but when you realize the article spends most of its time showing the internet was never designed with the intent for names to have dedicated addresses instead of explaining why shared addresses are in use it makes more sense and completely changes what that section is talking about.
> I’m not sure what you are getting at with your example of RAM - if I had 64 bit addresses then I would expect that all of them should be treated as valid, which they are. It is the same here. Not every address in a 128 but space would resolve to something valid, but any of them could.
Well it's all a bit moot to bring up if you're not familiar but the whole point of the analogy is that many initially expect all 64 bits of virtual address space to be used then are surprised to learn half the added 32 bits aren't even valid as virtual addresses then sometimes even less than that are valid physical memory addresses.
Similar things happen in IPv6 but to an even greater extreme. The internet of having such a large 128 bit address space was not to have 2^128 addresses or the like it was to make things simpler by using vast swaths of the space on convenience or getting around real world scaling limitations. That is to say the goal in expanding the address space so much wasn't to forgo things like shared addresses in favor of always unique ones even if one of the goals was to reduce address scarcity.
If that is too much of an ask, what are some examples of why we do it now? I ask purely from curiosity.
As for the RAM thing, I believe I understand you but I do not see how it relates. TFA says that ip blocking wont work even with IPv6 because the IPv6 space doesnt map 1:1 with the DNS space. That appears to be the wrong analysis, who cares if you arent able to exhaust the IPv6 space anyway? The way I am understanding what you wrote is that the space isnt as big as labeled, but the bar I am measuring against is "functionally limitless".
After writing that I think i can sum up as; if the goal is limiting or preventing collateral damage from ip blocking then ipv6 would fulfill that goal by providing functionally endless addressing space because the natural inclination would be to use as many addresses as you pleased, so blocking one or a set of ip addresses would typically only impact access to one web site.
p.s I appreciate the attention to detail, caring about nuance is never pedantic imo
The RAM thing (and the IPv6 doesn't map 1:1 with DNS thing) aren't about whether or not there are enough addresses it's about explaining why we use large address space to enable something more than having the most addresses possible. That is to say to show the intent in adding more addresses wasn't to turn IP into something that's supposed to be good at providing unique identity it was to do other things. E.g. arguably IPv6 is really a 64 bit protocol, the upper 64 bits are really there for convenience of the end subnet always being the same size (/64) and easily encoding existing client info into the address. Even the largest network gear isn't designed to handle much more than ~16 bits of unique IPv6 client endpoints in all subnets combined yet a single subnet has 64 bits of client address space. A similar thing happens on the internet itself, when we advertise networks it's never smaller than a /48 because we need to be conscious of how the internet route table scales and fitting it into hardware over time. Again that's not to say there isn't some way to encode 10 billion services into IPv6 and have it work it's just further proof the IP layer was never the one meant to provide this type of functionality so we shouldn't try to shoehorn it in and should instead let it focus on being the reachability layer.
And that really boils down the reasons for why shared hosting - it's convenient, it scales better, and it's the better way to do things. One could chose to do things inconvenient, in a poorly scaling fashion, and with more limitations in how you do things and gain the ability to block by IP instead of by name but it just seems a horrible trade off.
And to clarify I'm not one of those anti-IPv6 nuts - I actually run a lot of IPv6 only infrastructure directly on the net through AS400503. Even though I have a /40 all to myself, enough for 65536 /64 subnets, I still do shared web hosting with the v6 addresses because it's less work to do so.
Im having a hard time believing it was an editing mistake or clouded thinking because the IPv4 vs IPv6 thing is entwined through the length of TFA. Not that I think they have some nefarious agenda, if I had to guess it reads like they already knew what conclusion they wanted to reach and had a deadline. Which is still dishonest.
Still, if it didn’t read like there was a glaring admission to you then I am glad of it. I would rather be wrong about that kind of thing anyway.
Most likely Roskomnadzor at work. Proving it definitively is difficult.
This site had an awesome writeup which helped:
https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict...
(Packet trace showed a match to one of the methods listed)
I posted already in the thread an OONI analysis of how Russian ISP's are blocking IP's. Super awesome stuff they do.
But no.
Easily unblocked by going with a different DNS service.
> The second approach is to block individual connection requests to a restricted domain name. When a user or client wants to visit a website, a connection is initiated from the client to a server name, i.e. the domain name. If a network or on-path device is able to observe the server name, then the connection can be terminated.
Isn't that what encrypted client hello is meant to actually prevent? I just turned it on just for giggles
1. Each domain you loaded that had Cloudflare asked you for a captcha. You would normally have to solve one captcha to unblock yourself from the site, then right click on the images that didn't load to get the CDN subdomain of the site, then solve another captcha to unblock yourself on that subdomain.
Eventually, in 2018, Cloudflare started allowing Tor traffic iff you use Tor Browser (it does fingerprinting). So I was right, they didn't need to block Tor, contrary to the idiots who would argue that there is no alternative.
Also, it was a multi minute process to solve one captcha as they used recaptcha, which has always been the worst captcha in the world, and varies from giving Tor unsolvable problems to blocking it, to giving borderline unsolvable problems. So after a few minutes you can get past one captcha after getting lucky enough to get a few borderline unsolvable ones. Eventually hCaptcha came along and Cloudflare eventually switched to it and now there is no problem, so I'm right, contrary to idiots who argued with me claiming there is no alternative.
It's still garbage though, thanks to Cloudflare the web is crippled in yet another major way. You still need to solve a captcha if you use Tor and trip the fingerprinting and WAF crap, despite that giving no security benefit to the website.
Oh yeah, and they leaked private bank sessions across the internet because their little blocking scripts were too buggy (the Cloudbleed vulnerability).
I wonder if that judge wrote an open letter saying "damn I really hate to block IP addresses [...] so here's why I decided to block IP addresses"
But of course zone files only list domainnames, not websites. More details needed on how they came up with 255M websites using top-1M website lists.
Using "top-1M" lists will bias the results because those sites are more likely to use large hosting providers and CDNs. In other words, it will exclude smaller, less popular websites using a smaller hosting providers.
"By looking at the CDF there are a few eye-watering observations:
Fewer than 10 IP addresses are needed to reach 20% of, or approximately 51 million, domains in the set;
100 IPs are enough to reach almost 50% of domains;
1000 IPs are enough to reach 60% of domains;
10,000 IPs are enough to reach 80%, or about 204 million, domains."
No surprise here. If we restrict the www to only what is popular, i.e., high-traffic, e.g., using concepts like "top-1M", then yes, the number of IP addresses we need will likely be fewer. For one because these sites all use the same handful of service providers that target high traffic websites. "Top" lists make it easier, more tidy to work with what the web actually may comprise.1 It lets "tech" companies and their service providers focus on what is commercially viable. Just ignore all those unpopular websites that cannot serve advertising. But what gets filtered out. We are prevented from knowing. Similarly, centralisation lends itself to convenience. There are obvious benefits. However, needless to say, centralisation is not appropriate in all cases.
1. Another analgous situation IMO is mobile apps. Allowing consumer to know what apps actually exist, i.e., all the millions of apps people have written, including the ones that will generate no revenue for anyone, is jettisoned in favour of "Top 10/Top20" lists or similar popularity filters. The "app store" middleman censors certain software and shows users only its view of the world's mobile app production. Searching is blunt. In the same way Google only shows the user its view of the www, rather than the actual www. Filtering can be useful, but ultimately these filtering decisions are for the benefit of the companies behind "app stores" or "web search engines". A "tech" company selling CDN services, or advertising services, is not a librarian at a university/public library. It has commercial interests. At some point "filtering" becomes "funneling", or herding.
Help, I am drowning in the irony of Cloudflare of all people making this statement after being the company who normalized service providers kowtowing to internet mobs. Sending threats to the family members of Tier 1 provider executives to get sites blackholed is now a thing because of Cloudflare and they're going to complain a court ordered a block of eleven IP addresses.
It would be funny if it weren't so sad, the fact that either Mathew Prince was lying to everyone or Cloudflare CEO Mathew Prince doesn't know how to take a screenshot.