Hacker News Reader

Cloudflare just isn't worth the tradeoffs: the risks associated with their centralization, how they made Tor basically unusable on non-onion sites, the lack of transparency when content-moderating the internet, etc.

The space is in need of solid competitors to break the stranglehold they have on the internet. Whether it's the right combination of services, documentation, etc.

there are many solid competitors: Amazon, Fastly, Akamai, Imperva to name a few

Bunny

There is an easy way to get the banhammer from Amazon, and it is possible host a JavaScript page that triggers it for any visiting user.

I did tell Amazon about it, but it fell for deaf ears. The ban lasts for about a week and the internet is mostly unusable in that period

Fuck Akamai. Have you worked with them? They are the most archaic internet company you can think of. Their UI is stuck in 2000. Just like their procedures.

Tor made Tor unusable on non-onion sites. I feed a netfilters table with the list of exit node IPs that Tor publishes (https://check.torproject.org/torbulkexitlist) as a standard part of server deployment, and it's the single most effective way to reduce form and login abuse on hosted sites. I like the idea of Tor, but there's no denying that it's a huge source of nuisances.

I live in a country with censored internet. What you are doing is harmful. I can only hope whatever you provide is irrelevant enough.

I'm sorry. I have a colleague based out of Venezuela. We've had to work together to get tunnels and vpns configured so that he can get uncensored and secure internet access.

But Tor is an enormous source of abusive traffic and if I don't filter it, then that's harmful to site owners. I'm being forced to choose between the needs of people that I know, work with, and depend on financially, and the needs of people in countries with issues that are far outside my ability to resolve. It's not a hard decision.

> It's not a hard decision.

Depends on what you imply under 'hard'.

As a IaaS provider I endured alk the hurdles about that and ten years later - I don't care, at least not until my outbound bill is bigger than usual.

Like some of the clients are on CentOS6, on a public facing machines.

There are probably more sophisticated options that would solve your problems than simply blocking it.

Such as?

The answer depends on the type of service you host. I don't know what you need to do, but I do know that filtering IP space is merely security-by-obscurity, it is a cheap and broken solution to the hard problems of sybil resistance. If you need IP filtering to operate on a day-to-day basis, then the security of your service is fundamentally broken.

Tor users do not have any special properties over clear-net users besides low accountability for their IP space. There are other ways to acquire this type of setup that don't involve broadcasting a public list of known exit nodes as an act of good faith. Any sophisticated attacker will be able to easily get ahold of the IP space and bandwidth they need to do their work, whether it's through a botnet or simply because they operate out of some less-accountable country like China or Russia.

IP filtering: now you have two problems!

This is why I'm strongly against spam filtering for email. Spam filters are fundamentally security-through-obscurity. I mean, they don't protect your email from targeted bombing attacks or phishing. If you need spam filters to operate your email on a day-to-day basis, then the security of your email is fundamentally broken.

/s, obviously, I hope.

Blocking Tor isn't a security measure, it's a nuisance reduction measure.

>This is why I'm strongly against spam filtering for email. Spam filters are fundamentally security-through-obscurity. I mean, they don't protect your email from targeted bombing attacks or phishing. If you need spam filters to operate your email on a day-to-day basis, then the security of your email is fundamentally broken.

You kid, but this is completely true Email is simply an incredibly flawed, outdated and broken system, especially when used without PGP. Phishing is a massive problem, and it has only continued to grow in scale because spam, uh... finds a way. At the same time, spam filters regularly create false positives, making email an unreliable transport (leading "oops, it got lost in my spam folder").

>Blocking Tor isn't a security measure, it's a nuisance reduction measure.

You should block all IP space, this will reduce nuisances by 100%. In fact, this will save you from having to consider any real security practices or do your job properly.

The correct analogy here would be implementing spam filtering by blocking large segments of email addresses. Eg, dropping mail from all non microsoft/gmail domains (as a nuisance reduction measure!), with predictable impact on smaller providers and self hosted email.

I think it would be sensible to block new account registrations with addresses from email address aliasing services (e.g. duck.com) or disposable email address services (e.g. mailinator.com).

You're reframing this to make Tor look a lot better than it is. The signal:noise ratio for Tor is epsilon. It's almost entirely garbage. If a network generated spam at rates analogous to network traffic from Tor, yes, I guarantee that network would be on every single email service's block list.

Tor's advocates in this thread keep trying to argue it from ideology, as though anybody's obligated to deal with Tor traffic on principle alone, and not one of them so far has tried to argue that Tor is not 90+% bots and garbage. Funny, that.

With all the blocks in place, is it ever possible to know whether the 90% is still an innate effect of Tor, or actually an effect of sites blocking Tor?

I have Tor installed, figured it would be worth adding my boring browsing to the mix sometimes, but since most sites I try to load block Tor exits, Tor browser now sits unused.

On the other hand, if I woke up tomorrow deciding to start a bot farm or whatever other malicious thing, or course I'd be interested in hiding through Tor and might try it again (don't worry, I won't wake up that way).

So even if a hypothetical 100% of global internet users really wanted to do all their browsing through Tor, they might all reach the same conclusion as me that too many sites are blocked and therefore leave Tor to mostly bad traffic. Of course it's nowhere near 100%, but hopefully you see my point that the sites blocking Tor IPs (and I absolutely appreciate why) can become a self-fulfilling prophecy - and I'm not sure how you'd get out of that loop?

And if everyone blocks all non-gmail addresses then soon enough the snr of non-gmail addresses will also be garbage because you are actively preventing any legitimate user from using them.

I am strongly against any kind of spam filtering that drops/rejects messages that the recipient did not intentionally configure for those kinds of messages. Sorting suspicious mail into a separate folder is fine, preventing two humans from communicaing based on heuristics, IP block reputation and other such bs is not.

Outlook did that actually (preventing two humans from communication without reason)

https://www.linode.com/community/questions/22305/entire-ip-r...

Is using CAPTCHAs one of those?

Most captcha services are just used to force users identified as having few other options into giving free tagging labour.

captchas are fine. recaptcha is not.

between 2015 and ~2020, my home ISP was blessed with every recaptcha being 3 rounds of slow fade-in bullshit. I have also seen infuriating gaslighting of "please try again" after certainly correct solutions, as well as 5+ rounds followed by a notification that my network is entirely blocked.

I've developed a reflex to Ctrl+W upon seeing it, unless that is absolutely vital for me to get past it - which is exceedingly rare.

if I had a genie lamp, I'd waste one of my 3 wishes to do terrible things to the people responsible for that shit.

I'm a noob, can you give me a pointer?

What kind of abusive traffic is coming through Tor and why do they do it?

https://blog.cloudflare.com/the-trouble-with-tor/

> . Based on data across the CloudFlare network, 94% of requests that we see across the Tor network are per se malicious. That doesn’t mean they are visiting controversial content, but instead that they are automated requests designed to harm our customers. A large percentage of the comment spam, vulnerability scanning, ad click fraud, content scraping, and login scanning comes via the Tor network. To give you some sense, based on data from Project Honey Pot, 18% of global email spam, or approximately 6.5 trillion unwanted messages per year, begin with an automated bot harvesting email addresses via the Tor network.

Say you're running an account take over script that spams login forms with a list of known username and password combos. If a website owner sees thousands of login attempts coming from a single IP address they're likely to block you to prevent abuse on their website. This is annoying for you as you then need to rotate your IP address.

Using tor hides your IP address from the website and makes switching exit nodes very straightforward, so you can run your account take over script in peace.

That's not that easy in practice. There's less than 2k exits normally, not all of them usable. Your abuse script competed with other malicious traffic for those exits and their reputation gets burned pretty much immediately.

So yes, you can switch exits easily, but effectively your switching from one known bad IP to another bad IP.

Mainly forms -- login forms, comment forms, signup forms. Bots use Tor pretty heavily because it's anonymous and hard to block them without blocking the entire network. Login form abuse is mildly irritating but not a huge deal if you have other measures in place. Comment spam is annoying but there are some options that deal with it pretty well.

But the signup spam was a headache. I didn't want to just blackhole Tor traffic, and tried to reduce the abuse with other tools, including some custom stuff. The final straw was a customer's small business site that had a MailChimp or Constant Contact signup form. Those vendors want you to embed their code by default to render the form, so you have less control over the form itself. There were workarounds, but they all sucked.

Tor bots would sign up email addresses through this newsletter form, and then I'd have to go through and manually scrub them before newsletters went out, or the service would penalize my client for too many bounces/unsubscribes/complaints. Very nearly 100% of the abuse on that particular form came from Tor IPs.

I do not want to spend my limited time on this Earth manually sorting out bots from humans because of one particular network. Blackholing Tor made that problem disappear immediately.

VPNs are dime-a-dozen now, cheap VPSs are available from lots of vendors, there's Wireguard, there's ssh, a clever person could even set up Apache or nginx as a forward proxy with ssl from LetsEncrypt. Tor is well over 90% abusive traffic (https://blog.cloudflare.com/the-trouble-with-tor/). This is a Tor problem, not a me problem. There are better alternatives available.

I think the workflow is the issue with http(s)-based email list sign-ups.

Solution: Require sign-ups by email, so the end account must actively send your mailserver a registration message. This also turns an open-loop control system into a closed loop control system, which is inherently easier to secure / keep safe.

How would this be better? It's trivially easy to spoof email addresses. Someone could sign you up easily, for example.

It's also easy to send "from" an addresses that passes SPIF/DKIM but bounces inbound mail -- not sure what reason someone would have for this other than hurting the service reputation or acting as a DoS of sorts, but it can be done.

> It's trivially easy to spoof email addresses. Someone could sign you up easily, for example.

Proper DMARC configuration is table stakes to send e-mail, which makes that anything but trivial.

But neither the newsletter host nor the email user has any input into how dmarc/dkim/spf are implemented. Only the user's email provider does. And if that's a small business domain, it's likely not very strict with the rules.

I thought DMARC/DKIM was necessary for delivering to Gmail for years now; in any case, there should be few who can't use a backup email to subscribe, as your newsletter won't be the only thing that has these anti-spoof requirements.

Not necessary. Just very highly recommended. I can still deliver my cron emails from a rando host successfully.

That doesn't rule out DKIM, which only requires the `From:` header's domain to list a pubkey and the email to include a DKIM signature from a matching private key. SPF is the one that regulates which hosts a domain's outbound SMTP servers are on.

> Mainly forms -- login forms, comment forms, signup forms. Bots use Tor pretty heavily because it's anonymous and hard to block them without blocking the entire network. Login form abuse is mildly irritating but not a huge deal if you have other measures in place. Comment spam is annoying but there are some options that deal with it pretty well.

Then put the form behind your monopolistic internet gatekeeper. There's no reason for a GET to redirect to a sysiphean captcha treadmill.

How often is the list of exit nodes updated?

Daily, I believe. I don't have the file git-controlled. That would be a good idea, though.

> how they made Tor basically unusable on non-onion sites

I wonder if that's such a bad thing. Tor is safer when the traffic never leaves the network. In the ideal world, everything that matters would be inside the Tor network instead of being merely accessible through it.

Just 10 minutes ago, I got the following email from a housemate (I'm not home at the moment):

> The past few weeks I've been getting tons of redirects to verify my humanity before being allowed to view a webpage. Usually I just have to click the box that says human, not find all the ladders in a photo. SoFi is doing it every single time I log in. Petco, too, along with others who are more sporadic. This is happening with and without uBlock on. Same browser I've always used. ...

SoFi and Petco both use Cloudflare. I do exactly zero web crawling / scraping / abusive anything from my home connection.

I'm noticing a recent increase in volume of complaints about Cloudflare's human verification filter. I'm starting to wonder if they touched a dial.

I had already started pulling some infra back from Cloudflare after their last appearance in the tech news cycle. Now I've got an additional reason to continue doing that.

> I had already started pulling some infra back from Cloudflare after their last appearance in the tech news cycle.

What triggered your reaction? That they terminated a customer with zero notice?

> I do exactly zero web crawling / scraping / abusive anything from my home connection.

That you know about. Your house mates share the internet connection.

I’m guessing you have WiFi, so you may have unintended guests.

You probably have lots of devices, one of which may be infected.

Your ISP may have issued you a different IP which may have a negative reputation score.

You could be using a malware infected browser or browser extension.

There are lots of variables. You haven’t isolated all of the ones in your control, so assuming CloudFlare is the only possible cause isn’t rational.

It's simple, really. Cloudflare is the single root cause of the issue. All the others are not a huge issue until Cloudflare notices. It's perfectly rational and reasonable to blame the company trying to gatekeep the whole internet, without taking any responsibility for false positives.

Scaling to infinity isn't a right, it is a privilege. Any company that builds this sort of no-human-decision systems are abusing that privilege and hoping that anyone who suffers wrongly under their systems doesn't have enough voice (google seems to be the worst for this, though cloudflare seems set to follow).

I'm not sure how to phrase this without sounding like a prick, but I'm not exactly new at this stuff. You missed on all of those examples. I appreciate the point you're trying to make, but Cloudflare is in fact the primary factor here.

> That you know about. Your house mates share the internet connection.

This is actually the least likely these days... the no.1 cause would be CGNAT, the vast majority of residential endpoints share an IPv4 address with a huge number of users, mobile networks are even worse... that's before we even get to IP recycling for dynamic IPs which happens at high frequency with mobile networks again, so you will inevitably get affected eventually.

This is why it's a bad idea to block IPs outright, because today one IP address never equates to a single individual or the same set of individuals over time. The other problem with blocking IP addresses based on abuse is considering them equal in user weight, yet one IP might have 2 users, another might have 10000 users - Blocking a TOR exit node is a good extreme example of this... people think of it as an effective defence because of the concentration of abuse on that single IP address, but they fail to consider the concentration of users behind that IP address - TOR exit nodes probably are a slightly higher source of abuse per user, but not any where as high as per IP - if you measure abuse per IP you are more likely looking a rough picture of users per IP for highly NATed IPs.

You're looking at it all wrong. From Cloudflare's point of view, this kind of blocking is a feature. Anyone doing legitimate web crawling, or offering alternative web services such as Starlink, now needs Cloudflare's permission.

Essentially, for a broad class of web-based businesses, they have made themselves gatekeepers. I'm sure they'll find a profitable use for this position. Charging outright would look bad, but investing in businesses that just happen to not run into Cloudflare-based trouble, but whose competitors do...

I'm familiar with that perspective, and biased towards it... Cloudflare is certainly in such a position, but they are a relatively young company (for their size and reach) and I've seen good things come from them.

I'd guess the intent is unlikely to be anti-competitive or monopolistic, just over-aggressive. However regardless of intent their position does cause an absence of market forces to put pressure on fixing such issues - Similar to how it's become acceptable to have downtime when it's on AWS, because "everyone is affected".

It's true that any wall around you that protects you from something unavoidably comes with a gate that someone else guards, unless you want to guard your gate personally (and that entails filtering out armies of spambots, worst case manually).

As with any power and control you delegate to any entity, only time and good behavior will earn them your trust. That's theoretically what companies are competing for, your loyalty.

Isn’t there an config option to dial down the anti-bot stuff so that you still get the benefit of Cloudglare’s caching but with much less chance of dropping legit traffic from schools, VPNs, etc? I think their lowest setting only really kicks in if they think an ip is participating in a DDOS attack.

My dude, it isn't about money. At least not directly.

I encourage those of you attempting to block Cloudflare to try and host your own website for a bit. Make sure you don't do it on a metered/paid connection. I know one eCommerce site with 1,300 employees that went bankrupt overnight thanks to the AWS bill (and lack of options to get back online, this was prior to companies such as CF). Bankruptcy as in the company filed for bankruptcy and no longer exists. They were profitable for a decade prior. One DDoS attack...

Also make sure you don't have a democratic opinion if you are in the US, like a 50 person manufacturing company. They were shut down completely thanks to saying a single wrong thing about Republicans. CF existed there, but they weren't aware thanks to not having IT folks. They were a non profit.

CF may be evil to some, but there is a reason they exist. I use CF. I don't like throwing money at them every month, however, many of my websites have also been attacked, usually via competitors. We can either deanonymize the internet or allow companies like CF to exist. There is really no other way.

> I know one eCommerce site with 1,300 employees that went bankrupt overnight thanks to the AWS bill

Had they just called AWS and explained the situation, they would likely still be in business.

I keep a backup DDoS mitigation service for my entire network that costs me less than $200/mo to mitigate up to 100 Gbps.

> the company filed for bankruptcy and no longer exists. They were profitable for a decade prior. One DDoS attack...

Being milked dry by a single DDoS is a hosting issue in my opinion, there should be sensible limits in place, AWS is notorious for making it very hard to understand and control this...

Even if you disagree and consider it a problem that must be solved with a separate DDoS protection service, this is not what I am talking about, I think it's a good idea - if there is a clear ongoing targeted DDoS attack, that system needs to engage and do it's best to try to filter through only legitimate users (which is the only point in time it makes sense to potentially block regular users - because the alternative is that no one can access the site).

The problem is this is not how cloudflare's protection operates, there is no throughput trigger, it's always on, it attempts to block bots at all times and has a very high false positive rate.