Hacker News Reader

Comment by grishka | Hacker News Reader

Cloudflare captchas in particular, and any checks and roadblocks to see something publicly available in general, are terrible, period. It doesn't matter which form they take. Every time you see one you feel like a second-class citizen and get reminded that the internet is no longer what it used to be.

I personally simply close the tab when I see a cloudflare "one more step" page.

View on HN

This is completely wrong. Site administrators can put any controls they want in place to limit access. I don't know where you get the idea that things on the Internet need to be publicly available or without restriction.

Unless you're an original ARPANET contributor, there have always been attempts to control access and stop attacks. You're making the same mistake every conservative does. Longing for a nostalgia that never existed.

> I don't know where you get the idea that things on the Internet need to be publicly available or without restriction.

That's not what they said at all. They complained about breaking the access to things that were already chosen to be public.

what? Have you ever dealt with a DDoS attack and the consequences on your availability and infra health?

Of course from the perspective of the website operator it’s great, but from the perspective of the user it’s frustrating.

I’m not sure whether this is true, but it seems like with Firefox I get these captchas much more often than with Chrome. Sometimes they’re so difficult to solve it really takes a minute or two to do so, and it’s incredibly disturbing / an unfriendly interaction.

Surely there must be a better way to deal with this? Why do I have to keep proving again and again and again to Cloudflare I am, in fact, a person?

Because you don’t actually want Cloudflare to store your browsing history on their servers. If they did it would be an insane privacy concern. So every time you get there it is like the first time.

Are ddos attacks a common enough occurrence to warrant putting half the internet behind ddos protection? In my impression you need to do something really wrong to deserve one.

That's unfortunately not true at all in my experience. Maybe if you're an anodyne SAAS, but if you host any user-generated content, especially if it's adjacent to gaming (my personal experience was mostly with gaming-related forums and IRC networks), politics or any other charged topic, expect to get hammered on a pretty frequent basis. IoT botnets are pretty easy to rent at this point, so the attack is accessible to every skid known to mankind.

I actually agree with your overall point as I try to use Tor for a lot of "normal" browsing, but I'm not sure what the correct solution to accommodate both is. It's a hard problem, and having been in that position myself I have a hard time faulting small website operators who have no alternative defenses.

e: just to add to this, I see the existence of ddosing as a significant driver towards centralized monolithic services. If your blog on Palestinian rights or whatever is getting hit, that's an incentive to move it to a platform that takes care of networking for you. It's a little absurd to go all-in on decentralized self-hosting without at least an acknowledgement that with current tech and typical personal-computing budgets, doing so is giving a heckler's veto to literally everyone. Cloudflare isn't the only dimension things can be centralized along.

Yes, they absolutely are. Hell just getting a few random bots scraping stuck in a loop or being overly aggressive on your site is enough to double your bill. So yeah it's 100% required.

> to double your bill

Do you pay a variable amount for your hosting? How and why? All VDS and dedicated server offerings I've ever seen are fixed amount per month. And more often than not the network is limited by speed, not by data transfer.

In that case then the service will just hang as it won't handle the requests caused by even a simple malfunction (not even an actual attack) like the one mentioned by GP.

Mind you, I see your point and I generally don't like the captchas either, but it is definitely a trade-off and I won't blame webmasters that use the DDoS protection.

When you get DoSd you either pay to absorb the traffic or go down. Usually paying a DDoS service is the cheapest option but if you're not, you're paying for more infrastructure (or going down)

Did we collectively forget rate limiting exists or something?

One bot that's just stuck on a loop or being overly aggressive is going to have one IP.

That is fair but instead of bothering to set that up I can just sign up for a free service in a hour once and just never worry about it.

any sophisticated bots are rotating IPs or pulling from a large pool of IPs. In many cases of fraud/abuse, rate limiting based on IP doesn't work well enough.

Yes, attacks on the web are very common for any decent sized site. Just because something is publicly available doesn't mean you get unfettered access to do whatever you want.

How do you mitigate ddos attacks and other bad actors hitting a page?

What does your cdn solution look like?

Route optimization from your (single) endpoint to clients literally half a world away?

As I user, I simply don't care. I repeatedly get punished for doing nothing wrong. It's almost like airport security.

> What does your cdn solution look like?

> Route optimization from your (single) endpoint to clients literally half a world away?

And as a developer, I don't understand this newfangled obsession over CDNs either. Yes, there will be 200 ms RTT in some cases. So what? Get over it. Optimize your website to load in fewer round-trips. TCP congestion control adapts well enough to any latency. RTT only really matters in gaming and VoIP.

I don't think you understand why that captcha is there in the first place then.

Cloudflare prevents a bunch of crap that site operators just don't want to deal with. Especially for smaller sites that are run by one person. Dealing with a wordpress site getting hacked because you missed an update by a day, or a bulletin bored getting swarmed with bots, or some asshat ddos'ing your site because you banned them. Suddenly that site just isn't worth running.

Complaining that about a thing that prevents that headache because it's a minor inconvenience to you is so self centered it boggles the mind.

Yeah, so centralizing the entire internet around a black box that sees all your traffic in cleartext is clearly the right solution. /s

> Dealing with a wordpress site getting hacked because you missed an update by a day

Maybe don't use something this vulnerable then and rely on a third party to protect you from exploits.

> or a bulletin bored getting swarmed with bots

Maybe require email verification and/or a captcha when signing up or posting. Don't punish people for passive actions.

Somehow, there are many forums that aren't behind cloudflare, yet there are no spam bots.

> or some asshat ddos'ing your site because you banned them

Sure ddos is such an everyday occurrence?

I just don't understand. I run a personal website. There's literally nothing to "deal" with. I set it all up once and it works. I only have to pay for the server and for the domains on time.

Monocultures are always bad, but I don't see any alternative services with this level of ease of use.

You're definitely overestimating the technical expertise/available time of a lot small time admins out there.

You don't see bots and spam on those forums either because they are actually using cloudflare, and you're just not seeing the captcha, or because in the backend they're feeding all their posts through akismet (in plain text). I don't think you're considering how many services see your posts, even when you don't trip a captcha.

email accounts are trivial to sign up for, especially for bots. I always recommend charging $1 (or local equivalent) for an account, that's a lot harder to fake.

My point in all this is that bitching that site is using cloudflare to not have to deal with crap, is a self centered view.

Saying "well it never happened to me, so it must never happen" is similarly self absorbed.

Maybe consider that your experience is not everyones experience

> My point in all this is that bitching that site is using cloudflare to not have to deal with crap, is a self centered view.

Who is serving whom here?

If a business thinks it's ok to impose cloudflare inconvenience on me, the customer, for the priviledge of giving them my money, who is self centered here?

The simple answer is I'll close the tab and go buy it from a competitor. I'm not playing captcha games to buy something.

>prevents a bunch of crap that site operators just don't want to deal with

>I don't see any alternative services with this level of ease of use.

Both of these boil down to laziness, IMHO. And for how awful they are for half the internet, that's not okay to me. Not as a user or as a developer.

When a user requests a site they expect to get the site. When you utterly disregard their utterly reasonable expectation and force them to train someone's ML algorithm for free for a minute or two you utterly destroy the user experience.

If that's my experience as a user, your entire site gets an instant 1/10 rating from me. I'll likely leave instantly and never return.

that 200ms rtt does matter to users. it becomes very noticeable. especially when you're writing an app, not a brochure site. You need to tree shake so you're not serving a huge spa all at once.

I've see much worse times for users, and a cdn absolutely help with our staff in asia dealing with our internal apps.

Of course they're not always tripping up cloudflare and being shown captchas. I almost _never_ see a cloudflare captcha either... huh...

What use has an app that requires a full RTT fot every buttonpress?

None? Why would every button press require a full rtt?

>that 200ms rtt does matter to users. it becomes very noticeable. especially when you're writing an app, not a brochure site. You need to tree shake so you're not serving a huge spa all at once.

Because this implies that.

No it doesn't. at all.

Page load speeds by themselves can be painful. Open devtools and have your browser throttle to poor 3g speeds.

Try browsing around. even well optimized sites.

Now try uploading a couple dozen files through an api.

This is legit what some users deal with. In New York state even, you don't need to go that far to find poor connectivity.

Even if all your users have awesome home connections, think sales people taveling to a client. or on site inspections of a manufacturer in a warehouse that's mostly metal and has bad wifi.

Then what does the 200ms have do with "writing an app"?

Apologies, your username looks like the one that tossed that number out as a what they assumed was a high number.

200ms latency isn't that bad, but I'm seeing more 800-2000ms latencies with some users depending on physical location. at some point latency kills usability. Especially when trying to get through a complicated QA or inventory process.

That latency is literally impossible unless they’re up in space somewhere, beyond satellite orbit. Your latency is most likely caused by CPU on low-end devices. A CDN won’t help you with that (and probably be harmful having to manage another TCP connection).

I see latency spike like this often on T-Mobile LTE. One second your under 40ms ping, then your next ping clocks in at 794ms or 2100ms despite having good signal strength.

I few years back I was seeing about 15-30s latency on 2G EDGE. Overloaded links can have high latency due to queuing which CDNs tend to help

Any latency over he speed of light is possible. In this case you’re seeing people in bad places, like rural China, or Long Island...

If you're on 3G I would expect sites to load in a similarly bad way with or without an extra most of 200ms of RTT.

The throttling in dev tools is meant to represent that latency...

If setting it to 3G is supposed to represent just 200ms latency, that's going to give you a very exaggerated and misleading impression of how bad it is. It's a meaningless test.

I thought you were giving an example of how bad connections can get, and saying that the extra latency would make it worse, but in that situation it's a drop in the bucket.

> Yes, there will be 200 ms RTT in some cases. So what? Get over it.

You're missing a zero in that RTT for users in places like Asia if your server is anywhere in the west. (It's actually somewhat revealing when someone throws out a number like this without any qualification; what exactly made you conclude 200ms is the magic number?)

> Optimize your website to load in fewer round-trips. TCP congestion control adapts well enough to any latency. RTT only really matters in gaming and VoIP.

You don't need a very big imagination to think about cases where RTT will have significant impacts e.g. in the event you need to issue multiple sequential requests that are dependent on one another. These are unavoidable and occur often in more than just websites, but anything that e.g. uses HTTP as an API (a very simple one is something like recursively downloading dependencies.)

This comes across as a classic "I don't actually understand the problem domain very well at all, but get off my lawn" answer to the problem.

> You're missing a zero in that RTT for users in places like Asia if your server is anywhere in the west.

Well, it does say 130 ms in here: https://www.quora.com/How-long-would-it-take-for-light-to-fl...

And that's around the planet, to go around and end up at the same spot. In practice, with sanely-configured routes, your packets should never need to traverse more than half that distance. So, divide it by 2, then that cancels out because RTT is a measure of how long it takes for a signal to travel back and forth. You then add some time on top of that to account for buffering and processing in the various equipment along the way.

> These are unavoidable and occur often in more than just websites, but anything that e.g. uses HTTP as an API (a very simple one is something like recursively downloading dependencies.)

If you mean REST API requests, the kind that trigger some code to dynamically generate a response, how would a CDN solution like cloudflare help? The request still needs to get to the server and the response still needs to come back, all the way, because that's where that code runs. CDNs only really work for cacheable static content, don't they? I mean it's in the name.

A blog or a news website certainly doesn't need a CDN.

Of course, light doesn't travel at vacuum speed in fiber, it travels a bit slower, but it also bounces around the cable, so ends up traveling a significantly longer distance. Multiply by around 1.5 for more real world numbers (copper is similar), just measuring raw distance.

And switching adds significant delay, especially when you get out to the edge.

> If you mean REST API requests, the kind that trigger some code to dynamically generate a response, how would a CDN solution like cloudflare help? The request still needs to get to the server and the response still needs to come back, all the way, because that's where that code runs. CDNs only really work for cacheable static content, don't they? I mean it's in the name.

For a simple example, imagine assets that are dynamically loaded based on feature detection in JS. All of the assets (js included) can be cached on the cdn.

> Well, it does say 130 ms in here.

If you have a fiber backed, all-switched network with no routing, buffers, congestions, or detours, you may get that value, if you're lucky.

Pinging tty.sdf.org which is a direct access shell service in USA from somewhere between Europe and Asia, from an academic network backbone roundtrips in ~190ms. I'm traversing a little less than half a globe with the whole journey. In your terms, it should be around ~60ms, but it's not.

> If you mean REST API requests, the kind that trigger some code to dynamically generate a response, how would a CDN solution like cloudflare help?

By using Cloudflare workers, so your code is also distributed around the globe?

> CDNs only really work for cacheable static content, don't they? I mean it's in the name.

JS files are also static content. Unless you don't use code distribution like Cloudflare workers, using a simple CDN can cache 90% of your site if not more. CSS, images, JS, HTML, you name it.

> A blog or a news website certainly doesn't need a CDN.

Actually, CDN is the most basic optimization for distributing heavy assets like videos and images, which news websites use way more than text. Why not use a CDN?

> Pinging tty.sdf.org which is a direct access shell service in USA from somewhere between Europe and Asia, from an academic network backbone roundtrips in ~190ms.

Actually I get around 200 from Russia which is also "somewhere between Europe and Asia":

   round-trip min/avg/max/stddev = 196.504/197.581/199.833/1.360 ms

> By using Cloudflare workers, so your code is also distributed around the globe?

Great, let's give that company even more control. That's sure gonna end well.

> CSS, images, JS, HTML, you name it.

It all gets loaded once and then cached in the browser. The initial load takes long regardless of whether there's a CDN. Oh, and many websites also use stuff from like 10 different domains, which doesn't help this either.

And, it doesn't matter whether a JS file loads in 50 ms or 300 ms, if it then takes 5 seconds to parse and start running.

> Actually, CDN is the most basic optimization for distributing heavy assets like videos and images, which news websites use way more than text. Why not use a CDN?

So put them on a separate domain and serve that from a CDN if you really care whether that stock photo no one notices loads in 500 ms instead of 2000. That doesn't explain much why anyone would put their main domain behind cloudflare.

> Great, let's give that company even more control. That's sure gonna end well.

We have enough evil companies who invade our lives through the platform develop. Cloudflare is not one of them. Using them is voluntary (by the service providers), and I think they're one of the more useful companies around.

BTW, I'm not a web developer or Cloudflare employee. I have no skin in this stuff, however they build some cool stuff inside the Linux kernel, which is interesting from my PoV.

> It all gets loaded once and then cached in the browser.

Then cleared and/or invalidated by the user or browser's logic itself due to plethora of reasons.

> The initial load takes long regardless of whether there's a CDN.

Actually, no. A reasonably fast internet connection (>12 Mbps we can say) can load a lot of things very very fast. The biggest overhead is DNS, even with CDNs. With a good local, network-wide DNSMasq installation, if the server is close, I can load big sites almost instantly.

> And, it doesn't matter whether a JS file loads in 50 ms or 300 ms, if it then takes 5 seconds to parse and start running.

I think 5 seconds is long time for even the old Netscape Navigator's JS parser. You need to run something akin skynet to parse the JS file for straight 5 seconds. How's that even possible?

> So put them on a separate domain and serve that from a CDN if you really care whether that stock photo no one notices loads in 500 ms instead of 2000.

I don't know you, but world news generally contains live/new footage or fresh photos from the ground, not some stock photos, also we humans are visual animals. Many people want to see the images first, read the text later.

> That doesn't explain much why anyone would put their main domain behind cloudflare.

Load balancing, DDoS protection, CDN, workers, Bot/Scraping protection, cost reduction, rate limiting, you name it. Even my DSL router implements some of the protections, to my surprise.

Internet is not the same beast now when compared to 90s/00s. I miss the simpler times, but alas.

There's no need to guess based on the speed of light. Test it yourself:

https://www.cloudping.info

For me, the highest was 310ms round trip to Singapore, so higher than your estimate but not too bad.

But this is completely beside the point. As far as I know, if you're using a CDN effectively (i.e. a large proportion of requests are hitting cache), it should be cheaper than having all requests hit your server, not more expensive. So even if you don't "need" a CDN, you might want one. This is orthogonal to the issue of bot traffic, which exists whether or not you use a CDN. If you want to use a CDN but don't mind the costs of bot traffic, you can configure CloudFlare to not show the CAPTCHAs, or use a different CDN.

338 ms to Sydney is my worst. 234 ms to Singapore.

AWS does offer a CDN, right? Somehow they do it without captchas and without me ever noticing. So I'm somewhat right at cursing at cloudflare because it's the only one actually announcing its presence by actively disrupting your browsing.

Cloudfront, the aws cdn, is not equivalent to the part that's showing a captcha. Only when it would have hit your server do you see the captcha, because it's proxying it, not serving cached results.

I live in New Zealand, definitely a first world country with first world infrastructure. Ping times to US East or Europe are over 300ms right now from my home[1].

My old business had users in countries around the world, and the assets were highly optimised for speed. However adding CloudFlare (a) significantly sped up our service to clients, especially those in Asian countries, and (b) significantly improved reliability of connections because CloudFlare have their own dedicated network links between countries and/or optimised for reliability.

[1] https://www.cloudping.info/

Most end users are not in a position to control the path their packets take. Their ISP could route their packets around the world 6 times over if they can save a nickel doing it

For regular public web pages, serving the actual fucking page should not be more expensive than serving the captcha page! What the hell is a "bad actor" in relation to GET requests to a public page? To a public page, all actors should be inherently neutral.

There's a lot of potential side effects. Not every GET request retrieves a static asset. A list view with filters for instance. Either way, you could be dumping any arbitrary data along with a request. Or just try fuzzing parameters. some pages might be poorly done graphQL endpoints, and your might find your db tied up. There are MANY ways a legit a get request can cause issues, let alone someone with bad intentions.

All users are hostile users and all users are preferred users. Define what your system will allow through rate limits and caching. Assume users would destory your site if given the chance because they will. If you are exposing private data through graphgl config it or drop the private data or drop graphgl and user a backend.

I 100% agree you should be looking at all incoming traffic as hostile or at least potentially hostile. The other comments are contending that there's nothing a hostile user could do to a 'public' page.

One of the things using cloudflare gets you is all those protections without having to know how to do them yourself. Which a lot of the developers don't know how to do.

There's also something to be said for catching a lot of this at the network layer on a cluster of machines that can handle any incoming traffic that your one poor neglected vm can't.

If you use cloudflare you give up freedom not to force captcha. If you avoid them you can choose where you want to captcha. When cloudflare goes down you go down unnecessarily.

If worried about a ddos attack cloudflare or another provider might be a good choice. But adding ddos support by default seems unncessary. In my 20 years of running 100s of sites I haven't run into a situation where I need ddos support. The vast majority will never be the target. Once in awhile google or bing will ddos you but using cloudflare to block that seems like overkill.

There are tradeoffs, yes.

We might weight those tradeoffs differently. That's ok.

Cloudflare gets you very paranoid, overly aggressive, user-hostile protections by default – and people just take the defaults even if they use it with STATIC HOSTING LIKE GITHUB PAGES!

Why is every and any TOR and sometimes VPN user deemed a DoS attack... it discriminates against users who value privacy by forcing hCaptcha on them by default. Worst of all... it could be a de-anonymization attack as well, hence why I as a regular TOR user, just exit the page immediately when that happens.

For any of my pages that do happen to use Cloudflare, I am luckily able to disable this discrimination in the CP so kudos for that at least, but terrible defaults imo.

From experience, traffic via Tor was always 99%+ fraud.

You can conduct fraud by accessing public, read-only web pages? You can conduct fraud by searching on Google?

Those are the two I find repeatedly blocked when accessing via Tor. The former by Cloudflare, the latter by Google.

I use Tor to lookup phone numbers that have just called me, to decide whether it's a good idea to answer. Since I don't want to be personally associated with such numbers I prefer to search anonymously. But often it's impossible to get a result.

Sometimes even spending 5 minutes solving captchas isn't enough. (I'd only spend that long to see if it's just an outlier. No, it's quite common.)

This creates an immense pressure to tell various services exactly who is phoning me, which is a terrible attitude to privacy.

Then don't use sites that are behind cloudflare?

It's not your choice if the site owners/admins use cloudflare. It IS your choice not to use those sites.

In practice the information I'm looking for is behind Cloudfare. There are other sites; they tend to lack the information.

There is no "don't use" if I want to get my task done.

I can choose not to obtain the information, but then I still have the problem I started with.

For the companies I worked for, we usually allowed Tor as read only. But net ops might override that, particularly when things moved to https traffic.

Well, if you keep throwing impossible captchas at them, no wonder that normal users just close the tab, but bots and fraudsters keep trying.

Because that's a not insignificant portion of traffic they see from tor and vpns?

tor has some absolutely valid and import use cases, but what percent of tor exit traffic is actually someone trying to keep their traffic anonymous from the eyes of an oppressive regime, and what percent are script kiddies, or someone hiding torrenting from their isp?

Are you one of those people that answers the door with a gun, even when you’re expecting a friend?

> How do you mitigate ddos attacks and other bad actors hitting a page?

Not sure what "bad actors hitting a page" even means. I host public info so people can see it, be it good or "bad" people. Let them see it.

DDoS is different and can be devastating of course. Also, very rare. In decades hosting content (started my first hosting business in 1994) I've never experienced anything remotely like a DDos. I know it happens, but definitely very rare for most people. Driving tons of legitimate users away with relentless captcha annoyances for the once in a liftime possibility of a DDoS is not a good tradeoff.

If you're in a business that attracts DDoS like flies then deal with that, otherwise lay off the captchas.

> Not sure what "bad actors hitting a page" even means

I would refrain from commenting on this topic then.

> I host public info so people can see it, be it good or "bad" people. Let them see it.

If your site ever gets big enough, you'll understand.

We're solving the wrong thing here. We should be going directly to the source: The mis-behaving (for whatever reason) IP and the ISP that owns it. We should be fostering an ecosystem that solves that problem, not one that results in a CloudFlare popping up. Which proverbially just throws spaghetti (money, technical-expertise and infrastructure) at the problem, and then using that accumulated power to shape (or manipulate) the web to its liking, and to kick-off anyone it doesn't like off it, of course.

Reply on news.ycombinator.com