Hacker News Reader

Comment by bArray | Hacker News Reader

I keep telling people 'you probably don't need a massive server if you're just running static pages', again this seems confirmed (for some use cases).

A few questions:

1. Do you have any protection against DDoS? I currently implement my own custom solution, people seem to be transitioning towards Cloudflare but I see such a move as dangerous. What's your mitigation and/or opinion?

2. You mention the ease of spinning up a new server - I personally just run a bash script for this. How do you automate the transference of your URL over to a new IP?

3. Have you ever experienced slow down and if so, how did this affect your site?

4. What are the most resource intensive aspects of your website? (I.e. What is using your bandwidth/CPU/RAM?)

View on HN

Regarding DDoS protection, it's simply not possible to implement a reasonable defense against anything but the most naive attacks without spending a ludicrous amount of money. I for instance was working on a site that got targeted, and at one point tried to load-balance the traffic across _30_ machines efficient blocking rules, I was saturating their connections.

After giving up, tried to use cloudflare. A few clicks later, they just ate the entire attack for me. All on their free tier.

That said, I strongly suspect cloudflare probably works hand-in-hand with the NSA or some shit, and it's a dangerous trend. But I just don't think there exists a viable alternative, unless you're a multi-million dollar internet company

> at one point tried to load-balance the traffic across _30_

> machines efficient blocking rules, I was saturating their

> connections.

I have a slightly more complex "load-balancing" method, as long as the network bandwidth itself can hold out (hosted in the cloud) I've been able to deal with all attacks so far. My current method is to have a "whitelist" and "blacklist" look up table to quickly begin filtering incoming connections. If you end up on the blacklist (reasons include too many connections over a time period, bad requests, too many requests without authentication, etc, etc) then you instantly get killed - not a single byte gets sent. Being on the whitelist (reasons include being an already auth'd user, low traffic density, unique requests, etc, etc) then you shortcut some additional checks. Connections not in either list (potentially either attackers or new users) go through an additional per-generated (the check and answer is statically generated, so it's not added to connection overhead) check to "test for humanness". Both lists are decayed over time and connections on both lists can be re-assigned if they start behaving badly. The additional checks are mostly only activated under high load.

After that there is a resource management layer that is essentially "service temporarily unavailable" on anything too heavy (large file GET/POST, large database read/write, non-essential database writes are dropped, etc, etc).

This is all then run through a high-level network simulation to test where the bottlenecks will be. Each "release" goes through the test before making it into production.

I can't give too many details, but there is also a slightly malicious protection layer - if we detect somebody using a known/obvious bot (which is harmful) we have a few methods to crash them (based on known vulnerabilities). Some of these also act against aggressive search bots.

I have a beautiful graph sitting somewhere showing an attack ramping up and then mostly disappearing when the defense was triggered. It was quite worrying at the time because we just assumed our system went down or we had accidentally started blocking real users.

> That said, I strongly suspect cloudflare probably works

> hand-in-hand with the NSA or some shit, and it's a dangerous

> trend.

I also highly suspect this.

> But I just don't think there exists a viable alternative, unless

> you're a multi-million dollar internet company

I agree and that is a massive problem.

Your defense mechanism is actually the exact thing described as protection against a simple DDOS attack.

With a REAL ddos attack you will get so many incoming traffic that your bandwidth is saturated before any software will come in effect. A real ddos is a hardware problem, and it cant therefore be solved by software.

> Your defense mechanism is actually the exact thing

> described as protection against a simple DDOS attack. With

> a REAL ddos attack [..]

I'm not entirely sure what is really classified as a "simple" vs "real" DDoS attack.

> With a REAL ddos attack you will get so many incoming

> traffic that your bandwidth is saturated before any

> software will come in effect.

I specifically said: "as long as the network bandwidth itself can hold out". If the network is saturated, it's saturated - it's game over, for anybody. But you'll find that most services will die way before this limit is reached. On a WordPress site for example you'll usually find the database will give out long before the bandwidth is saturated.

Even with legitimate users coming through some great filter like CloudFlare, it doesn't prevent a hug-of-death if you don't have some "smart" application level handling of high numbers of users. Like Linux OOM, when you hit your limits, the only choice left is to try and quickly and intelligently start killing without affecting the well behaved.

Not to mention his solution really does nothing against ping, UDP, and TCP traffic. It's really only effective against http traffic. Which means his upstream or provider is probably going to cut him loose for the duration of the attack. A single person cannot effectively block a concerted DDOS.

> Not to mention his solution really does nothing against

> ping, UDP, and TCP traffic.

Please bare in mind that the description is heavily simplified. If you're talking about bandwidth, of course this is the limit.

> It's really only effective against http traffic.

How so?

> Which means his upstream or provider is probably going to

> cut him loose for the duration of the attack.

There's nothing we can do about that, but being in the cloud offers at least some protection. DDoS'ing some random Pi at home is a little different from DDoS'ing an AWS server.

> A single person cannot effectively block a concerted DDOS.

Of course not, but that doesn't mean you have to make it easy.

The issue is that TLS handshakes are actually quite expensive. Assume a limit at about 3MHz of a modern non-hyperhtreaded core for each rps you want to do tls handshakes for. This is <40kbit/s in each direction. That's a measly 2.5 Gbit/s symmetric on one of AMD's 64 core flagship processors. They can normally push out ~1Tbit/s http unencrypted if the SSDs keep up, or about 20~50% of that if they have to encrypt it.

These numbers are all rough, but they are all for the absolute core/unflexible C-code that barely complies with the RFCs.

> The issue is that TLS handshakes are actually quite

> expensive.

Agreed and a very good point. IPs that end up on the "blacklist" are rejected before any handshake occurs. The description I gave was really quite simplified.

Why is moving to Cloudflare dangerous?

https://blog.cloudflare.com/why-we-terminated-daily-stormer/

Moving to cloudflare gives cloudflare veto power over any attempt to access your site.

But you are completely free to leave if they don't like you.

It's pretty simple, don't say Cloudflare are nazis are you will be allowed on their platform. They have been very vocal about being neutral in almost every case.

There's a long way between "neutral" and "neutral in almost every case".

What if I say something less hateful but still negative about Cloudflare? What if I run a website like 8chan?

Why though? If you don't like a service, move elsewhere. They're not obligated to serve your site. It's the same as any brick and mortar business. If my clients attack me, I cut them loose.

Not being able to speak badly about a powerful entity is a sure bad sign for things to come.

I don't think speakly badly of is the same thing as "framing" though

If we agree on what you mean by "framing" [1]:

> To make up evidence or contrive events so as to

> incriminate (a person) falsely.

I disagree that calling CloudFlare Nazis is framing them, as the purpose (assumedly) is not to incriminate. Criticizing them by likening their behaviour to that of Nazis or suggesting that the company is filled with fascists may be in reality false, but a valid criticism regardless.

[1] https://www.thefreedictionary.com/framed

There's an overblown meme floating around that cloudflare is a man-in-the-middle attack on your SSL connection. But even if you don't use cloudflare, your SSL connection typically terminates at your cloud provider's load balancers anyway. Would people then say the load balancer is a MITM attack? At some point you just have to trust the cloud infrastructure you paid for.

That's not the issue. The issue is that as more of the web uses cloudflare, it becomes a gatekeeper of the web. Any person that cloudflare kicks out will be cut off from much of the web. (It's already happening, with people having to solve captcha like it's their day job)

Even if cloudflare doesn't misuse their power, something worse happens. Cloudflare becomes an easy OFF Switch for laws, warrants and copyright trolls to track people or kick both content and people off the internet.

I'll go one step further and simply call it a voluntary (by server owners) centralization of the web. Blocking the Daily Stormer (rightly or wrongly) shows that they are a political entity.

Or that they're a business that doesn't want that kind of publicity. Just because you have a loudspeaker and a podium at your business doesn't mean you need to let everyone walking in to your office use it. Just as Chick-fil-A chooses not to be open on Sundays, Walmart chooses to sell bullets not guns, and Dunkin chose to make their company name really stupid sounding they're allowed to make business decisions good, bad, it otherwise because they think it will be the best thing for their bottom line.

CF have booted off a neonazi site, and general filtering will very likely occur at the DNS level. At the very least, some filtering of internet content probably isn’t the worst thing in the world.

I’m not disregarding the possibility or risk, but there are far greater problems that deserve our communities attention more than CF.

So, rethorical question, how many people don't know that and have login credentials sent over a plaintext link (inside your provider but even then, between CF and your site)?

Reply on news.ycombinator.com