Comment by postfuturist | Hacker News Reader

Comment by postfuturist | Hacker News Reader

Parent Full thread

Any given site might not experience a DDoS attack, but if you run a hosting company, it will happen. The frequency depends on how many customers you have, how popular they are, etc. If a DDoS attack catches you by surprise, then you are very ill-prepared. One possible strategy is to throw the targeted customer under the bus and call it a day. For hosts of a certain size, that may be reasonable, as long as you clearly communicate that to customers.

I've seen these attacks mitigated single-handedly by an experienced fellow with no access to fancy equipment. I believe it went something like this, change the DNS to point at some EC2 instances to do front-end load balancing with some scripts that detect and drop connections from attacking IP addresses and do severe rate limiting. You can proxy legitimate requests back to the original servers.

Granted, not everyone knows how to do that in a pinch, but it shouldn't be a big deal for a hosting company. You only have to figure out how to do it once and when you detect a DDoS attack, flip the switch for that customer.

DDoS attacks are too expensive to protect against. Here's a price sheet from Gigenet, starting from a low low price of $750.00/mo with $1000 setup that only protects against 50 megabytes/sec.

https://www.gigenet.com/order/index.php?order=true&form=...

95% of dedicated hosts out there will immediately nullroute you if you get a DDoS attack like Softlayer, Ubiquity, etc... Unfortunate as it is, this is a common practice for hosting companies.

At the risk of getting downvoted again, I am going to re-paste a comment I made.

DDoS attacks are way too easy to do now, and something needs to be done about it.

Anyone can go to www.hackforums.net for a free UDP flooder (called shell booters there), or rent one capable of over 20 gb/s for $5.

Or if you want to do it yourself, pick a cheap throwaway vps from www.lowendbox.com, go to www.gametracker.com and grab IPs from COD4, Wolfeinstein ET, Medal of Honor, etc... and send UDP query packets with your IP spoofed as the victim. This will amplify the size of your attack 20x or more and hide your IP address. You can easily get 10-20 gb/sec attacks like this.

> I've seen these attacks mitigated single-handedly by an experienced fellow with no access to fancy equipment. I believe it went something like this, change the DNS to point at some EC2 instances to do front-end load balancing with some scripts that detect and drop connections from attacking IP addresses and do severe rate limiting. You can proxy legitimate requests back to the original servers.

That isn't how it works, but I appreciate you trying to explain it to me (I have personal experience as a former employee of a hosting company, in this exact position). A typical DoS attack is designed to flood the pipe, usually with UDP. UDP spam attacks are far simpler than layer 7 attacks that you describe, and layer 7 attacks (if small enough) are easier to handle.

As a point of reference, I don't even have access to a botnet these days but I could knock your home cable connection offline in about five minutes of my time.

In your scenario, if you were to update DNS to point the victim at some EC2 instances, you'd accomplish DoS attacking EC2 instead as they only offer you gigabit connectivity on most EC2 instances. If the DoS attack is multiple gigabit, that isn't going to help you unless Amazon steps in and mitigates about as best they can. A DoS mitigation strategy generally has nothing to do with your application. If an engineer's first answer to a huge DoS attack -- as in, larger than the uplink -- is iptables or "scripts to detect and drop connections", that engineer is uninformed. I'm sorry.

I have personally witnessed a 40 gigabit/sec attack. I'm sure the smart engineers at CloudFlare have seen bigger. Upstreams don't give a shit what's flowing across the wire, and a small TCP SYN flood directed at your server won't gain any attention from your host or their upstream. A multiple-gigabit attack that takes down the entire uplink will.

Would have appreciated if you did explain what an effective mitigation strategy is.

Depending on your position, there really isn't one. The hosting company can implement one, which they might be able to insert in your path if you are at the receiving end of an attack. There are Cisco products and a bunch of up-and-comers that can do this.

If you're a "typical startup" with an Amazon footprint, you have no mitigation strategy for flooding attacks aside from not attracting them. If someone points multiple gigabit at you, there is just about nothing you can do except hope Amazon can do something.

That's a little pessimistic.

There's a range[2] of ISPs who will sell DDoS protection to you, either as an addon when you host with them, or as an external service (re-routing your traffic).

E.g. StormOnDemand just recently added it to their portfolio[1], which is note-worthy because they actually list prices right on the website.

Either way, even without "explicit protection" any ISP beyond mom&pop-size deals with these attacks every day and will sort them out for you for free the first couple times. Only when they turn into a habit or become so huge that they have to talk to their upstream they will politely ask you to throw some money their way.

Pulling the plug immediately is definitely not normal. However considering Pastie was apparently a sponsored account it's at least somewhat understandable (albeit a terrible PR move).

[1] https://www.stormondemand.com/ddos.html

[2] When in doubt, and pockets deep enough, there's always Akamai. They're the ones who can filter TBit/s-scale (yes, that was a T) attacks for you.

> Pulling the plug immediately is definitely not normal.

Nulling immediately is. You're also assuming that this is Pastie's first DoS attack, which we don't know based on the information presented to us.

Nulling immediately is.

For serious accounts (in the 6 digits/year) absolutely not, unless the attack is large enough to affect other customers.

Admittedly RailsMachine looks very small, in all likelihood their pipe was rather easily clogged and they simply didn't have the choices that larger ISPs have.

> For serious accounts (in the 6 digits/year) absolutely not, unless the attack is large enough to affect other customers.

If it doesn't affect other customers, a hosting company won't act or even be aware, in most cases. They'll just send you a bill for the transfer. If someone attacks you and it impacts other customers, you get nulled. I'm aware of 7 digits/year and 8 digits/year accounts through industry anecdotes that have had machines nulled. The engineer operating the null doesn't say, "oh, that's X, maybe I shouldn't fix the network for my other customers".

I don't understand what you're disagreeing with.

I'm disagreeing with your black/white record.

There's a bit of middle ground between "sending a bill" and nulling.

I've been hit by two larger attacks in the past (GBit/s range) and the respective ISPs were both extremely supportive, switching our IPs while they tightened their filters. Neither billed us a dime despite our ingress spike making quite a bump in their charts and a lot of handholding over 2-3 days.

I understood the original comment about EC2 as a mitigation strategy so the hosting company's infrastructure doesn't receive the brunt of the DDoS gigabits (EC2 will) and it can still service all their other clients. In that case, even if the DDoS'd site will still be flooded and unavailable, its hosting plan doesn't need to be canceled outright. I suppose they could just skip EC2 and null-terminate the DNS until the DDoS stops, but I don't know if this has further implications.

In any case, thank you for a very informative post.

All the hosting companies I've been hosted with will react to an attack like this with null routing the victim ip address. Completely agree.

Reply on news.ycombinator.com