Source: worked for AWS, was oncall during similar attacks. Nasty things with those they tend to start around 6-7PM (guess when does working day start in China).
Source: worked for AWS, was oncall during similar attacks. Nasty things with those they tend to start around 6-7PM (guess when does working day start in China).
The two colo centers we've hosted in have always helped us with DDOS issues free of charge. Maybe that's not normal, but even a former employee telling us to GTFO looks bad on Amazon to me.
My general point is: AWS is a business, and it operates as one. There are no hollywood style bad guys sitting there in cubicle dungeons on chests filled with gold thinking how to extract money, quite the contrary. It is understandable that customer cannot pay unlimited (from customers perspective) charges, but AWS pretty much incurs them, as customer being ddosed is consuming resources that would be otherwise be sold to others, or engineer time that would be put into developing new features and attracting new customers.
You can't disregard any business that doesn't fulfil that property as being "eventually unsustainable".
However my note was about general "traffic", if one sells video views for example, and revenues do not grow inline with adjusted to [almost always decreasing] bandwidth costs sooner or later that will become a big problem.
In this case - 700k QPS (gigabits of ingress) of well engineered HTTP/HTTPS DDOS traffic is not something an average colo can or even will be willing to handle at all. I'm assuming a hot-potato DDOS, when a customer comes along with a long tail of colos and providers that already booted him. All that traffic, servers and ultra expensive engineer time. Everyone wants it for free, but ALAS.
Assuming you can find yourself a transit provider that supports BGP flowspec updates (many don't, sadly), you can do this fairly cheaply. You'd obviously want some level of support from a network tech that knew what they were doing, but it's not insurmountable. There's a bunch of other options available too.
This sort of thing is one of the downwsides of having your infrastructure managed by someone else. If things go wrong and your provider doesn't feel incentivised enough to help you out, there's a lot less you can do about it, other than just pay whatever sum they demand.
What I remember pushing BGP flowspec updates upstream was thought about as something close to impossible though.
I suppose if you're not a Chinese hacker, it might pay to pretend you are by tailoring your working hours and days.
I know you can do this on the server, using many different techniques. But this does not help as the traffic still reaches you (that you have to pay for).
You can also do this with Geo DNS (and get much less of a bill).
And the ISPs, datacenters, and anyone with a router can block ASIA or China allocated IP ranges. Especially if it's not the type of a flood that's designed to attack the routers (instead of the web-server).
So what's stopping Amazon?
Having provided IP transit at a largish network provider in a previous life, you have no idea at the complexity involved what you're asking for. It could be done, but the costs involved are non-trivial.
If you're honestly interested in the complexity involved, start reading about BGP, dynamic routing protocols, router/switch fabrics, control plane integration, autonomous systems, peering agreements, etc.
While you may not like the trend, and some providers have better options than others, and you may require some operations over others... but very few businesses can afford to manage multi-site infrastructure that can dynamically scale. Most are probably served just fine with rented servers, traditional colocation, or don't even need more than one VPS...
That doesn't make the technology bad, and only makes you seem ignorant in your prior statement.
I'll go into it later, but for the most part I consider the cloud a really bad idea. I also regard most cloud companies as "buzzword-enabled" so as to attract investors.
My gripe with scrum and agile is not so much with the methodologies, but with companies that think they have a methodology when in reality they have a bureaucracy.
If ephemeral storage is a drive local to the virtual machine's host (which I think is the case) then having a cool down period would mean holding the hardware that used to be used by you in reserve until the grace period expired.
It's possible but it's a lot of work to solve a problem that can be better solved by not relying on ephemeral storage persisting.
Yes, it would mean holding the hardware in reserve for the cooldown period. I'm not talking months here, just enough time to recover from an accidental "sudo shutdown -h now" instead of "sudo shutdown -r now" (or similar). It'd be nice if Amazon sent an email warning about the condition and giving you an hour or so to go in and save your data/restart your instance. They could even make it a policy that you're charged for the time your instance is running + 1 hour to facilitate cooldown feature if they're really that worried about it; it's better than wiping data as soon as someone stops (from AWS console) or shuts down (from real console) an instance and providing absolutely no avenue for recovery, no matter how quickly you notice the mistake.
You have to specifically put something into the /mnt folder if you want it to be stored on the ephemeral storage. Any other location is safe and will persist through halts and stops.
In practice the only thing you should ever use the /mnt folder is maybe a Nginx disk cache, or as an alternative /tmp or something like that. Basically if stuff you don't want to lose is finding its way onto the ephemeral storage then you are doing something wrong.
There are plenty of plausible situations where an AWS user can find themselves with important, even just temporarily important, data on ephemeral. Whether those are the result of "correct" usage or not, it's beyond the pale to just zap that data away and tell the customer tough titties as soon as a shutdown command is issued.
You mean the company that provides caching services for pay DDOS services?
I recall only hearing about one time when packets from Chinese ISPs were completely dropped for some reason and only for short period.
I've also have an anecdotal reference that one can persuade providers that actually deliver traffic from China to do filtering on ingress on next hop routers after China, but that should be something very serious, that impacts their revenue as well and prolonged. As another commenter noted - costs for providers are very non-trivial.
In my experience DDOS is always money competition, it costs money to mount one, it costs money to defend against one. Unfortunately when one of the sides is [allegedly] a country it doesn't play very well.
I was working on an anti DDoS system for SIP, a UDP-based protocol. Basically the options were: 1. lockdown, just whitelist known good customers, and break many scenarios. 2. Attempt some kind of analysis, like sending out probes to determine good/bad IPs. 3. Scale the hell up. Write L7 stuff that can go at wire speed, and get lots of wires.
Needless to say, #1 is the easiest to implement, but allows you to get your pipe saturated. #2 requires compute + pipe, and #3 is the only thing that'll really work.
This matters because DDoS'ing a telecom can be very lucrative. I can say with good confidence that demonstrating DDoS capabilities are probably worth 5-6 digits in blackmail against many companies.
If they put a reCAPTCHA wall in front, the GFW can simply block reCAPTCHA (easy -- it is a Google property and they block everything else from Google anyway) and no one from China can access Greatfire without a VPN. Mission accomplished.
If the goal was only to block Greatfire for non-VPN users, then they could just use the GFW for that from the start. The use of a DDoS can only imply that China wants the site offline for everyone, even VPN users.
With this DDoS, they are taking the different route of attacking the infrastructure of Greatfire such that they can't serve traffic from China at all. Causing massive bills and outages for Greatfire is probably a bonus, but I don't think that is their main intention.
China is attacking them to prevent Chinese people from reading the website.
Your suggestion is to make the site unavailable to China.
Do you see why it is not a solution? You are basically setting up a market for censorship-- the attack doesnt ever have to end-- depending on how much China is willing to pay to keep the website offline.
Maybe I'm not understanding.
Attacking a single site is like more pro-active movement of the GFW itself, and a sign it can be so much worse.