AWS said it mitigated a 2.3 Tbps DDoS attack
zdnet.com
zdnet.com
Even without that, I wouldn’t be surprised if it’s covered by other forms of insurance, such as policies that would cover an internet or power outage. It would probably be harder to get the insurance to pay out, but if your legal team is big enough, maybe...
These days incident response retainers are very common, where you would pay a security firm like Mandient or Verizon or IBM a small fee every month and in exchange they guarantee that they will have an expert on site within X hours if you have a security breach. Almost every major company has a retainer like this.
The real benefit to this situation is you get to avoid haggling over price and waiting for a quote from the sales guy and running it past your legal department. All that enterprise sales nonsense is already done so when you pick up the phone it's all business with no negotiation.
So your account is all active, but you don't have the proxy-through-CF part.
As a result all the traffic goes directly to the origin (as a side-effect, your origin IP address is revealed to the attacker).
You receive the email "Sorry, CF offers limited DDoS Protection, call our Enterprise team for help".
From your blog post I understand you don't do it anymore, which is good. Really, it was quite bad to see the DDoS protection failing you when you need it the most :|
Could be called "A business plan upgrade moment"
To be fair, at the time I believe "DDOS protection" was a feature only explicitly listed on their $200/month plan, and I was on the $20/month plan. I think it's part of all plans for the last few years now.
At my job now, we use cloudflare workers, but I'm almost hesitant to rely on them too much because I know they'd all break immediately if CF ever decided to surprise-switch our account off of the proxy-though-CF mode like I had happen before. I don't know if that move is still part of CF's playbook or not given the changes in how they handle DDOS attacks; maybe I'm being a bit irrational since the DDOS attacks were a painful experience to deal with overall. I hope this time CF will just ask us for more money if there's a problem before pulling out the rug. Other than that bad experience around that DDOS attack, CF has been one of my favorite services.
Workers actually launched four days later. Funny, I didn't realize at the time that unmetered mitigation was really a prerequisite for Workers -- obviously you couldn't build very much in Workers if a random DDoS attack could cause it to just shut off. Huh...
(I'm the lead engineer on Workers.)
TL;DR: This doesn't happen anymore.
(Disclosure: I work for Cloudflare but I don't know anything more about this than is written in the blog post.)
Standard: free; only pay AWS egress charges. Advanced: $3K/yr + ~$0.05/GB data out
So blocking 3Tbps costs nothing additional to the user.
Is there an economically positive criminal activity that involves DDoSing an AWS-hosted UDP service (probably video calls... probably like Zoom)?
I believe they are more like an attempt to discover limits in the network or some targeted systems. Some systems are also vulnerable while they reboot, so attackers only need a one-time reboot.
From what I've been told, you're right. DDOS attacks can routinely expose information through failure modes the ops team never prepared for. What happens when your failsafes fail? If they didn't test for it and put mitigations in place then it's rather likely that sensitive error messages or service details, or whatever, is being exposed over the wire. So aws mitigated this attack. Does aws know for a certainty that they revealed nothing sensitive in the process? Maybe, maybe not. If the attacker is good, and 2.3tbps is pretty fing good, then could the victim even be positioned to know what to look for? In uncharted territories the attacked is always down from the attacker.
That used to be the case, but with the popularity and widespread use of IoT devices it won't hold true for long. If you can hack home appliances you could hold an attack for hours, if not days.
History has, the company used to be DDoS'ed regularly, sometimes offline for days, before moving to cloudflare.
https://interconnected.blog/why-zoom-chose-oracle/
See previous HN thread here: https://news.ycombinator.com/item?id=23032029
https://www.datacenterdynamics.com/en/news/most-zoom-runs-aw...
One time a customer came to us and asked us to PenTest their server, checking it stands up to a DDoS. They said they owned the server and it was their network, so we said "we can run a small one for you which should give us an idea of some pain points".
We run the "mini" DDoS against the server, it takes a little more to sink the server than expected, but we just ramp up a few more connections and it is fine. We lift off on the test attack, but customer site doesn't come back up. We contact them and they say they will contact the VPS host. * Heart sinking moment *
We test other websites running on their cloud from a different connection - we had taken out their entire cloud infrastructure (this was a small provider). After a short while they were back up, but not before another few conversations with the customer. I really don't even want to know how badly positioned we were legally that day.
Lesson learned: Always double check.
There was a Windows vulnerability that came out in 2010-2011ish, when I was working there, that I had to deal with. I ran an nmap scan of the entire network looking for the bug, and accidentally BSOD'd half the office...
For web applications, you can use ALB to route traffic based on its content and accept only well-formed web requests. This means that many common DDoS attacks, like SYN floods or UDP reflection attacks, will be blocked by ALB, protecting your application from the attack. When ALB detects these types of attacks, it automatically scales to absorb the additional traffic. This scaling activities are transparent for AWS Customers and do not affect your bill.
When I moved reddit from datacenter to AWS in 2009, I no longer had to deal with DDOS attacks. They just magically disappeared. I'm pretty sure reddit was still getting DDOS attacks after the move. :)
The article says this ^. 1tbps in 2012 might have been a record but it’s been nearly a decade
How much traffic is being generated by a single endpoint in one of these?
ISP infrastructure is symmetric fiber. A single fiber can do 100 Gbps without troubles. And when you have to bulldozer the road to lay the fiber, you don't put a single fiber but a pack of a hundred. There is no crosstalk between fibers unlike with copper, so one has interest to put as many as possible.
But this was a reflection attack, so most of the bandwidth was coming from poorly secured servers. In datacenters those would most likely have 1 Gbps uplink speeds.
Also, even cable is capable of Gigabit speeds. I briefly had 1.2Gbps via cable, downgraded to 400Mbps as it became cheaper.
Did you have GB upload speeds with cable too?
[0] https://en.wikipedia.org/wiki/Denial-of-service_attack#Ampli...
More detailed descriptions:
https://www.cloudflare.com/learning/ddos/memcached-ddos-atta...
https://www.cloudflare.com/learning/ddos/ntp-amplification-d...
https://www.cloudflare.com/learning/ddos/dns-amplification-d...
Of course, that's under the belief one's above the law.
People doing a better job with securing their internet exposed servers would also help.
Yes. Such services are developed by most, maybe all, the major CDN and CDN-like providers because they can sell DDOS protection/prevention to their customers. It's an active area of cat/mouse between the people selling the services and the people selling the DDOS attacks.
Plenty of folks keep this stuff secret.
[1] on public record
Ok, it's no longer the 'largest ever' in the title above.