Not all attacks are equal: understanding and preventing DoS in web applications
r2c.dev
r2c.dev
Still to do: developing our own online ML based monitor which can detect abnormal user behaviour before it becomes a problem (ie. aggressive HTML scrapers, burp suite scans, etc.)
Can't you drop all incoming traffic on "protected" IPs other than that coming through known-trusted networks like CDN's one? Your networking equipment still needs to be able to handle that load as well, which is not a given.
I guess DDoS attacker could fake the IP packet headers to appear as if they are coming from Cloudflare (in your case), though even then there'd be some obscurity in figuring what subnet you are allowing through.
I guess CDN providers could provide a secret-key-seeded time-based (like TOTP) source network, and your firewall would let packets in by that.
The benefit of this is that it is cheaper than doing full packet encryption, and probably much easier to implement in hardware/software.
Search first. I would try something like,
detect irregular OR abnormal behavior internet OR requests OR databases
I like the approach. It'll still return a 200, but just slow them down. In a botnet, that's a pretty interesting approach instead of returning a captcha so they'll keep using the IP.
Something to consider...
Summary:
1. Don't publicly expose endpoints that are either slow or require a ton of resources.
2. If you can't get rid of a slow endpoint; put authentication in front of it so you have a lever to pull in the event of an attack.
3. Throttle / rate limit everything with high barriers so not to impede normal traffic patterns.
4. Don't make it easy for someone to DoS you; reduce or eliminate well-know attack vectors and vulnerabilities.
5. Scan your app for regex and zip bombs.
6. The bad guys will sniff-out your N+1 queries, so fix them.
7. If necessary: pay for DDoS mitigation from a cloud provider.
For Example, the attacker will send a GET request using many small packets with a large time gap between them. They do this by first sending the “G” in its own packet, after a while they will send the “E” and after some more time the “T” and so on. While this is a legitimate behavior according to the HTTP and TCP protocols, this behavior is consuming a lot of resources from the server – it has to keep the connection open, waiting for the full request to arrive. Since the connection pool is limited, it is very easy to reach pool saturation, with very little traffic.
"There are ways to mitigate or reduce the impact of such an attack. In general, these involve increasing the maximum number of clients the server will allow, limiting the number of connections a single IP address is allowed to make, imposing restrictions on the minimum transfer speed a connection is allowed to have, and restricting the length of time a client is allowed to stay connected."
Even with async IO buffering requests it's still possible for a slow-client DoS to exhaust other system resources (such as file descriptors), but with properly-tuned limits the required attack will need to be many orders of magnitude greater to succeed.