Stack under attack: what we learned about handling DDoS attacks

"These API routes, like any API that pulls data from a database, are necessary to the day-to-day functioning of Stack Overflow. To protect routes like these from DDoS, here’s what you can do:

- Insist that every API call be authenticated. This will help identify malicious users. If having only authenticated API calls is not possible, set stricter limits for anonymous / unauthenticated traffic.

- Minimize the amount of data a single API call can return. When we build our front page question list, we don’t retrieve all of the data for every question. We paginate, lazy load only the data in the viewport, and request only the data that will be visible (that is, we don’t request the text for every answer until loading the question page itself).

- Rate-limit all API calls. This goes hand-in-hand with minimizing data per call; to get large amounts of data, the attacker will need to call the API multiple times. Nobody needs to call your API a hundred times per second. Filter malicious traffic before it hits your application. HAProxy load balancers sit between all requests and our servers to balance the amount of traffic across our servers. But that doesn’t mean all traffic has to go to one of those servers.

- Implement thorough and easily queryable logs so malicious requests can be easily identified and blocked."

When using HAProxy you can also try to leverage TLS/JA3 fingerprints (https://haproxy.com/blog/announcing-haproxy-2-5/) While it can still be spoofed using several packages , e.g. https://github.com/Danny-Dasilva/CycleTLS it can still provide a meaningful/easy-to-manipulate signal.

Moreover, most bots conducting L7 DDoS don't use real/headless browsers in order to be able to scale their attack, so it's highly likely they'll have a discriminating/inconsistent TLS fingerprint.

This can also be done directly in Fastly using e.g. https://developer.fastly.com/reference/vcl/variables/client-...

Another approach to proactively flag malicious IPs is to scrape free proxies. Indeed, most DDoS leverage lot of cheap/know bad IPs. It's frequent to see these free proxies in these attacks.