Hacker News Reader

Comment by Bender | Hacker News Reader

Do you have a list of the addresses that were hammering your site? Have you tried any of the techniques I list here? [1] Do sets of the IP's show up in here [2]? Are the bots mostly residential, VPS, Tor? What is the HTTP protocol breakdown? HTTP/1.1, 2.0, 3.0? Are they missing any expected client headers? Have you tried blackhole routing any of them from an out of band management console?

    # only useful if not behind a CDN
    for Ip in $(cat /dev/shm/list-of-attackers.ipset);do ip route add blackhole "${Ip}" 2>/dev/null;done

[Edit] appears you are behind Cloudflare so the blackhole would be up to them. One could still return a 429 or 525 to the attackers.

[1] - https://nochan.net/b/Internet-Crap/20260606-How-To-Block-Som...

[2] - https://github.com/firehol/blocklist-ipsets/

View on HN

> Do you have a list of the addresses that were hammering your site?

I could get a list of IPs, but it wouldn't be particularly helpful. It was hundreds of millions of unique IPs. Most IPs made fewer than 100 requests and then disappeared.

> Have you tried any of the techniques I list here?

We were already doing essentially all of the techniques there and they were not helpful in this attack. This wasn't some spider run amok. This was a dedicated attack with intelligence behind it.

> Do sets of the IP's show up in here [2]?

If we're looking at individual IPs, we've already lost.

> Are the bots mostly residential, VPS, Tor? What is the HTTP protocol breakdown? HTTP/1.1, 2.0, 3.0? Are they missing any expected client headers?

The article outlines most of this. It was a mix of residential and major/minor hosting providers and some corporates. If I were to say which was the largest, it was minor hosting providers. However, the attack pivoted between coming from different sources. At first it was coming mostly from minor hosting providers/VPS. By the end, it was entirely residential and corporate IPs. Tor was not involved at all (RTD supports accessing through Tor, but you are more likely to get a challenge). All the expected headers were there and UAs cycled between a very large set of standard UAs for normal browsers/devices. The attack was 95%+ HTTP2/HTTP3.

> [Edit] appears you are behind Cloudflare so the blackhole would be up to them. One could still return a 429 or 525 to the attackers.

It's trivial to setup an IP list[1] in CF and then you can apply all sorts of rules (lower rate limits, outright challenges, etc.) to it. It can be managed through Terraform as well. You can also use CIDR notation. This along with more classifications to specific ASNs are something we are looking at.

[1] https://developers.cloudflare.com/waf/tools/lists/custom-lis...

Inside this .bz2 [1] there is a directory called bh_routes/bh_vps_120/ and in that there are many AS files named for many of the VPS/Server providers. If you site does not need inbound connections from such places that is a starting point to reduce the traffic a bit. Each file has a comment with the AS#/whois name if you want to double check who/what you may be blocking. Not perfect but it may make a dent in the traffic. Or ask Cloudflare if they have an option to block anything that is not residential or LTE wireless. Even if you want to allow bots, maybe have a "shields up" mode where bots are sent away until the attack passes.

Another potential option would be to configure nginx IP limits using the Cloudflare header that represents the IP, set the shared memory size rather high and return 525 to the IP's exceeding a limit to avoid them hitting your redirect rules though it feels like CF should be able to create something custom for this assuming its a paid account.

Any IP that is not VPS/server should be archived for the feds as they are working on shutting down residential proxy providers including apps that are turning peoples cell phones into proxies.

[1] - https://nochan.net/b/Internet-Crap/20260606-How-To-Block-Som...

Reply on news.ycombinator.com