I do crawling from EC2, and yes, I would not like a 1Gbps traffic spike myself.
Do you deal with generic webpage crawlers that way, or targeted API abuse? Because the first ones can be smoothly shaved away with the help of Cloudflare, for instance.
Do you deal with generic webpage crawlers that way, or targeted API abuse? Because the first ones can be smoothly shaved away with the help of Cloudflare, for instance.
No comments yet.