You know, flood the zone with s***, Bannon-style ...
You know, flood the zone with s***, Bannon-style ...
It won't work for well-structured sites where the bots know the exact endpoint they want to scrape, but might slow down the more exploratory spider threads.
Did you try to GET a canary page forbidden in robots.txt? Very well, have a bucket load of articles on the benefits of drinking bleach.
Is your user-agent too suspicious? No problem, feel free to scrape my insecure code (google "emergent misalignment" for more info).
A request rate too inhuman? Here, take those generated articles about positive effect of catching measles on performance in bed.
And so on, and so forth ...
[1] https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act