> I'm pretty sure that the webmasters responsible for this are using user agent string blocking as a naive attempt to block bots from scraping their site, but that assumes that the bots that they want to block actually send an accurate user agent string the first place.

That is exactly what they are doing, and it works really well.

We blocked user agents with lib in them at reddit for a long time.

Any legit person building a legit bot would know to fake the agent string.

The script kiddies would just go away. It drastically reduced bot traffic when we did that. Obviously some of the malicious bot writers know to fake their agent string too, and we had other mitigations for that.

But sometimes the simplest solutions solve the majority of issues.