My own experience briefly running sites - bot traffic is 100% trash on almost every metric. Actual conversions, spam, click fraud, other fraud (spam talking about work from home), privacy violations (trying to scrap all user profiles, capturing deleted user content etc).
The real reasons people implement these draconian measures range from inept cargo culting to nefarious business models. If you have a problem with spam, add user moderation and call it a day. That’s just a justification though, there is never a pro-user reason to destroy ux and privacy.
Huh?
I require users to register (little to no privacy) and pay a bit - this results in a both better user experience and is a pro-user reason to destroy privacy. You can also blacklist most of the anon email providers / do SMS verification etc
I’m also curious what your site is. Will you say, or do you wish to remain anonymous?
site owners want the advertising that was embedded to be seen and thus make revenue from it.
Slack limits the amount of searchable messages to 10k, so if you save them regularly to local DB, you can search without limits.
All kinds of owners don't sell/show ads, or care about scraping.
There is no need to destroy privacy, waste resources, create terrible ux, gate content…
Things that work for small sites fall apart once you get 10~100 million+ people.
Look, I help do the actual user moderation for a large (10m+) subreddit, and it is a god damn nightmare. Plus, on top of that, we have to do automated bot detection anyways, because otherwise it's just neverending bot spam.
Reddit is actually a good example of having millions of users with no captcha. They “fixed” their problem with user moderation.
You chose to become a moderator, so it must not be that much of a nightmare.
EDIT: Reddit does have a captcha now. My guess is it came along with the other users hostile changes in the new design.
edit: They have for at least the last 6 years based on online questions. Long before the new design.
Automation detection and privacy are orthogonal. The ideal bot detector yields a single bit of information: bot or not. It doesn't need to try to de-anonymize users or anything like that. As for UX, well, spam is a bad UX.
The incentive is money. You can't just wave a magic wand and get rid of the incentive to spam things. Asserting that you can is wishful thinking.