What's so wrong about crawling or using automated tools? With today's networks and hardware performance most websites shouldn't concern themselves with denial-of-service type of attacks, unless they're past a certain threshold of popularity.
Every time I found a way to fingerprint and block them, they'd change their bot to avoid detection. Captcha for rate limiting seemed like the least bad option.
However, eventually I decided that instead of blocking them, I'd return random results from the db. They weren't checking the data too thoroughly because to this day they haven't changed the bot to avoid my most recent pattern detection. They're still merrily scraping useless data.
I would have set it to only show the captcha if the delay was active, so users would effectively have had a choice of wait or captcha.
There's very few friendly bots like archive.org, and those respect meta robots tags anyway.
I agree with you, for "read only" content, let the robots crawl.
Why waste time catering to people who are too lazy to write a decent scraper? Most people would prefer to spend their time working on things that will benefit the other 99% of their users.
(in reality though reddit has a huge bot / astroturfing problem anyway, so perhaps the value of captchas really is overstated)