The inequities of crawling are in my opinion a huge issue with the web that receives relatively little discussion.
If a majority of websites were to ban Googlebot, Google would be brought to its knees. Slowly.
Their index would go stale.
I like Google. But I do think crawling, who gets to do it and who does not is an underappreciated issue. Google is very lucky.
But anyway, this isnt really relevant. Can you tie it in for us?
Impossible to block Google? Probably true. But only because they have been allowed to grow so large as to be indefeasible. And the reason they've been able to do that is because of what the commenter said: websites allowed them to crawl, fast and hard, year after year. This is not true for all bots.
I'm not sure how this is relevant to the article and the specific issue. I don't disgree with what Google is doing in this case. And I understand why and where Google is headed.
The issue of who is allowed to crawl and who is not is something the commenter raised. It's a huge issue that people take for granted, in my opinion.