Hacker News Reader

It's the web developer that doesn't want you to interact with their site programmatically.

Google and Facebook tend to do it as a matter of policy, and while they say it's to protect privacy and prevent abuse, it also furthers the "walled garden" effect.

> It's the web developer that doesn't want you to interact with their site programmatically.

Yeah because too many people abuse any hint of such functionality to peddle fake Viagra, pennystock scams, MLMs, ICOs or the good old SEO link spam.

> Big rant, there are few things I hate more than filling out their endless useless CAPTCHA's when browsing websites that have nothing to do with Google

In some cases, the blame should be put on the site runners. I get a ReCAPTCHA when logging into my Patreon account. I've been paying then $10+/month for years now, they should know by now I'm not a spammer

They're not hypocritical but cynical. There is an appreciable difference.

I thought you could disallow Googlebot in your robots.txt file?

Google respects robots.txt