Hacker News Reader

Comment by tiagod | Hacker News Reader

Google publishes IP ranges for GoogleBot. You can also reverse-lookup the request IP address - the resolved domain should in turn resolve to the original address.

View on HN

Does anyone else remember 10 years ago when Google would penalize sites for serving different content to GoogleBot than to normal users? Those were the days.

> Google would penalize sites for serving different content to GoogleBot than to normal users

Listed under spam policies:

https://developers.google.com/search/docs/essentials/spam-po...

   "Cloaking refers to the practice of presenting different content to users and search engines with the intent to manipulate search rankings and mislead users"

The top of the pages says sites that violate the policies may "rank lower or not appear in results at all".

It's infuriating when you see part of your desired information in the search results and then open the page to find a paywall. IIRC ExpertsExchange were doing that for a long enough time that it was obvious the policy was not enforced. At least not evenly.

Reply on news.ycombinator.com