Hacker News Reader

Indeed, Google just has too much market share.

Last week I blocked Google from my domains (blog: lucb1e.com/!130), hopefully others will follow suit and degrade the search quality until people get better results (at least for some more obscure content) elsewhere, or perhaps until Google notices we are really not okay with their behaviour.

View on HN

Are you blocking GoogleBot by IP range or User-Agent match? Why aren't you using your robots.txt file to block GoogleBot instead or in addition to your server-side logic?

Robots.txt was my first thought as well, but that is said to not actually block your site from appearing in the results. They'll gather from other sites what the page is about (think <a href=mysite/somepage>how to knit a sweater</a>) and show that as title without page summary. Maybe if it looks like the site is down, they won't bother.

Blocking is based on user agent, they seem to set that reliably and the IP addresses change. You can do some reverse lookup magic but this was way easier than looking up every single IP that visits my site.

Reply on news.ycombinator.com