Comment by jakewolf | Hacker News Reader

Top New Best Ask Show Jobs

Comment by jakewolf | Hacker News Reader

Parent Full thread

I couldn't resist checking: http://search.yahoo.com/search?p=%22How+long+will+it+take+fo...

http://search.live.com/results.aspx?q=%22How+long+will+it+ta...

http://www.alexa.com/search?q=%22How%20long%20will%20it%20ta...

All duds for now.

PG talked about blocking all non-Google crawlers in an attempt to cut down on traffic ... not sure if he ever did or not

There doesn't seem to be anything about it in HN's robot.txt.

Of course, it may have been implemented in other ways.

What are those "x" URLs blocked in the robots.txt for HN?

Reply on news.ycombinator.com