If that's true, how does 3taps know that CL is blocking search engines? Also:
$ wget -q -O- --save-headers http://www.craigslist.org/robots.txt | fgrep Last-Modified
Last-Modified: Fri, 04 Nov 2011 18:13:24 GMT
$ wget -q -O- --save-headers http://www.craigslist.org/robots.txt | fgrep Last-Modified
Last-Modified: Fri, 04 Nov 2011 18:13:24 GMT
No comments yet.