Or searchyc could simply ignore robots.txt