i) Javascript: Very few crawlers can read your javascript code. Look at the source html of Google. Is pure Javascript! ii) Accept crawling based on IPs. iii) Use captchas. iv) Use cookies and Measure the crawling speed. v) Think that if someone wants to copy your content, they can just look at the cache of some search engine, they don't need to crawl you.