I saw a saas recently that has something targetting this. They will automatically update your robot.txt to include different crawlers they find.

This will only stop crawlers that obey robots.txt though.