I personally don’t even bother with it for these reasons.

And why should anyone believe you when you didn’t give any evidence of anything?

Robots.txt? Just send a fake user agent.