Google Isn't Just Reading Your Links, It's Now Running Your Code

On some sites what I do is make a fake virtual directory and just ban the directory in robots.txt while prefixing the links with the fake directory path.

Oh and I swear google sometimes follows stuff in robots.txt, it just doesn't index it. Then there's the "alternate" user agents they use like pretending to be IE (from a real Google IP).

I'll also note that I've never obfuscated my email address (chris@improbable.org) and spam arrived in roughly equal volumes there and to less public addresses which were guessable (except for test accounts created with MD5 hashes[1]). People do harvest addresses from web pages but there are enough other ways spammers get addresses that you need a robust filter in any case; once you have something like SpamAssassin or Google's filters, the value of keeping your address private disappears.

[1] That also yielded the interesting datapoint that hotmail used to sell addresses to spammers in early last decade as the first spam arrived within 20 minutes to a long, random address created with the "don't publish my info" option.

My approach is a little more convoluted than simply document.write or something like that, so maybe that matters. I get crawled by bad bots all the time, but these addresses never seem to get spammed. The same is not true for addresses displayed in plain-text somewhere.

My explanation has always been that there are so many plain-text addresses on the web that it isn't necessary to add JavaScript parsing logic to most harvesters. Much like car alarms or door locks--you don't need to make them unbreakable, simple harder to get at than the one next door. Someday we may need to start displaying email addresses as (or behind) captchas.