Hacker News Reader

One would assume legit spiders obey robots.txt.

This, to me, is the strongest argument to offer these slop generators. It provides an incentive to follow the robots.txt.

Exactly. You disobey robots file => we'll make your crawl gain a net negative.