That's probably not the primary reason to use 80legs - but avoiding to implement a whole crawler.