Deepcrawl does that, but again it entirely depends on the website type and infastructure you have got.