Block any crawler that doesn't come from known ranges that you whitelist.