URL targets are secrets as they are expected to be non-public.
It would be a serious breach of trust for them to publish the database. It likely includes links to non-public YT video URLs, for example.
It would be a serious breach of trust for them to publish the database. It likely includes links to non-public YT video URLs, for example.
> there are about 230 billion* links that need visiting
> * Thanks to arkiver on the Archive Team IRC for correcting this number.
Also when running the Warrior project you could see it iterating through the range. I don't have any logs handy since the project is finished but they looked a bit like
https://goo.gl/gEdpoS: 404 Not Found
https://goo.gl/gEdpoT: 404 Not Found
https://goo.gl/gEdpoU: 302 Found -> https://...
https://goo.gl/gEdpoV: 404 Not Found