Robots.txt meant for search engines don’t work well for web archivesblog.archive.org3 points·nathcd··0 commentsOpen articleSaveView on HN