Twitter.com/robots.txt # Crawl-delay: 10 -- Googlebot ignores crawl-delay ftl | Hacker News Reader