Reddit's robots.txt disallows all web crawlersold.reddit.com8 points·rawfael··1 commentOpen articleSaveView on HN