Formalizing the Robots Exclusion Protocol Specificationwebmasters.googleblog.com4 points·manjana··1 commentOpen articleSaveView on HN