HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by xnx | Hacker News Reader
Parent
Full thread
xnx
·
Google uses the same crawler and robots.txt file for training data.
View on HN
inkysigma
·
It's actually a different crawler for training data: Googlebot-extended so you can exclude yourself from the training data though not the search summaries.
Reply on news.ycombinator.com