Full threadgroby_b·Somebody remind me again why either robots.txt or serving a 402/Payment required to crawler UAs doesn't work?View on HN