It keeps certain web scrapers and inline transparent proxies/IDS/XNS from fetching, 100%

Certain browsers will suffer. Meh.