> so it is better to find a way to cache your website's HTML so you're not bombarded
Of course, scrapers should identify themselves and then respect robots.txt.
Of course, scrapers should identify themselves and then respect robots.txt.
No comments yet.