Hacker News Reader

> While scraping competitor sites for pricing info etc is exactly as sleazy as it sounds

Why is it wrong to scrape your competitor's site for pricing info? If it's publically accessible, it's free game. As long as such scraping doesn't cause any undue DOS'ing or prevent anyone else from accessing such info.

View on HN

It typically becomes rather "challenging" to justify to management why one shouldn't scrape "aggressively" until the competitor's site degrades or falls over. That's why it's generally kind of sleazy.

Large eCommerce sites are constantly scraped by competitors, market research companies and even vendors (to ensure sellers honor MSRP). Scraping activity is usually throttled at reasonable levels.

Could you tell us more about your "typical" experiences?

Reply on news.ycombinator.com