Hacker News Reader

I was thinking about this. I'm lazy and not very talented, so is there any decent code existing and open sourced that could be adapted to spider only sites without advertising?

View on HN

Just crawl, and then disregard sites that have certain scripts present.

Yeah, technically, it'd be very easy. I've written countless spiders, crawlers, etc.

Basically, only index items where webpage.indexOf('google') < 0

Reply on news.ycombinator.com