Hacker News Reader

Data hoarder that I am I would like it to scrape more of the site than I actually visit ... in case I missed something when I'm browsing offline later.

View on HN

Yeah, a "spider this page with depth X" would be a neat addition.

Edit: Though the way it works as a sort of proxy, it seems like you could combine it with something like this, and it would just work: https://github.com/naoak/chrome-site-spider

Cool thanks for that, indeed it seems like it would work.

Would something like this work for single-page applications? Given that most applications today are built entirely through the DOM, would regex matching on embedded scripts be that useful?

I've actually started thinking about how to build a spidering/scraping tool that extracts links/JS, etc from an SPA but would rather not reinvent the wheel if possible.

Reply on news.ycombinator.com