Hacker News Reader

Currently I am using Selenium RC + jQuery for scraping. It is slow but it is the most reliable (due to using a real browser) solution you can get.

Anyway, will try your method soon.

View on HN

Could you provide mode details on this, either in the comments here or in a blog post? I almost went with Selenium but ended up with a custom system to drive Firefox, would love to learn more about what the Selenium based solution looks like.

Just a few lines of code, no magic needed.

Example: https://github.com/tszming/Selenium-Google-Scrapper

Reply on news.ycombinator.com