I use YQL to do web scraping. It lets me do something like this:
`select * from data.html.cssselect where url="www.yahoo.com" and css="#news a"`
Could you elaborate on the benefits of using Upton instead of this?
`select * from data.html.cssselect where url="www.yahoo.com" and css="#news a"`
Could you elaborate on the benefits of using Upton instead of this?
http://railscasts.com/episodes/190-screen-scraping-with-noko...
As well as Mechanize when working with sites that require session cookies and all that.
I'm wondering too what the advantages of Upton are?
Upton can scrape a whole set of pages. If you have a page that lists the pages you're interested in; suppose you're interested in HN commenters on front page posts, you could specify the front page URL and a selector for links to comment pages, and Upton would automatically scrape those pages and return them to you.
Upton could even write the commenter names to a CSV for you with just a filename and a CSS selector/XPath expression.
It's not stuff you couldn't do with YQL or Python/BeautifulSoup. But it's stuff that I didn't want to have to write over and over each time I wrote a new scraper.