I tend to tell this to people quite often, but depending on the use-case – it might actually make sense to use something like Mechanical Turk to get complicated scraping done.

But if the content is pretty neatly and easily scraped then the other recommendations make sense.

Though a bit of Javascript can now take you a long way ;)