I developed some hot strategies for the next level ...

http://edinburghhacklab.com/2013/09/probabalistic-scraping-o...

Have you thought of implementing a system for custom handling of edge cases? Realistically you can't handle all edge cases, so leaving the last 2% up to the user (which is a developer) would be a good idea. It'd still save him 98% of the work, but also give him the comfort of knowing it won't break even if site X is doing something weird with their HTML in the future.

I'll leave it to you to figure out how one would implement such a feature ;-)

Sounds good. I've made a note of it for the next time I need to screen scrape something.