As someone who writes web scrapers for a living, I have only come across one site where I have been unable to reliably extract the information we need. If we were more flexible, we would be able to deal with this site too. Defending yourself from scrapers is an arms race you are almost certain to loose.