do not understand why iframe is a must here, why can I just scrape the whole page directly? still learning web scraping using scrapy.
Normally if you click a link with jQuery, you lose the current context after the next page loads.
By controlling it inside an iframe it's more convenient