Hacker News Reader

I enjoy using scrapy because it allows for a bit more functionality. Check it out if beautiful soup is too simplemente for your needs.

View on HN

Which would you say is easier for a beginner(to scraping and to python in general) to use?

BS4 is easier but scrapy is more powerful. E.G., with scrapy you can program to follow all 'next' links and download 100s of pages in parallel. BS4 is more for grabbing content from a single page.

Scrapy is a framework and does require knowledge of python oop and idioms. One thing is that knowing scrapy is a career boost because there is a good market out there for it.

What are your goals? Simple scraping? Then bs4 is ok. If more complex then go for scrapy. Its the django of web scrapers.

Reply on news.ycombinator.com