Web Scraping with Pandas and Beautifulsoup

This is a spam article posted by a throwaway HN account with very little content mainly focused on pushing traffic to a for-pay udemy course.

Edit: looking at every article posted on HN from "pythonprogramminglanguage.com" is the same junk... https://news.ycombinator.com/from?site=pythonprogramminglang...

Each account that posted each of the links in the above search was created just to post that link and nothing else. All spam.

This type of high-level scraping is what I'm trying to achieve with https://github.com/joelgriffith/navalia, but also handle use cases like JavaScript SPA's. Interesting how differently some libraries/runtimes treat operations.

I've recently started using Pandas as my main tool for data-cleaning in my Machine Learning projects. The ease of manipulation is unmatched compared to other tools I've used.

Anyone had a similar experience or have a better tool to suggest?

I prefer this approach over building custom crawlers with scrappy, because: it is easier to debug, and easier to manipulate the data.