Scraping tools have become more powerful than ever, but bot restrictions have become equally more strict. It's hard to scrape reliably under any circumstance, or even consistently without residential proxies.
I then started randomising every aspect of the scraping process that I could: The order that I visited the links, the sleep duration between almost every action, etc.
As long as they don't implement a strict fingerprinting technique, that seems to be enough for now