That’s part of the reason that “it’s to stop the bots” is BS. Academia and other similar uses will pay or just stop. Bots and other malicious uses will find work-arounds and just change where the game of whack-a-bot is being played.

Twitter could add anti-bot protection like Cloudflare’s CAPTCHA. I’ve seen that make a site unusable with Puppeteer. By “unusable” I mean that correctly solving the CAPTCHA either just gives you another CAPTCHA, or gives the “click if you are human” thing, and clicking that just goes to a CAPTCHA or another “click if you are human”.

It didn’t even require doing anything to/on the page with Puppeteer. Merely using it to open a browser window and then waiting, and using that browser window by hand during the wait to go visit the site that was using Cloudflare anti-bot protection ran into the problem.