I'm a dev who's built both APIs and scrapers...

The API-first dream is nice in theory, BUT in practice most "public" APIs are behind paywalls or rate limits, and sometimes the API quietly omits the very data you're after. When that happens, you're flying blind if you refuse to look at the HTML...

Scraping isn't some moral failing... it's often the only way to see what real users see. ALSO, making your HTML semantic and accessible benefits humans and machines alike. It's weird to shame people for using the only reliable interface you provide.

I think the future is some kind of permission economy where trusted agents can fetch data without breaking TOS... Until that exists, complaining about scrapers while having no stable API seems like yelling at the weather.