Show HN: Changedetection.io detect changes in websites and JSON feeds
github.com
github.com
For anything that I would need this particular service for, my highest-concern requirement is that I'm able able to "set and forget" the service for literal years at a time. But almost always, services like this just drop dead eventually; or they tweak their scraping algorithms in ways that make previously-working configs break; etc.
Is there a change-detection service like this, being provided with stable SLAs by one of the cloud IaaS providers?
I suppose I could always find a self-hosted software product for this, and deploy it myself. But though the software would be stable, that's not very "set and forget" either; I'd be on the hook for applying updates, etc.
I want something that is to the "HTTP response change detection" abstraction, as S3-alike object-lifecycle event notifications are to the "object-storage change detection" abstraction. Some simple plumbing primitive that does exactly one thing, exactly one (obvious) way, that can be used to build the higher-level abstractions I want, with nine nines of uptime behind it, and the expectation it'll stick around for at least the next decade.
You're looking for Zyte (formerly ScrapingHub).
Are you sure? it seems focused on scraping/crawling/data extraction, and I'm not seeing any built-in capabilities to (for example) simply trigger a callback when a resource changes.
I think that also points to why what you propose would be very hard to do: in HTML pages, even if the page structure (a new div level) changes, it's hard to say if that's actually a fundamental change to how they've architected the information hierarchy you're trying to get notified out, or if that is indeed the information itself. It's ambiguous.
Maybe that's okay? Like, if all you want is a notification about every time a thing changes, regardless of the meaning of the change, then what you're describing is possible. But I think the meaning of the change is important, and that's ultimately what shifts over time in ways computers cannot grok.
They have an okay free tier (65 checks per month).
I just wish their pricing model was less check-based. I would use it a lot more if I could pay $10–20/month for unlimited checks, but that's not something they offer. Their pricing model scales up really aggressively compared to the underlying resource usage.
I have been using it for years, and—judging from it's visual design—it has been around for years longer.
Small feature wish: optionally fetch and display favicons next to the URLs or <title>s in the website list.
Were in a similar space, coming out with a SaaS in a few weeks.