Hacker News Reader

How about: if you want me not to scrape it, keep it off my internet??

Actually I'm considering building "API-fication" of websites with bindings for major languages (Java, Python, JS). With luck websites could participate by providing & maintaining a parseable API-sitemap.

This would open door to my 2nd project: orchestration a-la BPEL on top of websites. visual editor, macros, scripting. Call this PIPES 2.0

View on HN

Can you provide some use-cases for why this would be useful in a way that wouldn't violate most sites' ToS?

- a lot of online stores, hotels need to constantly update prices based on what competitors do.

- cleaning (big) data. Automatically reconcile data to canonical format / names using authoritative source (say wikipedia)

Can you understand even the simplest TOS? I'd argue most (all?) are too restrictive to be enforceable. https://tosdr.org/

Reply on news.ycombinator.com