Comment by kodablah | Hacker News Reader

Comment by kodablah | Hacker News Reader

Parent Full thread

There is definitely a market for this and has been an idea I've been toying with for some time. Instead of a proxy per se, it'd be an API on top of a headless browser scraper then a simple web UI that uses that API. It'll have easy-to-update content extraction/interaction scripts for popular sites. Essentially you'll have "craigslist-ified" much of the popular web into an almost AOL like portal. Users will be encouraged to run client side (this won't be hosted, too many legal issues) and while this won't prevent tracking per se (headless browser still tracks), it is an easily exposed web server from your desktop. So you could even expose via Tor (time lost is made up for lack of content) and if you wanted to share, you could. Caching/storage and eager ahead of time loading/scraping of common sites, and it'll perform even better. Anyone can use the API for whatever they want.

Wrap it up with an easy to install bow and easy self updating to keep up with sites (and easy fallback to original sites/links), and I'll never browse the mainstream sites again. There are a couple of things that do this, but none as well as I'd like to see.

There are certain kinds of websites that usually have category->list->individual image/video that are otherwise obnoxious to use, but all follow this structure, where this is the only way to get reasonable UI. Fetch data and make a simple UI yourself.

Fetching data from such websites is the only thing so far where I've found ES async generators very very useful.

You don't even need a browser. Most of the time simple HTTP requests from node work just fine, and makes tor use safer and more effective since you're not running any foregin JS code, just parsing data. Other thing that improves privacy is that you're downloading everything, so it's hard to see from the service's side what you're actually consuming.

Actually many usual services follow this pattern. List of accounts->list of transactions->transaction details. List of categories->list of articles->article detail. List of product categories->list of products->product detail.

Simple abstraction can get you very far, even with a fairly simple DB schema.

Speaking of browsability, Google Images by far outshines the majority of webshops in search relevancy, SNR ratio, results per page and speed.

I'd like a locally-hosted IFTTT+Appium type thing that'd use my local hardware/software/applications in place of myself... basically turning all interactions (viewing data, posting data, checking for updates/changes) into an API for accessibility software, home automation software, custom notifications, etc.

Reply on news.ycombinator.com