Hacker News Reader

This reminds me a lot of the (I think) very underrated YaCy project: http://yacy.net/en/index.html

I know there are a ton of technical reasons why this is difficult, but web-scraping and the like always seemed like a problem that could be done fairly elegantly with distributed systems.

View on HN

AFAIK YaCy "peers" are just instances of solr doing search/indexing on their own (as set by the user) and the "distributed search" page only redirects the user to one of the available peers with external port open, not much p2p going on.

It requires a lot of resources with little to show for it.

web-scraping and the like always seemed like a problem that could be done fairly elegantly with distributed systems.

Elaborate?

Reply on news.ycombinator.com