Hacker News Reader

Because ~75% of clicks on search results are the top 5 results, we were less interested in the long tail of results, and focused on the top results from the most popular sites.

We developed this experiment to see if the top 5 results in context of the top sites would give a better result set than the top 15-20 results from any single search engine.

View on HN

Well unfortunately, they just don't. I too have helped built a limited domain search engine. The site, www.trailbehind.com, crawls about 20 of the best hiking related sites, including wikipedia, everytrail, localhikes, etc.

Unfortunately, the long tail is where all the action is, and my site and your site will both be marginal compared to Google.

One way to think about it is they can always do what you do, but they can also use the context of the link graph that you don't have, so you really cannot ever hope to compare with what they do.

Everything you/I can do with 20 websites, Google could do if they wanted. But it's just not worth it.

Reply on news.ycombinator.com