Hacker News Reader

Comment by epi0Bauqu | Hacker News Reader

The harder problem IMHO is customer acquisition. Many companies have built (and still have) link graphs of reasonable quality, but they all struggle to gain real customers.

I started out before those APIs existed, and did all my own crawling & indexing. When they came out, I decided to focus on my value-adds because I thought that was a quicker path to customer acquisition.

Furthermore, I don't use Yahoo/Bing straight up, e.g. I re-rank, omit, etc. I also mix them with my own my index/negative spam index from my own crawling efforts.

View on HN

I don't use Yahoo/Bing straight up, e.g. I re-rank, omit, etc.

All re-ranking you can do is very limited with these services because they are black boxes: you don't see what factors went into ranking a page the way it happened, you can't tweak the weight of different factors, you can't add new factors. All you can have is hardcoded rules like "if there is a wikipedia page in the first 20 results, bring it on top" that don't really add much value, because if that wikipedia page is any good it would be on top already. With spam results it's similar: you can provide impressive customer service blacklisting spam on user requests, but major search engines are already pretty good in low-ranking spam so much that you don't see it if there are any other meaningful results.

Your marketing here on HN has been brilliant, you have some very interesting UI decisions and possibilities that Google doesn't have, but your added value is definitely not in improved ranking of results.

Thx. I can't reveal too much here, but I do a lot more in the reranking area such that the top 20 will look very different for many queries when compared. I think these improvements go a long way to improving search UX, but ranking is subtle and so doesn't get noticed much (except when failing miserably).

That's almost certainly the right strategy for you. The people who founded cuil had a solution in look of a problem, in that they came from the search infrastructure teams at Google and wanted to use those skills. It turns out that running a search engine on very few machines isn't much of a competitive advantage, or even having a very large index.

They also got a lot of marking out of their "ex-Googlers take on Google" narrative which probably wouldn't have worked out as well if they were using something like your strategy.

Reply on news.ycombinator.com