Wikipedia founder to challenge Google & Yahoo | Hacker News Reader

Wikipedia founder to challenge Google & Yahoo

news.yahoo.com

3 pointspython_kiss5 comments

I don't think the idea of winning by having super-duper rocket scientists is antiquated. In fact, I think the world is just learning how fearsome a company can be if they get all the smart people.

I entirely agree. The best way to build something is to get alot of smart people working on it--or rather, just enough smart people. The thing that Wikipedia leverages so powerfully is the ability of a wiki to let people work in parallel. Unfortunately, that advantage does not translate to building a search engine--you can't have people working separately, each contributing intelligence in their own domain, and expect to build a working algorithm.

I'll give them the benefit of the doubt, but as far as I've seen, Jimbo is pretty much just flying around in his roflcopter talking up vapor on this one.

It is hard to believe that any small startup can compete with Google anymore. Search is Google's domain and it will defend that turf to death. With more than 15 billion webpages indexed, it is no longer about superior technology alone but also the sheer mass of content available.

While I agree that it's very difficult to compete on generic search, it's certainly not hard to compete in niche domains. And you don't need anything like 15 billion pages for those.

However, you need the correct subset of the 15 billion pages, which is only fractionally easier, and in some ways harder: google can just grab everything, and then pull semantics out. If you want to leverage your limited domain, you need to be able to be able to have semantics in your indexer/crawler, otherwise you're going to end up having to index everything anyway.

One exception, of course, is in genuinely finite-domain search engines, like Octopart. There, you know exactly where to send your indexer, so you can be very efficient.

Reply on news.ycombinator.com