Hacker News Reader

Comment by nine_k | Hacker News Reader

I'm afraid that the real future is Kagi and the like. Kagi in particular is pretty good, but the point is that it's paid by users, not by advertisers. It's rather hard to stay user-aligned because large commercial operations hich depend critically on their web sites will tempt a search engine with sweet, sweet lucrative deals for slight rigging that "nobody would even notice".

View on HN

Last I saw Kagi was highly dependent on Google for their search results and seemed much more interested in LLMs and other side features than in replacing that core part of their search stack.

FWIW, my understanding is that there just isn’t a real drop in replacement for the Google Index. Kagis focus on LLM seems targeted to me, and I wonder if they’re trying to figure out how to put it to work indexing for themselves, more than providing the search results to the user.

I believe Kagi uses most major search indexes as well as its own.

Additionally, I don’t think it’s fair to say it’s more interested in LLMs than focusing on search. I think it’s fair to say they’re interested in ensuring they’re offering a better, non ad-based search replacement.

Disclaimer: Not affiliated with Kagi in any way, just a long time happy user.

In 1998, the Web was incomparably smaller. Google could put their whole infra into a dozen boxes, and then grow along with the Web, gradually. Modern competitors don't have this advantage.

By now, crawling and indexing is a herculean task, and also quite expensive, due to the sheer size. There is Common Crawl [1]; at 400 TiB it is huge, but at 60 days refresh interval it's far from being very comprehensive or very fresh. Good for research, but likely not good for a commercial search engine.

[1]: https://commoncrawl.org/

Reply on news.ycombinator.com