Hacker News Reader

Comment by not_your_vase | Hacker News Reader

  > Oh, and by the way, here’s how you can ditch Google and switch to DuckDuckGo.

The closing sentence is... well... seems futile?

DDG is Bing, and if you say that Google is all about AI-generated crap, then Bing turns it up to 11...

(Don't read it as schadenfreude. I couldn't be more depressed about it)

View on HN

I'm afraid that the real future is Kagi and the like. Kagi in particular is pretty good, but the point is that it's paid by users, not by advertisers. It's rather hard to stay user-aligned because large commercial operations hich depend critically on their web sites will tempt a search engine with sweet, sweet lucrative deals for slight rigging that "nobody would even notice".

Last I saw Kagi was highly dependent on Google for their search results and seemed much more interested in LLMs and other side features than in replacing that core part of their search stack.

FWIW, my understanding is that there just isn’t a real drop in replacement for the Google Index. Kagis focus on LLM seems targeted to me, and I wonder if they’re trying to figure out how to put it to work indexing for themselves, more than providing the search results to the user.

I believe Kagi uses most major search indexes as well as its own.

Additionally, I don’t think it’s fair to say it’s more interested in LLMs than focusing on search. I think it’s fair to say they’re interested in ensuring they’re offering a better, non ad-based search replacement.

Disclaimer: Not affiliated with Kagi in any way, just a long time happy user.

In 1998, the Web was incomparably smaller. Google could put their whole infra into a dozen boxes, and then grow along with the Web, gradually. Modern competitors don't have this advantage.

By now, crawling and indexing is a herculean task, and also quite expensive, due to the sheer size. There is Common Crawl [1]; at 400 TiB it is huge, but at 60 days refresh interval it's far from being very comprehensive or very fresh. Good for research, but likely not good for a commercial search engine.

[1]: https://commoncrawl.org/

Have you tried https://search.marginalia.nu ?

If you thought ".io" domains were unstable, you ain't seen nuthin' yet!

https://en.wikipedia.org/wiki/.nu

Of course. But they (well... he) intentionally avoid crawling mainstream internet, and also, the results are censored (just like with other engines) - it's not exactly what I'm looking for.

Reply on news.ycombinator.com