Kagi employee here. We're actively working on building our own indexes beyond the limited ones we have now, not just a general index but also purpose built indexes for things like programming, etc.
> Teclis is an attempt to surface the less known web, the web of creativity and self expression, the more humane web.
> Teclis includes its own crawl as well as results from Kagi Small Web index and results with permission from Marginalia Search.
> Teclis works best with broad queries such as 'machine learning', 'vegan diet', 'religion' etc..
Is there another crawler doing the general-purpose stuff?
I'm not sure antitrust will help you.
For everybody else there’d Google I guess.
I imagine there is still a whole load of stuff out there on the internet that Google would never surface because it doesn't have enough adsense or whatever. Are you finding that?
I mean it sounds like that already has a lot of overlap with our Small Web indexing efforts, so that part of our indexing efforts could be an extension of that. A lot if this is still in development though so I can't speak on specifics just yet.
Lip service. You'll have some token index of Wikipedia or something so you can say your results are "a blend of our own index and other sources".
https://blog.kagi.com/waiting-dawn-search#:~:text=Wikipedia,...