Agreed. So much so I started two search engines, https://bonzamate.com.au/ which is explicitly for Australian sites and https://searchcode.com/ which is about to get a large index update and make it more useful.

I have often wondered if perhaps a distributed search engine is the real answer. Yacy was interesting but the results terrible. But perhaps it’s possible to use activitypub protocol to build a distributed search that has different implementations on the backend. It’s something I keep toying with in my spare time, and with what I did with bonzamate we’re the search is done in lambda functions on aws possible to roll out in a way that’s cheap for everyone to build out. The ability to federate between those providing good results, even at query time has the ability to be very powerful if implemented correctly.