Qwant: A European search engine that respects privacy
about.qwant.com
about.qwant.com
Sadly, the search engine reports many duplicate results. For example, I searched: "arid land reforestation", and results were (https://screenshots.firefox.com/wl6JuVa5sbWNgy2o/www.qwant.c...):
* 1st: http://haringey.store/reforestation/in/reforestation_in_arid....
* 2nd: http://haringey.store/reforestation/in/reforestation_in_arid....
* 4th: http://maestron.store/reforestation/in/reforestation_in_arid....
* 5th: http://pipcoins.store/reforestation/in/reforestation_in_arid....
etc, etc. All of which were "bad" websites. It's a shame because some good results (science direct, …) were inside the lot. The content isn't even a pdf.
You should implement a "unique" webpage result fingerprint in order to avoid showing duplicates (well, I suppose it's easier said than done..).
It is, mainly because people have very different ideas what "unique" actually is. At one time I had add the host of a page to their hash because people complained that if we hit the same content at two different hosts we filtered one out, which wasn't okay for political reasons.
The two first results in your example are "fun". That page returns different content with each call ... yay.
The basic principle of decentralised search is an individual only needs a tiny fraction of all knowledge available, so the solution needs to focus on indexing their interests and social connections - see http://www.mathaware.org/mam/04/essays/smallworld.html
It has been tried already. There's YaCy https://yacy.net/en/index.html but the UX is horrible ( https://yacy.net/en/Screenshots.html ) - not for the mainstream software consumer.
Then there's http://www.faroo.com/ ( see https://en.wikipedia.org/wiki/FAROO ) which has been ticking along since 2005 and tried to appeal to consumers but arguably aliened early adopters most by targeting Windows. They seem to have stalled otherwise ( http://www.faroo.com/hp/p2p/history.html) but it still works as a service I believe.
Lastly there's https://www.bitclave.com/ - the new kid on the block. They seem to be aligning themselves with crypto-currencies somehow - perhaps there's a path there to monetisation - see https://www.cryptocoinsnews.com/bitclave-search-engine-taps-... and https://hackernoon.com/bitclave-the-decentralised-search-eng... - one to watch.
That may be a reasonable assumption but it omits a major tradeoff: some types of "knowledge" require the search engine to "see" the entire index and synthesizing such knowledge cannot be delegated to decentralized nodes.
To reduce the search engine tradeoff to a simple math analogy: You can "decentralize" the calculation of a SUM to individual nodes. However, you can't decentralize the calculation of an AVERAGE. Divide-and-conquer strategy of calculating an average will give the wrong result.[1]
Some searches are like "SUMS", and some are like "AVERAGES". If a search is pure text retrieval without regards to relative weighting or ranking, decentralized nodes with each holding partial segment of the index will work. (E.g. you want to retrieve a chili recipe so your search query hits the decentralized node that has recipes starting with "C" or whatever hash algorithm.) That's a "SUM" type of problem. However, if you want to find the top 10 ranked recipes by links, the decentralized nodes can't answer that. That's a "calculate the AVERAGE" type of problem. Also, determining which webpages in the index are "fake news" or SEO "link farms" and other "bad actors" is also an "AVERAGE" type of problem. The machine learning algorithms need to see the whole index (or pedantically, a very large portion of it) to "learn" what "fake news" looks like.
Google's ability to comb their entire index in a centralized place lets them discover new insights to evolve the Pagerank algorithm from Panda->Penguin->Hummingbird->etc.[2]
Yes, you gain more privacy with decentralized or p2p search, but you also lose the ability to search on certain types of queries.
[1] https://math.stackexchange.com/questions/115091/is-the-avera...
[2] https://conversion-hub.com/wp-content/uploads/2016/08/Compar...
Another way is by connecting to a search engine within the Tor network, like DuckDuckGo's hidden service: https://3g2upl4pq6kufc4m.onion
Exalead, a former real Google competitor made in France is a real search engine, used to be sponsored by European Union, France and Germany, and features a real large scale web crawler. Unfortunately it got bought by Dassault (Catina 3D CAD) and stalled, though it's still online: https://www.exalead.com/search/
I like this - the default choice should be to explain rather than the other way around. There's a huge "information symmetry" gap and the more the gap widens the more difficult it is for people who don't have the required information to prove something violates privacy.
Also, I don't understand why the searches only show up for 1/3 of the screen. Is it because they plan to show big ads on the right? Still 2/3 of the screen for that seems excessive.
* qwant.com did not return useful results at the top
* google.com did okay, but answer was behind the first link
* duckduckgo.com displayed the answer at the top of the page
Defaults to French Though