They’re basically reverse engineering Google by looking at user logs.
Google will always have a leg up here because they have all the Google data.
And even if it does work for a while, there still needs to be the original signal to copy. Someone will have to crawl the web and index content.
I’m super eager to find new approaches to search, but another Google clone is not that.
If that’s not a Google clone I don’t know what is.
They're looking at how users use Google Search because the data's there. They're making a competitor to Google Search. That doesn't mean they're rebuilding Google Search's SERPs, or making a Google Search “clone”; I've got results from Cliqz for queries I'm confident have never been put into Google before, meaning it's functioning as an independent search engine.
This. Having worked in past life for one of their competitors, can confirm - what users click on (in SERP) is one of the most powerful signals for ranking. And who got (almost) all the clicks in the world? Google!
That's why it's so damn hard to beat them. It's the unreasonable effectiveness of data: more data (which they have almost all of) usually beats a smarter algorithm, and with 20 years R&D, theirs is surely not dumb.
Do the clicks belong to users or Google, that's an interesting question, though.
So there are Google and Bing - any others?
Qwant is a France-based search engine that has a similar privacy focus to DuckDuckGo.
Million Short tries to find sites that other search engines miss.
Wikipedia has a more comprehensive list:
In 2013 they said they were temporarily using Bing results they purchased as "training data" [wikipedia].
They haven't given any updates on that I could find (in English, at least), but they recently proudly announced they have 20B pages in their index in an article on their partnerships with Microsoft [betterweb]. Google's index is 1500 times larger, so I'm not sure how competitive their own index is [goog]. And, if they no longer needed to rely on Bing results, wouldn't they announce that?
[wiki]: https://en.m.wikipedia.org/wiki/Qwant
[betterweb]: https://betterweb.qwant.com/how-microsoft-tools-strengthen-q...
[goog]: https://venturebeat.com/2013/03/01/how-google-searches-30-tr...
I'd say they need to start somewhere. Using other search engine's results is a ways to get things started, so that they can build their own index on crawled content later.
For now, it's great to see another competitor for Google coming up.
A sufficiently strict intellectual-property regime might find this a copyright violation. But without any inside info, I strongly suspect Google & other incumbents already do similarly-indirect modeling of their competitors' behavior, via extensive query/click-trail mining, in ways that ultimately feed into improvements of their own systems. So, they might not want to press the issue.
Still, this creates a dependency on the competitor you were hoping to displace, where most of your earlier values comes from "drafting" in the easy-path they've already cleared.
Copying, learning would be a bit more precise, and I'm not kidding. We do not answer queries 1 to 1, query-logs are used to build a more concise model of the page. It's not just a cache, we started like this, but we quickly learn to answer unseen queries.
A sufficient strict IP might find text snipped a copyright violation too. It's a trick area. Personally, I'm at peace as we get the content of web pages, as everyone else.
As for the dependency, it would be if we were not able to generate our own synthetic queries, which we are. So even if all other search engines of the world were to disappear, we would still be able to operate. That was not always the case, as you pointed out.
Interesting you bring this up. Wasn't the company funding cliqz in favor of text snippets being copyright violations when the big G does it? (German/EU Leistungsschutzrecht) [0]
I mean, I think bootstrapping from google query logs is fine, but following the money it does seem like a double standard. Cliqz "stealing" from Google SERPs is fine, but Google news "stealing" text snippets isn't?
[0]: https://leistungsschutzrecht.info/stimmen-zum-lsr/pressearti...
(Relevant quote: Das Leistungsschutzrecht halte man nach wie vor nicht für falsch. Man setze sogar weiterhin auf ein euop. Leistungsschutzrecht, mit dessen Hilfe man sich erhofft, endlich Geld von Google zu erhalten.
sloppy translation: We still consider the [we-want-money-for-google-news-snippets-law] to not be the wrong approach. We in fact contiue hoping it will work out on a european scale)
As all German media? Yes, it seems that they were lobbying for it, not clear if they still are part of the consortium or not. Why? No idea, really. Perhaps they are diversifying, lobbying on one hand, trying to build a competitor with another. But to make an assessment of the "goodness" of their intentions I prefer to stick to the facts: besides complaining to regulators, or not, they do fund a potential alternative. That's very commendable, cannot name many companies that are crazy/adventurous enough to put a ton of money on something as risky as what Cliqz is trying to do. To sum up, if the lobbying is a minus, the building is a massive plus, a clear positive outcome IMHO.
Further, they don't have to be better than google in the quality of their search results. As soon as the results have good relevance, it's good competition.
What matters is that they are 'good enough', that there is a hint of competition to the Google-Bing monopoly. Offering privacy centric 'competition' is what they say their main aim is and what their success should be judged upon.
> A web search engine or Internet search engine is a software system that is designed to carry out web search (Internet search), which means to search the World Wide Web in a systematic way for particular information specified in a textual web search query.
Just because they don’t do the crawling like Google and Bing doesn’t mean they aren’t a search engine.
It probably can't happen now, since there are billions of websites, but it was a simpler time, and finding something you needed wasn't THAT hard.
The best idea anyone had, which I thought wouldn't scale but didn't have better ideas, was implementing a keyword registry à la AOL.
Now compare the world of search to the world of mobile phones. Imagine if mobile phones only came with proprietary apps, and there were no app stores. That's where search is right now.