The world needs more search engines | Hacker News Reader

The world needs more search engines

0x65.dev

795 pointskkm519 comments

Google's dominance is almost entirely due to the fact that its by far the best at search. I use ddg, but I use google search via !g about 25-50% of the time after a failed attempts. And 9 times out of 10 Google gives me exactly what I'm looking for.

For instance:

"the actor that plays the news guy in spiderman"

ddg:

Tom Holland (side bar)

Spider-Man Homecoming (imdb)

Tom Holland (wiki)

Spider-Man: Into the Spider-Verse (imdb)

google:

Jonathan Kimble Simmons (splash, link to wiki)

J. Jonah Jameson (wiki)

J. K. Simmons (wiki)

The google result is exactly what I want. And the results were made in incognito mode so Google wasn't able to cheat with privileged information about me as a user.

At the end of the day, most people care about the product. I'm only willing to sacrifice so much to satisfy the ideal that there should be less concentration. Make a better search engine but trying to pull at the heart-strings of users about how Google is an empire and too powerful just won't work and it undermines your product and mission.

It is really good when you search for vague terms, but it seems to come at the cost of ruining precision technical search.

Google also censors some results in controversial categories, where Bing/Yahoo return what I am looking for. I don't trust a search engine that censors entire categories of results.

I am not an adwords user but anecdotes in another HN thread seems to affirm that Google is due to be disrupted: https://news.ycombinator.com/item?id=21667484

to give a fun example for those who didnt run into this silly agenda.

"free energy" use to produce countless results that now require extra keywords.

Under "free energy suppression" you find professionally crafted hatemongering. The actual list of claimants is huge, non of it is here. https://peswiki.com/directory:suppression#Wiki_2796702

"free energy device" also produces really crappy results compared to what it was.

It only seems like things argued not to exist may or must be scrubbed from seach results. Astrology wasn't scrubed nor was any religion. Everything has its history too!

I just searched for “free energy” with both google and bing and with both engines got a combination of the use in actual thermodynamics plus “omg free energy device exist coverup!” type results.

What kind of technical queries are you running? I just searched for "HLCD0438 datasheet" on Google and DDG. I think it is a great example of very precise, and very techinical query.

DDG's first page was only fake sites and wrong answers.

Google gave me the right answer right away (in the positions 2 and 3)

(for those who don't know legacy electronics: I was looking for the box with labeled pins, like on first page of https://www.turus.com.tr/class/INNOVAEditor/assets/PDF/MM584... )

> And the results were made in incognito mode so Google wasn't able to cheat with privileged information about me as a user.

that's not what Incognito mode does. It prevents your search from being included in the browsing history and doesn't send cookies from active sessions, but that's about it. Google still knows this is you being unauthenticated. You don't need to be logged into google to be reliably targeted with ads that fit your profile.

Why do you think Google Search still knows who you are when you search in Incognito mode? (Or, in Firefox, if you open a private browsing window.) How would it know?

There are advertising companies that use fingerprinting for ad targeting, but Google doesn't.

(Disclosure: I work at Google on ads, speaking only for myself)

How does it know to offer (in incognito mode) a list of your accounts you might want to re-login to? (Not always, but often enough to raise an eyebrow.)

Seems like a pretty good reason to think it still knows who you are.

As to how that works, you tell us.

Are you talking about the browser's autocomplete? That's not something servers can detect unless you interact with it. Or do you mean something else?

No. On login Google shows you your email address so you can click on it and only fill the password. Which means it knows you were logged in on that browser previously.

It knows that one from cookies, not applicable to incognito mode. Google probably does some other fingerprinting as well though.

That shouldn't be possible in private browsing, unless you've previously logged into Google in that same private session.

One way you could see something similar to this would be if you opened a clean session, logged into Google, logged out of Google, thought you closed the last incognito window but didn't, and then opened a new incognito window? Then the user cookie would still be in client-side storage

Are you using random Mac addresses?

The MAC address is not sent to the server, or accessible to web pages client-side, so that wouldn't matter.

It is if you're using Chrome or got any Google related properties installed on your PC.

I don't think this happens. How do you think it's being transferred? A header sent only on requests to Google properties? A special JS API in Chrome that only Google knows to call?

In theory. But it's not impossible for a nefarious browser to send it along as an HTTP header. Or a nefarious AP:

https://news.ycombinator.com/item?id=21463266

Your link describes the Milan airport captive portal putting the MAC in the URL (don't do this). The referrer is automatically attached to any requests the page makes. This is a comically broken configuration, not something at all common.

I realize that the last time I liked at this was in the context of wireless devices on a private subnet - which is why I thought it might be relevant. However I am curious, why is it, then, that Windows 10 has gone for randomisation?

Chrome is linked to your google accounts and saves your account auth tokens. When you use incognito, it doesn't immediately send that info. However, it does offer to sign you in. This is different from remembering login/passwords.

So you're confirming that Google tracks you without using fingerprinting methods? That's not really surprising considering how much they data have on people.

It's also a good example of their monopoly position; Android, Chrome, Chrome OS, advertising and analytics code on almost every website, ownership of multiple of the most popular websites and services on the internet puts them in a unique position that no one could ever hope to compete with realistically. Competitors have to rely on imperfect fingerprinting whereas Google can probably detect you with more accuracy than a DNA test.

> So you're confirming that Google tracks you without using fingerprinting methods?

That's literally the opposite of what s/he just said. The person you're responding asked "How would it know?", implying that they (while being on the Google ads team) think there is no way to know without fingerprinting (or cookies from non-incognito mode).

Right: fingerprinting is the general term for using something other than client-side storage (cookies, local storage) to determine identity. Incognito mode and other private sessions intentionally don't preserve client-side storage.

There are other ways to fingerprint users, though if Google is using them they’re certainly not making it obvious by allowing users in incognito mode the ability to sign in to their associated account.

"fingerprinting" is a catch-all term, so I'm not sure what you're saying with "there are other ways to fingerprint users".

I’m using it as “(ab)using metadata to associate activity with users in a nonobvious or undesirable way”.

If I may ask you a personal question: how do you feel about working on ads for Google, given that a lot of people find Google's tracking practices (to make personalized ads possible) questionable at best? Did you specifically choose that team, or was that just were Google needed more hands?

Personally, Google ads give me mixed feelings. I see how personalization is useful for everyone involved and, so long as only machines look at my data, I don't have any personal issues with it. But at the same time, Google collects everything on everyone worldwide to the point where I feel like the USA would have an easy time conquering any country they please (if a nation already has live data on pretty much all its enemy's subjects, war would be exceedingly efficient for them to start and quickly win), so that kind of threatens our freedom if you see what I mean; and secondly the data is not necessarily 100% secure, so in the event of a breach it might be seen by humans, specifically people that I would not want to know what I searched for (or pages I visited that have Analytics or an embedded YouTube video or ads or a map on their contact page or ...). So it's a mixed bag of feelings and your position (job) seems like the kind that would make one think about before accepting. I'm curious to hear your thoughts on it.

> how do you feel about working on ads for Google

I've written some about this: https://www.jefftk.com/p/value-of-working-in-ads

"Many people would put ad tracking on this list of downsides: sites pass information to data brokers that build custom profiles for each user and allow personalizing ads. From my perspective, however, while having this information collected seems a bit creepy, it allows showing ads I'm more likely to be interested in. This makes publishers more money than showing untargeted ads, and I'd much rather fund them through better ad targeting (invisibly intrusive) than through more obnoxious ads (visibly intrusive)."

I chose this team because I thought the work would be interesting and I liked the people on it, and they were interested in me because of my prior work on mod_pagespeed rewriting websites so they would load faster.

> if a nation already has live data on pretty much all its enemy's subjects, war would be exceedingly efficient for them to start and quickly win

Lots of thoughts:

* I think you're dramatically overestimating how much data Google has and how well that is mapped to the kind of identity the military would care about.

* I don't think Google would share this information unless legally required to, and I don't think such a request would be constitutional.

* Many other countries are in similar positions; for example Criteo is based in France and has a similar ad tracking reach to Google.

* I'm still not sure how this is especially useful militarily. Military targets are mostly not in the data one of these companies would have, and none of these countries would go to war targeting civilians.

Thanks for the response.

> I chose this team because I thought the work would be interesting and I liked the people on it

That is fair! I guess most people would make that decision if you already know people there and you think you'll enjoy the work as well.

> none of these countries would go to war targeting civilians

Not as if people in the army are somehow exempt from tracking though?

As for whether Google would share it in the first place: I don't think the government cares much what Google thinks if they're willing to kill (us) over something. Laws can be made by the same people that decide on this. I don't mean to pose it as a simple matter, but I'm pretty sure that's how it works in principle.

Now that I think of it: aren't "national security letters" exactly this? "It has something to do with the safety of the country, just give us that data [e.g. Lavabit private key]"?

Of course, the chance is remote in the first place. Much more likely, if it is ever used for this kind of purpose in the first place, it'll just be posturing and threats, and people will protect themselves better before it ever gets to armed conflict. Just imagine, though, if you're not in the USA, China, or Russia, and one of the three (the most democratic one of the tree, it is fair to add) has the rest of the world's data. That's kind of uncomfortable when I pause to consider it.

> how well that is mapped to the kind of identity the military would care about.

While not readily available, I expect that it's not hard to find a few datapoints to filter them out. Following someone for 10 minutes as they go through traffic and matching the coordinates against location history data is probably enough to find a subset of 1-5 possible accounts. But I doubt physical following is even necessary to find enough datapoints to find them in the data.

The phrasing in his comment was vague. The first part "Why do you think Google Search still knows" can be read like a rhetorical question, especially when combined with the outrageousness of the alternative (that Google isn't tracking you)

And if he really is saying that Google doesn't track you in incognito mode, then I'm going to go ahead and assume he's either lying, or he's not in a position to know about that system. This is Google we're talking about here.

https://www.theguardian.com/technology/2017/nov/22/google-tr...

You're morphing their plain English into a different meaning and seem completely convinced that you're right. I'm not sure there is a point talking if you are already convinced to an extreme extent.

For the record, I'm not saying that I expect Google not to track me when they detect some privacy mode. It'll sure try to set cookies, and it may use my IP address and connect whatever that IP accesses as a weak indicator of interest for anyone else with that IP address (for a limited amount of time, since IPs change in many countries). What I don't think is that, when they say they don't do fingerprinting, they're lying. This person may not be privileged to know and say "I don't know", but that's different from saying "Google doesn't".

Also for the record, I didn't downvote you (and when you reply to me, I can't; I don't have an alt account with 1k rep or whatever it is one needs to downvote).

google records: ip address, browser fingerprinting.

Try visiting from tor

For spam / bot / fraud detection, yes. For ad targeting, no.

Source?

It’s in their best interests to also use it for ad targeting (in a plausibly deniable way so they don’t get in trouble).

We’ve seen them using dark patterns to coerce users into opting into more data collection, and another advertising company got caught using phone numbers for ad purposes even if they originally promised to only use them for 2FA, so why should we trust them this time?

Source: I work on ads at Google, and if we were doing this I'm pretty sure I'd know.

If it was being used for targeting it would be practical to run an external study demonstrating that.

Apologies but I can’t really trust someone who works at a company whose best interests are to violate people’s privacy, confirmed by all the dark patterns (both on the web and in Android) and their lack of GDPR compliance.

I would be very curious as to how you’d prove this is or isn’t happening with a reasonable degree of accuracy considering all the factors involved in ad targeting. Unless you’re willing to give us access to all your source code and SSH access to the systems running it, it’s reasonable people have their doubts.

While you can claim Google and its employees liars only to strengthen your own belief, but that only deteriorates the signal to noise ratio of this discussion.

And surprisingly for most of HN readers, Google has been pretty transparent on the policy of its ads business. In fact, Google has pretty strong incentives for transparency in this area due to advertisers, who give all the money anyway.

> considering all the factors involved in ad targeting

An external study to evaluate whether Google is using fingerprinting would be some work, but pretty doable. Targeted advertising is generally very blunt: if someone thinks you're especially interested in a valuable category they'll often pay a lot to advertise to you. So you could set something up where test browsers visit pages related to high-value categories (mattresses, asbestos cancer, credit cards, ...), clear client-side data, and then visit a site that loads ad scripts only from Google (to make sure you're not getting someone else's fingerprinting) and see whether the ads differ from a control group that never visited those pages.

How is this information logged/preserved, and for how long? Just because it isn’t being used for ads doesn’t mean a person is comfortable filling up a database with their activity.

IMO, ads are probably the least worrisome way the data could be used. A boring but scary example is that aol search history leak (which is still searchable today):

https://searchids.com/

This person is identified by name for example: https://searchids.com/user/19431784-joann_whitman

Was this proof that google ads uses fingerprinting?

https://meta.stackexchange.com/questions/331960/why-is-stack...

Reading the link, it sounds like that's (a) fraud detection and not targeting and (b) a third party, IAS, that the advertiser is including in their ads

Do you have any evidence of what you claim?

Anyhow, based on what I've seen, when you go into incognito mode it definitely doesn't use your profile when you go to Google services. Search results, suggestions, etc... are different. In fact, one of the use cases for incognito is using a different Google profile on a someone else's or a public computer. Keeping contextual search results from the 'main' profile logged into the browser would be counterproductive...

This comment and everything downstream from it are a distraction from their point: Run their query from your machine or from Tor and you'll see the same results in Google and lack of results in DDG.

Another example is a query for "elm dict". DDG has little idea what you're looking for while Google links you directly to the docs of Elm's Dict data-structure.

Thanks for the "elm dict" example. That's a really good one which actually does highlight practical differences between Google and DDG when I just tried it myself.

Most of the time I see comments like this, people don't provide hard examples of queries they found unsatisfying with one search engine compared to another.

> DDG has little idea what you're looking for while Google links you directly to the docs of Elm's Dict data-structure.

You know you're looking for a data structure, right? Add structure to your query, and DDG will do fine. Easy fix.

The opposite case, when google thinks he knows what you're asking but it's wrong, is impossible to fix.

I don't want them to be smart, I want them to be predictable and search what I say, not trying to reinterpret the query.

See, my experience is quite the opposite when it comes to programming documentation. Whether it's StackOverflow or Django/Python docs, DDG seems to always get me there faster. I switch to Google for absurdly specific errors because it's database is simply bigger, but every time I have to spend a few minutes throwing quotation marks around parts of the query to get Google to actually find anything useful.

I get the same results with a VPN, so regardless of how you slice it they win on relevance.

For this query, Google was better at understanding the question. Knowing the user had nothing to do with it.

To be fair, that example query is more of a personal assistant kind of question. If I want to know who played in that movie, either I know which site has that information (IMDB) and go there directly, or I search for something like "actors spiderman movie <year>".

If you want DDG to understand natural language queries, I think their privacy policy may need to be adjusted so that our queries can actually be used to develop that, and then they need a boatload of funding to catch up with Google's semi-ethical money-generating practices.

Indeed. I'm happy to forego Google and then (in cases like this) click once more to figure it out myself.

Also because AI (if I may even call it that) is not yet good enough to answer these things with high accuracy, let alone judge sources. I don't use Google much so I can't really speak for it, but from what I've seen, I'd estimate that 20% of the answers (for non-mainstream topics) are either misleading or false.

I type "name of the movie !imdb", which takes me straight to IMDB's search results for that name.

!bangs are the main reason why it's always going to be difficult for me to switch from DDG to anything else.

https://beta.cliqz.com has some support for bangs. Not as many as DDG but adding !g (google), !s (startpage), !d (duckduckgo), !so (stackoverflow), !w (wikipedia), etc works for me. Needs to be at the end, not at the start.

Like it or not, NL queries are exactly what most internet users want. Without them, any search engine is doomed to remain a niche product used exclusively by the very privacy-conscious.

Any sources? What I see is the majority use the search bar to input the name of a website or a brand.

Don’t use incognito mode, use the tor browser.

Have you considered using !sp (StartPage) instead? It essentially does "incognito mode" for you.

Can use the !s bang in DuckDuckGo.

Also, lot of links are hidden in Google by DMCA requests. PR firms use it a lot to hide "bad" information about their clients. Some countries (e.g. Russia, China) bans everything that doesn't fit their propaganda. Google is useless now for many kind of politic-sensitive searches or for investigations.

You're helping google by tagging its searches, so many people ask for this kind of things that they can prioritise results that gets the most clicks (people rank searches for google everytime they click).

By contrast, the new privacy friendly search engine from the article, with a lot less money and users, can answer a simple question like "news guy actor in spiderman" with

Cliqz:

- And the new actor playing Spider-Man is... this guy (www.foxnews.com)

- J. Jonah Jameson (J.K. Simmons) | Spider-Man Films Wiki (spiderman-films.fandom.com)

Not bad for a small German company uh...

And even if it didn't, when I know it's Jonah Jameson, I can search for the actor that played Jonah Jameson. Search engines are not about having "the answer to life the universe and everything" but helping users refine their searches until they find what they were looking for.

That may have been what search engines were for, but Google changed that. Very few people have the patience any more to try multiple queries to get an answer. When DDG fails, as it inevitably does, I don't spend time refining my query: I repeat the search on Google and get my answer.

It's true.

But Google fails as well.

Most of the world population is not native english speaker.

Try "attore che recita la parte del giornalista in spiderman" (the actor that plays the journalist in spiderman) in Italian

- Spider-Man: Homecoming - Wikipedia (https://it.wikipedia.org)

- James Franco - Wikipedia (https://it.wikipedia.org)

- Spider-Man film: tutti gli attori | Popcorn Tv (https://popcorntv.it)

- Martin Sheen, dieci ruoli per scoprire un grande attore ... (https://www.consigli.it)

Or the simplified version "attore che fa il giornalista in spiderman" (same meaning as before, just more down to earth)

- J. Jonah Jameson - Wikipedia (https://it.wikipedia.org)

- J. Jonah Jameson - Wikipedia (https://en.wikipedia.org)

- Spider-Man: Far From Home, nel cast anche l'attore ... (https://tg24.sky.it)

Refining is still a very present need, it's simply that for common searches on common topics in english it's less so...

It's more or less the same in German, French, Spanish, Portuguese... I can't even imagine the results in languages like Arabic, Indonesian or Balinese.

Local, culture aware, search engines are the future, despite Google efforts, the generalist "one size fits all" search engine is not.

> Google's dominance is almost entirely due to the fact that its by far the best at search.

Yeah, and due to economies of scale and network effects it will stay that way, unless some people are willing to suffer the minor inconvenience of using a slightly inferior competitor.

Why would they tho?

Because they have the ability to use available information to predict the future and don't like what they see if they continue to use Google?

Google's dominance is almost entirely due to the fact that its by far the best at search.

But is it? I find myself often having to fight it to search for what I entered into the box, rather than what it thinks I really meant. If you know what you want just not where it is, DDG and Bing are both superior.

I tried using DDG and found myself using !G a lot for simple things. Then I switched to Bing about six months ago and have only had to use google a few times.

I know, Bing.. But I have used Windows for about 30 minutes in the last year and that was just to help my mom fix her printer. I don't think I have ever had a Microsoft related account since Hotmail. I know they track me but I don't use MicrosoftDrive or WintowsTube so I can live with it.

"... so Google wasn't able to cheat with privileged information about me as a user."

The way they "cheated" was to look at the tens, hundreds, thousands, or tens of thousands of previous queries that matched yours in substance.

If the competing websites for "web search" had the same volume of submitted queries that are substantially similar to yours, then they too would be able to give you "exactly" what you are looking for "9 times out of 10".

If all people submitting queries on the web are somehow convinced to visit only one website and submit a majority, (hypothetically let's say 93%) of their queries there, then it should be no surprise when that website "magically" starts becoming more "predictive" than other websites in returning "exactly" what searchers are looking for and becomes "by far the best at search".[1]

The question raised by the blog post is whether enabling such "magic" is worth the trade-off of also creating a "panopticon" in that single website. Giving this level of visitation and query traffic to a single website makes any competing website (working off less than 7% of people's queries) seem irrelevant, maybe even pathetic.

Without the enormous traffic, I would surmise the glory of the "empire" (to use the author's chosen term), and its appeal to "followers" (e.g., those who marvel at its "magic"), might dissipate rather quickly.

1. Especially the part of "search" that involves dealing with repeated, similar queries for popular information.

Knowing what the most common questions are doesn't help you find the corresponding answers. At best you can find the result that users click on most frequently, but if all of the results in the result page suck then that's not going to help.

Google's advantage is simply billions of dollars and 20 years of R&D into NLP tech.

> Knowing what the most common questions are doesn't help you find the corresponding answers. At best you can find the result that users click on most frequently

I doubt that users have a strict expectation of finding the correct (interpretation of "corresponding") answer. Even if the most clicked answer[s] are still not correct, they may still be the ones who suck less, which can still be acceptable/desirable.

Keep in mind that in this perspective, the concept is similar to Google Translate - Google built a translator that, at least originally, doesn't understand language, instead, it applies (applied) a static model on a large amount of documents. While they certainly poured a large amount of money on it, it's success can't be centered purely on the economical factor.

Well in the given example the user asks a question "who was the actor that..." and Google gives them the answer. This is really hard to achieve by accident no matter how much traffic you get. It requires NLP machinery to answer a question, it's not a matching problem but an understanding problem.

There are at least three kinds of queries that a search engine has to handle. Requests for websites e.g. "Facebook". Traditional keyword searches across the web e.g. "Twitter ban political ads", and questions "Who was the guy who voiced bender?".

> Knowing what the most common questions are doesn't help you find the corresponding answers.

Not directly, but it does mean that, say, the second 50% of people that ask the same question will get a better answer than the first 50%.

Google has been able to build fairly accurate instant results based on which sites users were clicking on before. I'd say that a majority of simple general knowledge queries are solved by quoting the first 3 sentences of the wikipedia page that match the search query.

But let's say that there is no easy match to show a quick result for. But after 1000 queries, 98% of users clicked on one particular site on the first page and never went back to the search results. Google then A/B trials how many clicks result from showing that website at the top of the results page in an instant result window. If clicks drastically drop, that's a sign most users are satisfied with that result. They were only able to do that, because of the 1000s of times people typed that query and interacted with the site.

So I'd say that yes, having common questions asked over and over again does help you find the answer that users are looking for.

The funny / scary part here, is that this may not be the correct answer. But it's the answer that satisfies the most users, and is therefore the one that will keep the most people coming back to the Google search engine.

Just because google has seen my search before, how does that translate to being able to give me good search results? Search is an incredibly hard and nuanced problem. I doubt ddg's main disadvantage is training set of search queries.

Plus, you're underestimating the percentage of queries that google has never seen before. Google processes trillions of searches every year, and still, 15% of those queries have never been seen by Google before. [0]

[0] https://searchengineland.com/google-reaffirms-15-searches-ne...

"Plus, you're underestimating the percentage of queries that google has never seen before."

No. I am highlighting that 75% are queries that they have seen before. Note also that the 15% is a figure that is declining, based on what it was in 2007.

There’s some truth to that but Google also has a massive knowledge graph. For imdb and freebase like queries they can give very accurate answers as cards. Other humans don’t need to teach every single fact. They just need to teach how to extract and organize facts from a site.

When you see google giving you a one/two word answer as a card, that’s very likely coming from it’s knowledge graph.

What we need is more of this open knowledge graphs. Google and Wolfram Alpha are both closed sources but have deep understanding in niche domains.

I am not a Google search user so I am not sure what they are doing with "cards", however it sounds as though they are trying more to themselves become a "secondary source" (versus, e.g., just providing some "context" around the matched terms). The goal being that the user never has to leave the Google website.

For example, say the film credits for Spiderman is a "primary source", and a cast list for Spiderman derived from the credits at IMDb is a "secondary source". Google extracts the information from IMDb and substitutes itself as the secondary source.

Does this raise an issue in that ideally users should sometimes be retrieving information from (i.e., accessing) primary and secondary sources directly, whereas a third party always acting as a universal secondary source, e.g., a third party funded by advertising, might introduce (more) bias into the information retrieval process, e.g., in competition for "eyeballs".

From the OP references: https://sparktoro.com/blog/less-than-half-of-google-searches...

You might find this philosophically objectionable but the truth is that most people go to Google to find answers, not "primary or secondary sources", and are therefore better served by the answer fished out and presented to them.

It's also another matter that so much of the internet is now filled with copy-pasted crap and artificially inflated content that it's actually hard to find what you are looking for on the so-called primary source webpages. If I want to find George Clooney's age, I don't want to sift through a 2000 page ad-ridden Page3-esque gossip blog about him.

At the same time, when I want detailed info about something, I will go in and try to read through the primary material.

For such a simple question, why wouldn't you just use Wikipedia?

In any event, neither a blog nor Wikipedia would likely be a "primary source". Maybe something like a driver license would be a primary source.

It could be that you have a "philosphical objection" to "copy-pasted crap and artifically inflated content". I wonder if Google could have a role in encouraging the continued existence of this stuff. The effective opacity it creates seemingly justifies having an entity like Google.

I can’t just enter my question in to Wikipedia and get an answer back, as I can with google as the default search bar. Why would I use Wikipedia when it is more work than googling?

Assuming you entered the name of an actor that has a corresponding Wikipedia entry you would be redirected to the actor's Wikipedia page that, in most cases, shows the actor's date of birth and calculates the age for you.

Even when I entered the actor's name plus the term "age" and was redirected to the Wikipedia search results, I could still see the actor's page as the third result and his date of birth in the summary text.

As for why anyone would want to search some things using Wikipedia versus Google, I can think of a few reasons. I cannot speak for other users however.

I have a script I wrote to search Wikipedia from the command line. For those who care, one can strip out "X-Client-IP" from the returned page html before opening the page in a browser.

The claim:

"Google's dominance is almost entirely due to the fact that its by far the best at search."

is, and I'm sorry to say, a make-believe

Google's quality is better than anyone else, that is a fact. Let's go to the point. [[I work on Cliqz, and in search]]

Do you know how much Google pays apple to be the default search engine?

According to you, nothing, because people will go to Google because it's the best.

Well, it turns out that last year was more than 9 Billion (with a B). Quite a lot of money poorly spend, someone should really get fired :-)

This is business. Apple and Google bring something to the table and they meet in the middle. For zero dollars Apple could just let the user choose from a randomised list which is arguable better and more impartial.

Of course it's business, but it invalidates the premise that people chose "mostly" because of the quality. If that was the main driver, Google could just pay zero, 9 Billion for few hundred million users is quite an hefty sum.

The other option is that Google does not pay to get the Apple users, which might come otherwise, but to prevent Apple to try something funny, either directly or indirectly.

In any case, the "build a better product and people will come" mantra is flawed when companies in the space pay each other billions for distribution.

This logic is only valid if you assume Apple cares a lot about their users getting the best search results. I don't think this is the case.

Instead, Apple has a thing to sell -- "position of default search engine" -- and it is selling it to highest bidder.

I bet if Bing would offer 10 Billion, they'd make Bing default instead. Hey, if DDG could offer them 10 Billion, I am sure they would set it as default, easily ignoring the fact that many people say they do not like its results.

Forget about Apple for a second, they will sell whatever to anyone, for sure.

But please focus on Google, which is the one putting the money!

Why Google pays so much money if they already have the best product and what the users want? Are they just giving B10$ as charity?

There are many reasons for doing that, but none of them is aligned with the make-believe statement that "build a better product and people will come"

I would agree that is a necessary condition, but by no means sufficient, which is kind of sad.

Let’s say that 90% of users don’t care about search engine at all, and other 10% do care (for the sake of argument, I am sure the real numbers are quite different)

Google pays billions to get those 90% of don’t care users. They already have the users who care, but more users = more money, so presumably it is worth it.

Cliqz, DDG, and others have no chance for those 90%, but they fight for the rest. It’s a hard fight because google is so good. If one spends a hour searching for stuff on alternative engine, then goes to google and finds the results in a few minutes, they will be unliketo visit other engines again.

The google result is exactly what I want.

Yes, when what you want happens to be the lowest common denominator - in this case, the most recent Spiderman movie. When the next movie in the franchise comes out in a few years and you don't like it so much because the news guy is now played by someone else, you'll be complaining that your preferences have stayed the same but Google no longer handles them as well.

This implies that the results won't change.

When DDG doesn't yield the results I'm looking for, I will frequently try Google, but more and more I find that if DuckDuckGo can't find the results that I want, then neither can Google. What Google can do however, is annoy me with results that it knows to be wrong, because it is excluded the most specific part of my query. Google will remove the bit that absolutely had to be in the query, to get more hits on the majority of the remaining words.

In the end, I think most people could use either Google, Bing or DuckDuckGo and be happy with the results. It's just that we're stuck on "Google is the best search engine", and while that may be true in some technical sense, many of the other search engines are just as good for most of us.

I’ve found the opposite. For the last year, I have to resort to Google more and more often. DDG was on par with Google when I started using it a couple years ago, but it has managed to get worse and worse over time. Note that I’m specifically claiming DDG’s performance has degraded; not just that it has remained the same while Google’s has improved.

try startpage

For the "most of us" point, keep in mind that Google has a superior international infrastructure. At least in Finland, DuckDuckGo's loading time for search results is about 2 seconds compared to Google's <0.5s. This makes using ddg surprisingly annoying.

Google? No, Yandex.. obviously not blocked in Russia, also not blocked in China.

I don't really notice any infrastructure problems for ddg in Europe though, so Finland may be atypical?

I have the same problem in Japan. Certain times of the day DDG seems like it’ll never load and I normally switch to Google after about 4 or 5 seconds.

In my experience DDG's results are mostly 10+ year old "yahoo answers" posts, while google searches tend to be up to date and relevant.

I remember a time when Google would turn up countless Yahoo answers posts. Now it’s all Quora.

I don’t think it played a role here but note that incognito mode doesn’t mask your IP so the results will still be somewhat personalized.

Same here, more or less. DDG is 'good enough' in a lot of instances but I think it's more that Google has taught some 'bad' searching habits. When I type a random Google search I'm basically expecting it to read my mind (and essentially it does through all of the tracking and analytics); DDG has none of that so the method of searching has to be more specific.

That said, it does't prevent the situation where searching for certain gifs or images gives you a page full of softcore porn, even with the moderate safe search enabled.

I used to do that, but it's not necessary anymore.

You can almost always, by phrasing more carefully, get exactly what you need with DDG.

I also think that it's better to learn to search properly than finding the best search engine.

Not to sound presumptuous but what are you working as that this is a query that’s representative of your search engine use habits? I am using ddg for a few years now, I don’t revert to google (except google scholar) and I don’t feel like my life has gotten worse due to lesser quality search results.

Maybe it’s denial or weirdness on my side... after all I am also a vegan because I don’t want to hurt animals or other people just for a little bit of taste and convenience and most people seem to think that’s stupid on my side.

In my experience, this was true maybe a year ago or before. But nowadays ddg is quite good at providing what I was looking for. If it can't, google isn't much better either, but makes it look like it is, making me spend more time sifting through the information all over the place. The "People also ask" is a huge distraction, why is that immediately after the first result? It can be maybe at 5+ place, but not at the top.

I looked into the Cliqz browser a few years a ago. Back then they claimed to be a privacy respecting alternative to Chrome. What I found back then is that they sent every keystroke I type in the URL bar to their own servers. They outright lied about that in their terms.

For me their mission is pretty clear: Google ate Burda's ad cookies and now they are trying to get their hands into the cookie jar again. Given that today we have widespread TLS adoption the war about the endpoint has begun. Cliqz, alias Burda Media, is just another combatant - the one who controls the browser controls the ads.

Previous thread with more info:

https://news.ycombinator.com/item?id=18626790

They have an autosuggest feature in the browser which shows search results as you type. Goes without saying that this requires a request per keystroke. If each request contained a unique user id, that would be concerning, but that is not the case. that the search query itself goes to their servers is just common sense, but those requests are anonymized. (Disclaimer: I'm a former employee)

Keystrokes + IP = NSA knows exactly what I'm typing.

This "feature" should be off by default on any software that claims to be a privacy respecting alternative to Chrome.

Can you ID someone positively by their typing patterns and an IP?

Yes. Also search WW2 war stories about radio operators, this technique dates back since then. US Pacific operators always knew which Japanese radio operator was transmitting Morse codes by listening to their typing patterns. And if a human was able to do it, imagine the accuracy of computers in this regard.

I was a ham radio operator for several years. It was easy to ID my frequent contacts by their morse code transmission patterns, and they me. An operator's pattern was called their "fist".

You most likely can. Just listening to people type on a keyboard gives a very unique feel with each person. Try it sometime, listen to family members and colleagues type or find some youtube video. I can ID all my family members and some coworkers by the sound of their typing.

Obviously, if you can get the content of what they're typing, it gets much easier still. I think I've seen papers where they ID programmers based on the code they've produced. This applies to other types of writing too.

You can build a model from keystroke timings and figure out people's SSH passwords too. https://www.usenix.org/legacy/events/sec01/full_papers/song/...

Now do it with random network jitter.

The main problem here is that even if you could do it, why would you? There are over 3.4 billion Internet users in the world. Given that people share IP addresses and even browsers, what's the actually gain identifying someone through typeahead search keystroke jitter? This would spend a lot of effort, and and then tell you what that a cookie doesn't?

I can't imagine that it's actually worth the effort.

Yes, just like you can recognize someone by their footsteps. Most likely you don't even need the IP.

It's been my impression that privacy from the NSA is, realistically, not something any commercially viable web company can offer you. An exception can probably be made for a site that operates outside of US influence serving customers outside US influence.

Uhh, most products can offer plenty enough defense against opportunistic blanket surveillance (but broadcasting your keystrokes is probably not going to help there). Protection against targeted attacks is a whole different story.

If you are ok with some extra latency you can enable our “search via proxy” feature which proxies all queries and hide your IP. Maybe that is what you are looking for?

Oh, the requests contained much more than the keystrokes and I looked into the anonymization claims as well. I'd have to dig out my Wireshark traces to tell you exactly what was transmitted, but I'm pretty sure it was enough to id the user. And by the way Cliqz never claimed that the anonymization happens at the endpoint, from what I remember they were very proud of their proprietary anonymization technology in the backend.

And just one more thing: In Google Chrome at least I can turn off auto-suggest. This was not possible in the version of the Cliqz browser I tested.

Thanks for the feedback, if you find something fishy in the requests then that's like a bug and we will gladly fix it ASAP.

Disclaimer: I work at Cliqz.

For the current Cliqz Browser (1.30.1) when I type "pythux" in the URL bar the request looks like this:

GET /api/v2/results?nrh=1&q=pythuxs=UTlnJzAh4ULkORaiZgLPO6LW9LOWZZoO&n=5&qc=0&lang=en&locale=en-US&platform=0&o=%5B%5B%22custom-search%22%5D%5D&country=de&adult=0&loc_pref=ask&count=10&suggest=0

HTTP/1.1

Host: api.cliqz.com

1. I saw UTlnJzAh4ULkORaiZgLPO6LW9LOWZZoO in other requests too, but not all of them. Can you explain what this parameter is good for and which information it encodes?

2. Can you tell if it is possible to disable auto-suggest (or Quick Search how it seems to be called in your terms)? If it is possible then how do I do it? I couldn't find it in the UI.

3. Can you specify what the exact legal relationship between Cliqz GmbH and FoxyProxy LLC is and if there is any shared ownership or if there are any common parent companies?

4. As you can see above the requests go directly to api.cliqz.com. While the terms go into great length to explain that information is routed via a third party owned proxy the information we are talking about here is exempt from that. I quote the relevant passages here:

> This channel collects signals about WHAT you search and where you land. That is why we do not collect any personal identifier here, which makes it impossible to associate searches with users. Moreover, all query entries and clicks on website suggestions are evaluated only as a single event, disentangling these signals from everything else. Thus, we are neither able to combine data from multiple entries or multiple clicks on website suggestions, nor to link this information with personal information like your email address or an IP address, either.

> Query logging data is used to further improve the Cliqz backend. More specifically:

> To be able to suggest websites in real-time while you are typing into Cliqz’ combined browser-and-search-bar, Cliqz sends your keystrokes to our servers. With every new keystroke, our backend scans our index and predicts the most relevant results for your search query.

> “Relevant” to that regard is (very simplified) defined by the frequency a given website is clicked on for a given query. In other words, Cliqz predicts the most probable site you will navigate to, based on the (partial) query that you type. In order to further improve this mechanism of relevancy, Cliqz logs the clicks in its drop down menu and the respective queries.

I wish the terms were more clear about the fact that crucial information is indeed sent directly to Cliqz and is not sent via the FoxyProxy route.

Man, you really got them here haha.

Thanks for taking the time to dig into things. Comments like these are why I continue to regularly read Hacker News.

Hi, Disclaimer: I work for Cliqz.

Thank you the questions, we are always looking for constructive feedback on and off HN.

1. These random values are used for grouping partial queries together, and they reset when you press enter or start a new query. Source code on how it's generated: https://github.com/cliqz-oss/browser-core/blob/master/module... We actually take one additional precaution of using crypto random and not plain Math.random(), which could potentially be used to link multiple sessions together.

2. There is no feature to disable auto-suggest. But I will pass your feedback to the team.

3. No there is no shared ownership, we don't have access to their servers. We also do additional encryption with bucketing on the payload sizes that we route through foxyProxy, so that the proxy provider cannot learn anything about the content of the message. We will have a blogpost explaining this on Wednesday - 4th December. Also, we are looking to add an option, where user can choose their own proxy provider too.

4. There are two parts: a. You can select the option from Control Center (Q menu) icon in the toolbar -> search -> search via proxy. (Now the calls should go through FoxyProxy) b. All calls to api.cliqz.com go through proxy when in private mode. The only reason it's not default is latency. As to what goes through FoxyProxy by default is: all Human Web data.

Once again, we appreciate you looking into details, and please keep digging, we would be happy to answer, improve our documentation and if there are bugs specially related to privacy and security they are on our uttermost priority.

IP Addresses are generally pretty sticky and make a great unique id for users. Even without uids being sent in the request, keying on ip address should eliminate a ton of anonymity.

" they sent every keystroke I type in the URL bar to their own servers"

Well, that's how you implement an autocomplete feature in the 1st place. If they'd sent every keystroke you typed outside their URL bar, now that's something to be aware off. So, did they?

Sorry you feel that way. In the Cliqz browser you search as you type. To search and serve results we need to know the query. We will have an article on exactly this product in a week or so.

We will also be sharing more details on the process on data collection in the blog posts scheduled in the next 3 days. Of course, you don't need to trust what we say, all the code used for collecting data is open-source for transparency and auditing: https://github.com/cliqz-oss/browser-core

Lastly, privacy policies are legally binding documents, and we take the law very seriously. We are located in Germany, where privacy laws are as tough as they get (e.g. GDPR was loosely just a re-wording of the existing data protection directive in Germany).

> I looked into the Cliqz browser a few years a ago. Back then they claimed to be a privacy respecting alternative to Chrome. What I found back then is that they sent every keystroke I type in the URL bar to their own servers. They outright lied about that in their terms.

That's called typeahead search, and that's how it has always worked since the invention asynchronous HTTP requests.

If you know of someone that sends a trie down to the client containing every possible completion a priori, I'd like know about it.

It is impressive what they built. The results are quite good! The UI shows innovative elements like trackers used on the page.

I see two problems with their approach:

1. The product is not built with the 'grandma test' mindset.

More sliders and widgets is not what your grandma wants in a search engine. This is why building a search engine is hard. You have to guess with very little information what the user wants and get it in front of them at first try, without the user having to tweak anything.

2. Google must not fall because it is a monopoly. If it was to fall it should be because someone built a better product.

Similar to how ICE cars had "monopoly" over transportation and the time for change has thankfully come. Not because monopolies are bad, but because electric cars are so freaking awesome.

Google perfected what the 'best search engine' is to 99% of population. This comes at an expense of really annoying 1% of users but it is the price they are willing to pay. To de-throne Google you really need to cater to broad population with a product that will be better both at capturing intent and delivering and presenting relevant results. This may or not come with a different business model.

The article says:

> One could rightfully counter that Google has a good product. They even offer it for free.

But Google Ads are not free. If Search is the product, then advertisers are the customers, not users.

Yes. Google has a superior product riding on top of a corrupting business model that cannot help but mis align Google's interests with those of the users they are claiming to serve. Google should unbundle the current business model from the technology and also offer clean, unbiased, private search service that is paid by the user on a subscription basis. I would jump on the chance to pay say $10 a month for Google quality search if I knew my search was private and the results were not influenced by outside parties.

Ballpark estimate, how much would you pay for the entirety of your digital life being private? All of it, bar none?

I would probably pay at least $50, in addition to whatever the services I use already cost me, for privacy and freedom from ads or any other manipulation I did not explicitly permit. Note I consider the latter at least as important as privacy.

> Note I consider the latter [manipulation] at least as important as privacy.

I second that. It's somewhat reassuring to know that others realize that as well. Most people just have no idea how marketing and now 'deep learned' algorithms (applied psychology) make puppets out of human beings.

I consider "user feed control" (that users choose what, how, where, when and why things are presented to them) of equal importance to e.g. democracy as far as human freedom is concerned. But the current manipulation is so insiduous that from now to mainstream awareness to generally implementing solutions is a long, long ways away.

> at least $50, in addition to whatever the services I use already cost me

I think this tends close to an upper bound — not many people willing to pay, and even fewer at that level — but certainly one more anecdotal proof that premium services are totally viable for a certain group. The question is 'how big' that group, what's the market for that, but I'd wager it's enough to sustain a few 'premium' businesses (or alternative plans) for most common services (some are harder; search notably).

The product is us.

The search results have gotten noticably worse in the last year.

When a Wikipedia/encyclopedia article is what I'm looking for, why is Google showing articles?

Wikipedia used to be the top result.

SEO is my first guess?

I've noticed things on the spectrum of articles <-> blogspam crowding out the top results in many queries, and that it's substantially more difficult to find forum content discussing related topics to the query. It's a shame, as I think there is often more interesting content on such forums.

Anti-SEO is my first guess, that and tweaking for fears of regulation or bad non-grown-up PR...

Google's algo changes since about panda have been burying good web sites and content while bringing quicker answers and 'sanitized' aka semi-censored results to the top.

Some of this is over reaction to SEO and trying to out do the spammers - but the collateral damage to the results and thus the end users who believe that google brings the truth is hard to calculate.

Combine that with regulation like dmca, right to be forgotten and others.. results are even more censored, and the general population does not know what they are not seeing, as they still trust google to be bringing the truth.

worry about bad PR from various factions - tweak the algorithms.. and you can say for sure that the results the more adolescent google brought a decade ago were often more of what people were looking for.. and the results today are often like cheap irradiated / sanitized snacks, not the full enchilada that was once a G search away.

Much of this started happening when whats his name became that adult in the room and started putting the finger on the scale to change what millions could find, it's gotten more and more censored every update since then, and less transparent about that.

in my biased opinion.. your searches and the results will vary. I still use other engines for different things, and I feel strongly that we need more search engines. Anyone who wants to create a better adult engine, let me know.

it’s useful for google to demote wikipedia in favor of showing more diverse results. wikipedia is very very very easy for you to just search directly before turning to google

Google does seem to have their 'recency bias' setting set to 11. If any of your search terms have been in the news in the last 2 weeks, those articles are all you are going to get.

Direct link for the lazy: https://cliqz.com/en/

>Google must not fall because it is a monopoly. If it was to fall it should be because someone built a better product.

One of the reasons for breaking up monopolies is that they make it harder for newcomers to build something better.

Even if you break up Google or Alphabet, the search division would probably remain a single company. It won't solve anything.

So You get Google Search Inc. Adsense Inc. AdWords Inc, Gmail Inc, I imagine.

This would be just restructuring, Googlers are too smart to get hold back by this and they probably already have an emergency plan if it remotely comes as a possibility.

If Google search is not funded by ads any more, they’ll suddenly have a serious problem.

It will, no worries. What would prevent Google AdWords Inc providing ads to Google Search Inc? It's like any other company signing up for an ad network.

My point is that if your company sells information rather physical products then breaking it up is pointless.

I don't think it would be possible to split off AdWords from Search. Search would either partner with AdWords or roll its own ads and then we'd be back to square one.

When Bell Telephone was split it was cut along geographic lines. That would also not work on a tech company for obvious reasons --- software knows no borders. I don't know of a sensible way to break up tech giants because the efficiencies of scale create a natural winner take all market.

So, as part of the deal, you prohibit the companies from engaging in the businesses that you broke up. So search, inc can't expand to ads, and adwords can't expand to internet search.

Search, inc is still going to want ads, presumably, so they'll contract with someone. Setup rules for the contract. Maybe require at least N ad providers with each getting a minimum of Y% of pageviews, and contract terms have to be FRAND.

Strongly restrict personal information passing between the companies.

Or, i guess you could go all Bell on them and divide the US into different territories and have Pacific Google and Southewestern Google and what not. Would be kind of weird to geofence search and ads though.

Google Search is aleady geofenced. Wven if you switch search language, it will still apply filters depending on the geolocation of the request IP. So splitting Google along geographic boundaries wouldn't change much on the technical side of things at all.

However Bell was broken up in a consent decree that didn't have to do with its monopoly over the networks, not explicitly at least.

Bell was broken up because they owned Western Electric and used it to vertically integrate the telco stack for the entire country.

A hypothetical breakup of Alphabet could mirror the breakup of the Bell system, as Alphabet controls the data collection -> advertisement stack. Spin off Search + AdWords as an independent business, while services targeting data collection services that feed it (Chrome/Chromium, Android, GSuite, Google Home, etc).

Personally I could see a lot of consumer benefits.

AdSense and AdWords are two parts of the same product.

I know, just an example to imagine breaking Google up in a really stupid way.

I don't think that if Google vanished tomorrow, a new crop of superior search engines would suddenly pop out of the ether to replace it.

Is Google somehow suppressing the creation of a good search engine? No, on the contrary it has created the best web search engine we've ever seen. I would personally be sad if you took it away from me, and I suspect that 99% or more of its users would be in the same boat (don't judge the zeitgeist by web forum echo chambers). You break up monopolies when they are harming users, not in order to cause harm to users.

Google's search engine is a unique portal into the WWW in that it heavily controls what parts of the web people get to see. Opinions can be shaped, facts can be suppressed, lifes can be ruined by the decisions that go into search result prioritizing.

With the world being what it is today, Google Search is in an extremely powerful position. We depend a lot on information that we find on the web. And these free form search queries are the best UI we have at the moment to retrieve it.

It is a good thing that Google Search is good and is getting better, but that doesn't take away from the fact that they wield a lot of power that needs to be checked somehow.

You are wrong. When a product makes the world a worse place—say fridges that deplete the ozone layer—the right thing to do is to ban the product.

In the case of petroleum fueled cars, they are significantly contributing to a massive climate emergency and should be banned regardless of the existence of a superior product. In the case of the Google search engine... Well in the case of the Google company, it should be forced to split up because it is a massive monopoly that hinders competitors from entering any of their dominated markets, and they use their domination of one market to increase their dominance of another.

And this is should be done regardless of the existence of superior products. Goggle as a single company is making the world a worse place, and the right thing to do in that scenario is to split it up.

That's such a vague standard - every single big company has people claiming they make the world a worse place, are we going to break up nestle, every oil company, every agribusiness and pharmaceutical and car company along with google? That could make the world a much worse place

Would it? Big food producers like Nestle buy ingredients globally, ship them in masses between comtinents to produce highly engineered products. Would a company that is smaller do all of the same things? Would less energy be wasted on production in those factorie? Would the products have fewer additives, etc...?

Would smaller farms with smaller fields and a larger variety of crops be better for the environment? Would those farmers grow more organic food?

I don't think that this is black and white. But smaller companies would likely be a whiter shade of gray in some cases.

In some cases highly engineered products are not ideal for human health.

But highly engineered products are often ideal for:

1. Having a longer shelf life. (Ideal for countries without the same standards for food preservation) 2. Having yields large enough to feed populations. 3. Increased ability for transport.

I agree that there are gray areas, but it's these qualities that help feed a world.

This seems like a pretty blatant advertisement for their company's "Cliqz Browser".

You should at least acknowledge your bias by being transparent enough to show you have a vested interest. It's really disingenuous otherwise, and makes it hard to take what you have to say at face value.

It seems more like an advertisement/justification/explanation for their search engine?

Yes, according to the post, the "primary benefit" of using Cliqz is so that you can get access to their search engine, which is exclusive to their browser.

That is factually incorrect: https://beta.cliqz.com

You're right, instead of saying exclusive feature I should have stuck with 'primary'. I can see how those two words have different enough meanings to cause confusion.

Though it's important to remember that not everyone on here has English as their first language, and we should generally be considerate of that when reading people's comments.

If anything it is likely that it will be the other way around in the near future.

There needs to be a new search engine focused on the niche of people who hate the current ad spam/SEO results that Google currently returns. A query like "best headphones" returns results like "The Best Headphones for 2019 | PCMag.com".

They've clearly optimized for the mass user base, since the average person is probably fine with a result like that. But I'm willing to bet no one on HN would ever click a link like that, just because of how "spammy" it feels.

When I search for something like "best headphones", my ideal results would be forum/discussion posts (eg. like "Ask HN") where I can read what other programmers/hackers are using, their experience with it, etc. And that's the problem with trying to make a one-size-fits-all search engine - it isn't possible to make everyone happy.

I've actually been working on a product to solve this exact problem. The best comparison would be the old "discussions" filter that Google used to have until they removed it. I'd love to show more people and get feedback. If this is something you'd be interested in trying, drop your email here ( https://degoogle.typeform.com/to/QzVy7c ) and I'll send you the beta

Just type reddit after every query and you've got the same function

As long as you're ok with only searching reddit...

what else is there to search, honestly? 1000 headphone affiliate sites? "honest" reviews sponsored by sennheiser? Reddit at least provides generally-ungamed reactions and high density of opinions. Adding "reddit" is the default for any search that matters for me nowadays. I was just searching "game payment gateway" , reddit had the most informative comments.

Reddit is a great option (and this is exactly what I do for a lot of searches as well) but its only one of many forums. If you're a programmer searching for headphones, results from HN would be great. If you're a designer, then results from DesignerNews. If you're using them for travel, then you probably want /r/travel.

Reddit is a great option, but there's a TON of hidden forums out there that are gold mines for information, which is something I've been also keeping in mind while building this product.

hmm maybe you re right but google no longer shows results from the tail end of the distribution. adding "reddit" nudges him to try harder

Reddit is frequently astroturfed.

actually there are tons of niche sites, which have more active discussions than reddit, specifically for headphones one is head-fi.org.

I don't even mind "best widget" searches coming up with blogspam and the odd forum post.

What bothers me is searching for a specific product and having Google promote a competing product to the top search result (or as a "featured" listing that looks like a normal search result).

The discussions button would be 90% of my search if was available. Maybe a filter for popular boards like reddit or hn would be a game changer.

Yep, same for me. Did you put your email in the link above? I'd love to chat more about this and show you what I've been working on

I totally agree with the sentiment. However I'm afraid the day "Ask HN" comes as first results in Google, is the day HN will be flooded with spammers recommending their products (like in amazon reviews).

I feel the force of small sites like HN is precisely to be in the shadow and too small to interest spammers.

Some feedbacks about the product.

I made some typical searches I do in my job. They returned reasonable results. Impossible to know if they are better or worse than Google. I need some days of usage. I'll do my best to keep using it this week.

Maybe autodecting the language would help. It gave me a German page without any obvious way to change to English.

I went to the settings, changed the language and discovered that it wants to know my country. I left Germany because it looks like profiling and I don't want to help them at it.

Then I disabled every feature (news, weather, etc). I'm interested into a search engine, not into a portal from the 90s.

However I'm afraid that Cookies Autodelete and other privacy extensions will delete those settings and I'll have to do it again. I'll probably hide them with uBlock Origin. For the language a URL ending with something like ?lang=en would be great for bookmarks.

And finally, do we need more independent search engines? Yes, definitely.

Many thanks for the feedback. We'll add the language autodetection for the interface and results this week. There is no profiling taking place. The country is just to select what index you would like results from - we'll try to make the UI better.

This is still beta, so please keep on using it and we'd love to hear more feedback [beta@cliqz.com].

I second the need to do some form of detection for language. My browser tells webpages I want results in english (Accept-language header), and this can be safely used to detect what is most appropriate initially.

If not that, then at least IP, considering I’m sitting in the US.

Yep, the The Accept-language header seems the most reasonable to go for. Will try to get it out as early as possible.

Please not IP. I hate it when stupid sites insist on switching to Latvian or whatever because that's my current VPN endpoint.

No, please do not use IP addresses to infer UI language preference. Google does that and it a right royal pain in the arse. Every time I visit a different country Blogger's log in page is in a different language ignoring the carefully specified browser settings giving me Polish when I am in Poland, Norwegian when I am in Norway, etc.

Thank you for your feedback. We have just implemented auto language detection for the UI based on your browser language.

beta@cliqz.com bounces mail. It seems it's a closed list and accepts mail only from addresses already included in the list.

You should create a contact form for the search engines. Everything reachable from the Contacts link at the bottom of the page is about other products.

We don't need more search engines like google, we need a new kind of engines that will supersede google/bing/etc and break the following axioms:

* search engine is a cloud service, which is not controlled by user of that engine

* search gives final results in milliseconds (why? because google cannot spend many seconds/minutes for you, that's why)

* search creates information bubble (that user cannot control) because it tires to satisfy user's expectations

Most likely this new 'google-killer' will be:

* open source, because user should trust the code

* self-hosted (easily deployed in a click), not a cloud SaaS. Our search preferences have ultimate value, no one should have an access to it

* more useful than google because of accumulated data about you that are processed by computational knowledge engine (something like WolframAlpha)

* background reasoning - this engine can work continuously and utilize your own computational resources on notebook/PC and bring you brand new search insights that google never will be able to deliver (because they cannot dedicate a lot of computational resources for each google user).

Sounds good, isn't it?.. Maybe this kind of software already exists, could someone point me out?

Disclaimer: I work at Google so I'm 100% biased. All opinions are my own. Edit: formatting

>search engine is a cloud service, which is not controlled by user of that engine

So is hacker news.

>search gives final results in milliseconds (why? because google cannot spend many seconds/minutes for you, that's why)

Do you want latency to be larger??

>open source, because user should trust the code

That would make it easier for SEOs to game the system. Or perhaps it would put everyone on a level playing field. I'm not sure.

>self-hosted (easily deployed in a click), not a cloud SaaS.

Do you mean users should keep their own index of the whole web by themselves?

>more useful than google because of accumulated data about you that are processed by computational knowledge engine

First I think you greatly overestimate how useful information about you can be. Second if that worked then it would make they problem you mentioned before (information bubble) worse...

> Do you want latency to be larger??

Instant results are good for sure, however very often few more seconds - in addition to instant results! - is not a problem if late, more carefully processed results can save minutes of my time - for now I spend it for opening the links and scanning the content with my eyes.

The same is about not very often but important searches that may be described as 'research about something', in this case I'm ready to make complex, well detailed query and wait even hours - then back and get well organized and intelligent results.

> Do you mean users should keep their own index of the whole web by themselves?

oh no! Users should keep only their personal data - in wide meaning, this includes all history of searches, search results, refinements, anything that ML currently uses to bring personal search experience. In addition to that, relatively small index of important content may be saved. For internet search this 'personal search' will use API of anything that can be used manually for now - google, bing, consume direct API of Twitter/FB/Medium/WolframAlpha and hundreds of connectors to other cloud services. It is important to say, that this 'delegated search calls' may be anonymized.

It will be important that search results are not limited only by what google decided to be 'top results for this user'. At this moment I can do all this manually - open N tabs, query many services, compare results, open most 'relevant' (from my human point of view, not google) links and scan them for most interesting information. I believe that all this can be automated.

> First I think you greatly overestimate how useful information about you can be. Second if that worked then it would make they problem you mentioned before (information bubble) worse...

As for now, all this just thoughts. I'm a programmer with almost 20YOE; I have understanding about how google works in general, how lucene works, how WolframAlpha works, modern approaches to NLP and search-driven queries processing, and I think - without a MVP that works, this is more belief, of course - that value of this 'personal computation engine' combined with modern ML approaches might be ultimate. Challenge, but nothing impossible!

I'm more than happy with DuckDuckGo. I would love more specialized search engines. Back in the day I've used several search engines depending the task i needed to perform, from Altavista to Yahoo to Google.

Beat me to it! Very happy here as well. I also love the bang short cuts. I am actually at the point where google is clearly inferior.

I will grant that Google has a more intelligent indexing and ranking of Stack Overflow. However, DDG is making major progress there, and I rarely need to add !g

Pro tip: with the bang shortcuts, you can add them anywhere in your query, it does not need to be in the beginning of your query string.

I’ve tried switching to ddg a number of times in the past. Every time the same thing happens:

“This result isn’t that great. I’m going to !g just this once”

Then, one week later I’m adding !g to literally every singe search so I switch back to google because what’s the point?

I really do hope to one day get off of google though.

For me it helped to switch over my mobile browser in addition to desktop. It's much more of a hassle to type !g so I try to make do with ddg and only use !g if I really have to. After a month or so it seems like my choice of search queries has adapted to better make use of Bing's algorithms, and when I do find myself trying !g, half the time it results in even worse results.

I use StartPage.com which is based on Google but strips the private data.

"Startpage is now owned by an advertising company" : https://news.ycombinator.com/item?id=21371577

Bangs are nice in DDG. In Cliqz, there is also limited support, but you have to put it at the end ("test !g"): https://beta.cliqz.com/

I use DDG but I still have to resort to Google for 5% of searches because it dosen't deliver the results as well. I desperately want to pay for something that has the quality of Google but the privacy of DDG.

There's the !sp on DDG which gives you Google results without all the profiling using StartPage.com

Isn't DDG just white labeling search results from Bing?

They largely are. Here's an interesting thread, which I and people from DDG partcipated in a few days ago: https://news.ycombinator.com/item?id=21653476

This new search engine doesn't even function without javascript enabled, unlike DDG which works just fine.

We're trying to get a version that works with javascript disabled (for TOR), but are trying to avoid having to maintain two separate clients. We'll try to push it out soon. Many thanks for the feedback on our beta.

We have just released a new version which allows you to use the SERP with Javascript disabled. Enjoy!

I try to use DDG as my default but it's so awful at knowledge-graph questions. I find myself using !g for knowledge queries before even attempting it.

An example would be something like "Rick and Morty episodes." I know google will give me a list with recent/upcoming episode names and air dates for pretty much any show. DDG will link me wikipedia and fan wikis. I make a "<show> episodes" query anytime I want to know when the next episode of something is released.

DDG's knowledge graph (and/or query parsing) is just so limited I skip it anytime I think google will be able to produce the answer directly. Similarly, there are things I'm confident asking a voice assistant, and there are things I won't even bother trying. If it's something I'd ask an assistant, I'm skipping DDG.

How do you design regulation that holds the giants accountable, doesn't impair smaller companies' ability to enter/innovate in the space, and isn't quickly corruptible by the (well-resourced) companies you are trying to control?

We're better served by breaking them up and enforcing and strengthening the pro-competition laws on the books. The government has a pretty mixed record with regulating industry, and a far more impressive record of investigation, enforcement, and prosecution - corporate break-ups are far more in the wheelhouse. Regulation requires constant vigilance, whereas a breakup is self-executing once the case is won.

It's not anti-success rhetoric, so much as an acknowledgement that tech has become increasingly concentrated and anti-democratic. The Sherman Act was passed in 1890 - these aren't "knee jerk" solutions - they've worked effectively in the past to curb corporate abuses.

My biggest problem is: How do you break up a software system? I understand how you break up a company that mines ore, produces steel, builds railways, runs transportation - those are all separable ventures that can negotiate contracts almost as easily as when they had a single owner.

But the tech giants run a single system: Amazon in the delivery, FB in the social graph (DB), Google in the knowledge graph. Those would be incredibly hard technically to break up. The less technically complicated split would mean that one new company has all the revenue and the other has all the costs.

I haven’t heard any credible proposal on how this “breakup” would actually physically work. If anyone has links to good sources I’d be interested.

Easy, stop letting them dump other services for free on the backs of their other businesses. Search can stay ad-supported. Youtube, Gmail, Drive, Cloud, Stadia, Android, News? Should all be broken out into their own businesses and forced to be profitable on their own. As it is, nobody can enter those spaces without being willing to lose hundreds of millions (billions?) of dollars.

Prevent a global company from giving away products? So other big company can gain space?

All of the virtuals you mentioned have different stories. Cloud is profitable, and is not the top company anyhow. News is not a product it's a grouping of news stories. Youtube makes money through ads and better access and could be considered a loss-leader but shutting it down will not make the field more competitive. Android is open source.. and perhaps could be seen as dumping to prevent others. Gmail is a mail service, others exist.. and starting a new company will not cost you billions unless you plan on serving billions of people. Drive is one of many companies that didn't cost a billion to start but might be worth it now.. try dropbox or box.com or rapidgator.

Yes, global companies should not be able to give away products or operate services at absurdly unprofitable rates (see: Ridesharing, a myriad of other Google services) just because they have billions to burn. It's horrifically anti-competitive and raises the barrier of entry unnecessarily. It's not like vertical integration is some new invention, its historically been fought against by anti-trust regulators. We just live in a world where they effectively don't operate any more.

Split the search from Adsense. Allow the search company to only sell ads on its own properties so they can’t reestablish ad-hegemony. For good measure, ban them from making their own browser.

Say Alphabet and its myriad of ad-supported services, such as Youtube. Instead of serving first-party ads, they would need to serve third-party ads, or charge their users, or a combination of the two (e.g. freemium model).

The facebook conglomerate could similarly be broken into Instagram, Facebook, etc. Each would get a portion of their current ad business unit.

In terms of software, each baby Facebook could get a flexible royalty-free license to all the software currently owned by Facebook, and they would be free to derive from it to differentiate over time.

If there is a will, there is a way.

They should break Alphabet up into 26 different companies, each getting a different letter.

This is the best logical conclusion so far

What does it mean to serve first party ads? The ads are (mostly) for third party products/companies. Newspapers also run ads for third party products. The only difference is that the ads/sales department at the NY Times is substantially less complex (and lucrative) than the same at Google.

Just to continue picking on Google:

Google has:

- Android OS

- Nest

- Search

- Advertising

- Chromebooks

- Pixel

- YouTube

- Google Cloud

There are clear lines where they could be broken up.

I’m not saying they should be broken up.

https://medium.com/@teamwarren/heres-how-we-can-break-up-big...

Fair point - there are certain cases where software platforms are too valuable and complex to break up, and should be regulated like utilities with rules for fair play. Amazon shouldn't be able to use it's marketplace data to monopolize entire categories, Google Search needs to be a public platform, with regulated rates for API access (which Google itself will have to abide by).

You don't have to break up many software systems to prevent the worst abuses. Facebook can keep the knowledge graph, but they can't keep WhatsApp or Instagram. Amazon can keep their store, but they can't own a FedEx competitor or AWS - those need to be broken off. Google can keep Search, but Ads, Doubleclick, Analytics, Waze, GCP, Google Home - all of that needs to be broken up.

Some of the resulting businesses may have to update their model or may not remain profitable, but historically break-ups have resulted in an increase in value for the new businesses.

>Google can keep Search, but Ads, Doubleclick, Analytics, Waze, GCP, Google Home - all of that needs to be broken up.

>Some of the resulting businesses may have to update their model or may not remain profitable, but historically break-ups have resulted in an increase in value for the new businesses.

All of this teaches one lesson: be more like Apple. Treat your customers poorly and massively overcharge them. Make sure you are never the biggest in your field. Essentially, don't offer products that are too good for the price.

I'd also like to point out that if those companies have to be spun off on their own, then the only way for some of them to make money is to sell your data to third parties. Right now Google doesn't seem to do that, but how else would Analytics monetize itself, if it cannot offer ads?

> All of this teaches one lesson: be more like Apple. Treat your customers poorly and massively overcharge them. Make sure you are never the biggest in your field. Essentially, don't offer products that are too good for the price.

Or just don't abuse your customers by hiding the ball on what you do with their data. Don't buy up all your competitors, so you can squeeze more and more money out of small businesses who have to buy impressions because the organic ways (which the same companies own) don't work any more. The products aren't too good for the price - we don't even fully know what the price is.

> I'd also like to point out that if those companies have to be spun off on their own, then the only way for some of them to make money is to sell your data to third parties. Right now Google doesn't seem to do that, but how else would Analytics monetize itself, if it cannot offer ads?

The same way Mixpanel, KissMetrics, Adobe and dozens of other web analytics vendors monetize (including Google's own 360 Suite). Or make it clear, up front, how the data will be used on free accounts, and give the user a way to monitor and revoke that agreement at any time. As an independent business, they will be free to find the best path.

I think generally that Apple treats its customers better than Google treats its users.

> how else would Analytics monetize itself

Charge money for the product. Plenty of other companies in the space do so successfully. There's ample business value provided by good analytics.

Google Analytics was originally the product of Google's acquisition of Urchin Software. At the time, Urchin's self-hosted version cost $895 with additional optional "modules" that cost up to $3,995 extra. Their "on demand" version cost $199 per month.

You’re wrong on both counts: Apple has its own share of potentially monopolistic behavior, and there are billions of people worldwide who disagree with you on how the company treats them.

So Apple is a “monopoly” over what exactly?

The entire ecosystem running on iOS/OS X. This is exclusive on Apple produced hardware, which effectively forms a single relevant market as you cannot easily substitute this with others even if there's a small but significant and non-transitory increase in price.

https://en.wikipedia.org/wiki/Small_but_significant_and_non-...

So is that how you define monopoly now? Does that make all of the console makers monopolies over their platforms? Does that mean that Apple also has a “monopoly on voice assistants that can be used with AirPods”?

Does it also have a monopoly on apps that can be run on the “HomePods”? Does every smart TV manufacturer with their own OS also have a “monopoly” on their ecosystem?

Let’s go further down the rabbit hole. Does Tesla have a “monopoly” on software that can run on its cars?

If non insignificant switching costs defines a “monopoly” every software as a service app would have a “monopoly”.

Unfortunately, neither the EU or the US has ever defined “monopoly” like HN posters...

This is not how I defined a monopoly, but a well established practice done by DoJ of the US. Defining a relevant market is not that clear and simple as you described, but it's more of an economical, data-driven process. Look up for hypothetical monopolist test.

I just explained one possible scenario how this could be defined as a relevant market and you're now throwing a bunch of random assumptions on my comment. I would read the link thoroughly (it explicitly mentions DoJ?) and study more on the history of the antitrust law and its applications before doing such.

So if it “well defined” by the DOJ, you should be able to find a specific example where a minority vertically integrated platform vendor was declared a “monopoly” and specific remediation steps were enforced. Not a Wikipedia article.

If that were the case, every single console maker since the mid 80s would be declared a monopoly. Why hasn’t that happened?

> you should be able to find a specific example where a minority vertically integrated platform vendor was declared a “monopoly” and specific remediation steps were enforced.

I already told you that the market defining process is a very case-specific one and big techs now are unprecedented. Why are you trying to find applicable prior arts on such cases? You asked how Apple can be monopoly on a market and I gave you one possibility which may or may not materialize due to its uncertain and complex nature. It's pretty hard to understand why you're being so defensive on this issue?

> If that were the case, every single console maker since the mid 80s would be declared a monopoly. Why hasn’t that happened?

There has been multiple antitrust lawsuits and investigations on Sony, Nintendo and Microsoft for their gaming consoles while their exercise of monopolistic power was nowhere comparable to Apple's nowadays. Why don't you google just 2~3 words before making such a false claim?

I already told you that the market defining process is a very case-specific one and big techs now are unprecedented. Why are you trying to find applicable prior arts on such cases?

Maybe because I don’t believe that people can randomly make up definitions instead of citing precedent?

Well seeing that Nintendo specifically has been very strict about what was allowed on its platform, forced third parties to use its manufacturing facilities since the 80s, forced all software whether distributed physically or virtually to be licensed and to pay a fee and had a much larger marketshare, where was the government intervention? Where were the consent decrees? Lawsuits?

Please provide one citation where any of the console manufacturers were ever forced to change their business practices?

If my claim is “false”, you should easily be able to find a citation.

> Maybe because I don’t believe that people can randomly make up definitions instead of citing precedent?

While you're trying to frame my argument as "make up definitions", the reality is not; this is a standard practice since 1982. Spend your time on searching and studying the topic, not mine. This is an area of vast complexities and I don't think it's effective to spend my time to enlighten you.

https://www.justice.gov/atr/operationalizing-hypothetical-mo...

> where was the government intervention? Where were the consent decrees? Lawsuits?

Your ignorant in the topic doesn't necessarily mean an actual lack of a prior. In Nintendo v. Atari case, there was not much arguments on the market definition, but its practice was illegal or not. Yes, I've been talking only about the market definition and you're intentionally conflating the concept of the relevant market definition and antitrust violation. Don't do that. And please don't even try to say "come up with evidence". You can spend your time on studying this.

The person who makes the argument has a responsibility to show citations.

And your citation were about hypothetical arguments, and went to show more of my point. All throughout the article it speaks about the government’s arguments being “defective” and still doesn’t show a single example where a vertically integrated minority player was called a “monopolist” nor where government imposed remedies or sanctions were implied.

In fact, Atari vs. Nintendo affirmed that Atari did in fact violate Nintendo’s copyright when it tried to circumvent Nintendo’s control over its platform.

So in fact, you still haven’t come up with a single precedent where Apple could be considered a “monopolist” on its own platform or where they are violating “antitrust”.

>Apple has its own share of potentially monopolistic behavior

But I rarely see Apple being included in these calls for breaking up the tech companies. Also, I'd be surprised if Apple even has a billion customers, let alone a billion who are happy with Apple.

> How do you break up a software system?

Turn the relevant parts of it (the ones which are affected by network effects, or other barriers to entry) into an open platform that can be accessed on an equal basis. That's quite a bit easier than breaking up a mining company. For example, Android is already "broken up" from this POV, since the likes of Amazon can use AOSP to create their own Google-free platform, and applications written to run on AOSP will work on either version.

What's the relevant part of building a search engine that's affected by network effects?

Is it the index itself? There's already a fairly decent open-source equivalent to that in the form of Common Crawl, which has been out since 2011. People (including me) have tried to build search engines off of it, but it never seems to work quite as well as Google.

Is it the computing infrastructure? That's already been commoditized and offered as a service by multiple providers - AWS, Azure, GCP, SoftLayer, etc.

Is it the serving infrastructure? Commoditized by ElasticSearch, which uses many of the same techniques as Mustang and in many ways does it better.

Is it the ranking algorithm? It used to be that Google would agree with you. However, the ranking algorithm changes basically continuously - the one in use now is very different from the one that was used when I left in 2014, which was different from the one where I learned how it worked in 2011. I've heard the new one is heavily machine-learning based: given a suitable training set and some learning-to-rank papers you could construct something similar.

Is it the log & clickthrough data? Imagine the privacy advocate conniptions if that were open-sourced. AOL got in huge trouble when they open-sourced their click-logs circa 2002.

Is it the evaluation system? Mechanical Turk exists, and Google's rater guidelines are public.

I'd argue that the real competitive advantage of Google now is the brand and associated consumer habits, and it's really hard to break up a brand. Same reason Coca-Cola remains dominant 150 years after they started selling cocaine-laced sugar water. This is a recurrent problem in the economy today - brands fuel not just Google, but also other giant monopolies like Coke, Nike, J&J, P&G, DeBeers, McDonalds, Wells Fargo, and so on, and in many cases the companies that own them get to practice some exceptionally bad behavior. But short of reaching into each consumer's head and getting them to consider each purchase on purely rational factors, I don't see how to fix this.

"Search" isn't a problem. Monetization ecosystem is.

> Is it the log & clickthrough data?

It's a huge advantage and critical ingredient for ranking in all major websearch engines as far as I know. Google gets billions of queries and clicks every day. Your startup? Pretty much none of that. How are you going to train your ranking algorithm with no data?

> Imagine the privacy advocate conniptions if that were open-sourced. AOL got in huge trouble when they open-sourced their click-logs circa 2002.

Right, logs with such level of detail as individual user sessions would never get public for this reason, too much legal risk. But even simple aggregated datasets of the form "how many clicks did this query-url pair get" would be very useful to bootstrap a competitor and can be anonymized much more effectively.

>Turn the relevant parts of it (the ones which are affected by network effects, or other barriers to entry) into an open platform that can be accessed on an equal basis.

Doesn't this run counter to private property though? Or what would you have happen when Google says "no"?

As many people point out, breaking up is a reductive measure. The whole legal and technical process of breaking Google up would be 10 years? Maybe longer? I heard some tech journalists give those numbers; don't quote me on that.

If you really want to change the landscape and make it actually feasible, force companies to expose APIs or impose standards on them. Both of these are very hard things to do in practice but at least they give a hope of a better future - increasing competition.

Breaking up big tech, depending on the specifics, will cause either worse productivity or (in the best case) change nothing.

Amazon is a deliverer, FB is a social graph, Google is a knowledge graph. Don't think of the companies nor their roles (such as they are) as immutable. Look at GM in the 50-60s, or Paramount Studios in the 40s.

>My biggest problem is: How do you break up a software system?

There are people who have created entire careers about exactly that: antitrust remedies. I defer to experts where I can.

We entrepreneurial wannabes on HN are only going to provide unrealistic answers that become battles of will for the rest of this Sunday before all is forgotten overnight.

Could you fork it instead? Make a copy of the whole thing and then let the two compete over time.

Counterargument: you can't copy the people who know how it works.

What is the corporate abuse here that necessitates a breakup?

Google/Facebook/Amazon have been acting anti-competitively for quite a while. Vertical integration, concentrating industry through M&A, shady data practices.

A democratic country has an interest in ensuring that our labor and goods markets also remain open, competitive, and democratic in nature. We've made this choice as a country repeatedly throughout our history, and it's time to do it again.

Many companies across many industries vertically integrate. Should they all be broken up?

“Shady data practices” is hand-wavey and non-specific. What do you mean? Many companies today (again, across All industries) have had issues with keeping user data secure, selling it to untrustworthy third parties, not giving users transparency or controls in what information is shared, etc. Google’s track record in these areas is far better than most. How does this necessitate a breakup?

Concentrating industry through M&A is a legitimate issue, in my opinion, but it’s also probably the easiest to regulate.

Our tech companies are the most competitive in the world, bar none. I’m not sure that cutting them off at the knees will help with that. ”We did it before,” isn’t a convincing argument that we should do it now, under much different circumstances.

To a substantial degree, yes. Vertical integration reduces the dynamism of the market.

I used a cover-all term, because there are too many to really name. Facebook is a co-conspirator in defrauding American elections, Amazon uses their data to kill small businesses, Google has been repeatedly fined in the EU for using search anti-competitively. Those are just 3 tech companies, and doesn't get into how data is used abusively by credit agencies, banks, and other financial institutions.

Regulating industry is a fools errand, and part of how we wound up here. Breakups are the only self-executing, corruption-resistant solution.

Are our tech companies the most competitive at in the world? The largest companies have been cutting off our startups at the knees for a decade, so we have no idea how competitive we could actually be. It doesn't really seem like our big tech companies have to compete much at all these days, actually.

If regulation can affectively demonetize consumer data it would be a huge move in the right direction.

This comment was originally a reply to https://news.ycombinator.com/item?id=21676505.

> breaking them up and enforcing and strengthening the pro-competition laws on the books.

So, suppose Google Search is broken away from alphabet. What changed?

I can understand that argument about Youtube (which is losing money and would probably fail), but google search will keep being dominant and their ad revenue won't change significantly.

Sad that such a blatant ad makes it to front page of HN. Not sure how much I trust Google, but I trust these guys even less. The blog is an emotional rant. Not a single example of Google breaking anti trust laws. Translation - instead of competing by making a better product, we want our competitors to be taken out by government.

That Google doesn't break any anti-trust law is missing the point. The post doesn't say Google is illegal. It says even though legal, there are many issues with one source controlling all access to information.

Every week we have posts about new products, often self-posted. If they're upvoted, it's that people find it interesting. Quite ironic writing the blog is an emotional rant (that I would disagree with, I find it quite argumented), while your whole comment is whining that people upvote things that you don't agree with.

I feel the problem most people have when thinking about monopolies is if a monopoly produces a product that people like right now (e.g. they like the features, they like the cost), people don't see the monopoly as a problem and even will even defend it thinking it's great ("I love company X, they make amazing products, they deserve their success!").

The problem with monopolies is when the company decides to e.g. price gouge or gets lazy by not innovating. Companies can only really get away with this and survive when they have a monopoly.

Monopolies also kill competing products and deter other companies from even attempting to enter the market - so you might love the product of a monopoly right now but if the monopoly had never existed an even better product could be available today.

I think people generally love Google's products but it would be good to see more competition. The upfront investment you would need to create a competing search engine is prohibitively high so most companies aren't going to attempt to make their own.

Hi, Marc from Cliqz (and one of the authors of the linked article): We at Cliqz do try and others (like mojeek in the UK or Seznam in Czech) are trying as well. But yes, it has very high costs. We did a lot of (I find smart) technical shortcuts (and will explain how in the next days in our blog), but it is still an investment. The main hurdle is however distribution. As said in the article, Google spends 1/4 of their revenue to block market access and this makes the investment case nearly impossible. It's their strategy to stop competitive innovation. This no company can solve and why e.g., in Europe you do see the 93% market share of Google. This is where this becomes a political topic. In fact since search tends to be a natural monopoly and information is so important this is where politics has to regulate and ensure competition. In an ideal world there would be 4-5 competing search engines in every region ...

search is not close to being a natural monopoly. you are vastly underestimating how strong google’s execution has been. perhaps because you need to convince yourself that you could compete, “if only” ...

One doesn't exclude the other. I admire a lot of what Google has done and agree on a very strong execution. This still doesn't change the fact, that a search engine (1) gets better with every query-click pair and hence favors the market leader, (2) is a highly profitable business, which allows to buy distribution (this is the biggest market entry barrier), and (3) there are many small points that add up (crawler access is one minor but annoying issue: Web-Sites very often see all crawlers except Google and Bing as "bad scrapers"; so small companies need to invest a lot of time to convince them to get access and many will still never allow it what users then rate as bad quality) ... but, yes, Google is doing a lot of things right, they have very good people. Many of them my friends or ex-colleagues. This still doesn't change the fact of the original article: A 93% market share is not great.

I played with the search engine some, and I'm surprised how behind their crawl is for my blog and some blogs I follow. For example, it looks like it last indexed www.jefftk.com in August 2017, compared to Bing and Google which crawled it this month. www.benkuhn.net looks like Summer 2017. pedestrianobservations.com looks like April 2019. apenwarr.ca looks like January 2019. No preview for danluu.com or sideways-view.com.

(Disclosure: I work for Google, not on search)

I don't get the point. Doesn't the world already have search engines. Bing, Yandex, DuckDuckGo. If the argument is these are not popular, then that is surely not Google's fault. If the argument is Google's brand is more entrenched in the minds of consumers, then it applies to all sorts of companies and everything should be broken up, like Coke, since the world needs more sugar water. Search engines, unlike social networks, have much less network effects. Even if your family is using Google, you can use duck duck go. That's not the case if they use iMessage or Facebook. If you aren't using the same platform you might be cut off.

I agree mostly, but there's a little more to it.

Google Search is pretty well embedded into Android. You can use Baidu or Yandex or Bing / DuckDuckGo on an Android phone, but there's substantially more friction. Considering mobile is now more than 50% of search traffic and Android is 80% of devices, that's close to 40% of all searches basically going to Google for free.

Combine that with the fact that Google is embedded into Chrome -- which is used by ~67% of web traffic -- and there's SOME friction to using a different search engine:

You have to set your default to a different search engine, or go directly to that page, rather than just type into your URL bar -- like most people do.

It's not hard to see that Google has a huge advantage.

The 80% assume China, but almost no phone in China has integrations with Google search. I don't mean to invalidate you're overall point though.

Why do you think Google is embedded into Chrome? I use ddg easily with chrome, and as far as I remember, chrome allows you to change your search engine to anything, even a nobody site, as oppposed to Safari, which only allows selection from a fixed list of 4 on my Mac, so I can't use Yandex as my default.

Desktop is dominated by windows, where bing search is embedded in the start menu, from where it is non trivial to remove it, and changing it to something else is not even possible.

How is there more friction on using another search engine on Android. What other platform has an even lower friction in switching search engines?

> chrome allows you to change your search engine to anything

Which they mention a line below your quote.

I am not disputing that, I am just asking how does any other browser make it even easier.

This is an advertisement for the Cliqz search engine. Let's remember that Cliqz is the advertising company that the Mozilla Corporation sent our private history browsing to back in the day:

https://blog.mozilla.org/press-uk/2017/10/06/testing-cliqz-i...

I want to see niche search engines:

- a search engine for blogs

- a search engine for dev questions

- a search engine for shopping

- a search engine for diy

etc.

Im really tired of sites trying to centralize and own this content. The key to decentralizing and empowering individuals to own their content is enabling distribution and discovery. New search engines are essential to the web we want.

> a search engine for shopping

That would be nice especially since hopefully it would mean there would be a search engine I could use and not get shopping results. It seems like anytime I search for something I get shopping results so I have to be more specific and type more which is inconvenient. Wikipedia used to be the first result for many things I search for now it pretty much never is. It's been getting worse every year.

On a pretty unrelated note, it would also be nice if I could search the play app store without getting games in my results. There's pretty much a game related to everything I want to search for. Very annoying.

Anyone remember Technorati when it had a blog search engine? https://en.wikipedia.org/wiki/Technorati I recently noticed one of my old blog posts had Technorati tags in it, and that was a blast from the past. http://consumingexperience.com/2005/02/technorati-tags-intro...

I'm pretty sure that we won't see a significant blog search engine again until blogs regain some of the power they lost to social media.

google will neatly incorporate their results in their index and then people will go back to searching in google. I mean it happens with all major sites (stackoverflow, shopping results, diy/pinterest). Only twitter and facebook , who completely block their content from google get away. Perhaps we should take a lesson or two from them.

Tweets do show up in search though

It's hard to take their "privacy by design" seriously. It is the same Cliqz that partnered before with Mozilla to run "experiment" exfiltrating browsing history (!) of 1% of new installs in Germany: https://blog.mozilla.org/press-uk/2017/10/06/testing-cliqz-i...

I thought Mozilla would loss all their credibility after this, but somehow they still market as privacy-focused..

Hi, Marc from Cliqz. This one haunts us (in HN and also in my dreams). Let's first get one thing clear: It was a terrible blog post. Second: The 1% who had Cliqz installed were actually safer than the ones without. Why? If you use a (most) browsers every keystroke in the URL bar gets send to Google. This is how autosuggest works. This was and is the case with Firefox also. Cliqz in the functionally implemented in Firefox was not that different. Except - it comes with privacy by design: We actually proxied those requests to not get the IP, we take special measures to not collect private data in the first place. And all this was tested and scrutinized a lot before and after the test. And if you don't trust it or believe it, it also happened to be open source. In the end, you were better off with Cliqz than without. Third (and maybe most importantly): If you don't support anyone who "collects" data, even if they do it in the most transparent way, without collecting any private information, in fact going a long way to delete all PII, open-source and with privacy by design, someone who has no business model built on collecting profiles, then you only criticize Google, but will never have an alternative or only alternatives that white label those that do collect all data. Because building a search without data is impossible. We were hoping that together with Firefox we would build a better solution. For some reasons this didn't work out, but lack of privacy was never one of them (lack of a business model more so, because sending every key press to some is more profitable than others, regardless of how private each is). In fact, our way of doing things is much more private than most people out there who claim to be private. This includes most browsers and search engines. And all of this is open source, so verifiable. And while we speak about it: In our blog https://0x65.dev we will over the next 24 days publish pieces on what we do and why we do it and why it is important, data collection and anonymization will be a big part of this. Last not least: All this is clearly not your fault, so my rant might be inappropriate, but as I said, this thing haunts us for the wrong reasons. (Me and the team are happy to answer any questions).

Obviously using search engine sends search queries to it.. okay. But that experiment was sending most of browsing history, without explicit opt-in. Collecting browsing history from users of popular browser without asking them is just evil. (and it seems you still defend that decision)

Yes, I’m defending it, because again: We took drastic steps to never send anything private (like checking within the browser whether the URL is unique or different if logged in or out and then never sending it, not to mention that there of course was no identifier and we made record linkage impossible, so no click profile, and much more). If in doubt we drop and don’t send. And again – there were tons of (pen) tests and scrutiny to make sure no private data point ever leaves your browser. It is built with the mindset “if it reaches our server, we should technically not be able to identify any single person or any surf pattern or any private URL” – this was and is also tested by many (privacy) researchers before and after the experiment. And again, all this was and is open source. This is way more than any industry standard, and I simply don’t know of any company that works with data that has a higher standard. Be our guest to validate it yourself. And please read our blog post Tuesday: we will explain how this is done. But if you simply oppose this (and similar methods from people who really care about privacy), you basically accept the status-quo of the worst data collectors, because no one else then will ever emerge (because you do need this kind of data to build a search).

[EDIT]: Just to clarify and not have anyone create the wrong idea - I defend my earlier point. But your question is loaded. Here's why: We do not collect browser history, which by definition implies being able to piece visited urls back to a profile in our servers. That is impossible - to us, each single URL comes as a detached datapoint - devoid of any information that can be used to aggregate them back to a user profile.

I didn't say it is absolutely inappropriate to collect any user data, it could be ok after obtaining explicit user permission. Firefox experiment installed data collection add-on to random users without opt-in, and I still cannot understand why you thought that was good idea.

>That is impossible - to us, each single URL comes as a detached datapoint

IPs could be used to aggregate those datapoints, and you obviously cannot avoid receiving these. It is only promises that you or your proxy provider doesn't store them. (though maybe it is possible to implement P2P mangling network? encrypt UDP data packet, send to randomly selected peer discovered from DHT, peer delivers it to your server. Or directly send UDP packet with spoofed source address, but this is not possible for browser sitting behind NAT)

There were extensive tests with opt-in before (Testpilot), but these are super biased towards techies/enthusiasts (by definition if you read HN or use Testpilot you’re not representative ...). At some point you need to both test and get data from more mass market and that would never work with opt-in. Hence the scrutiny about not even technically be able to do record linkage etc. And some of the measures you mentioned are/were applied (we post about this in the next days).

I also stick to my original point: those users who had cliqz had significantly more privacy than those without.

Having said that: I don’t think, you and me are that far away from each other. But: If we, who care about privacy constantly criticize or even shout at those who also care about privacy, those who build better products, but maybe don’t follow an idealistic “no data at all paradigm”, then we will always end with the worst data collectors, because non of the alternatives will ever have a chance (or people get frustrated and decide they can make more money at Google or ad tech).

By the way, we have a post about data and how we collect it in our blog today: https://www.0x65.dev/blog/2019-12-02/is-data-collection-evil... - you might find it interesting).

In any case thanks for challenging us. I don’t believe we’re perfect. But we’re trying!

And here are all details how we remove all personal information with a technology we call Human Web: https://0x65.dev/blog/2019-12-03/human-web-collecting-data-i...

We love to get scrutinized and get feedback on this - we’re very serious about privacy.

Does this mean the article is insinuating that DuckDuckGo is dependent on another search engine? Is this true? Or are DDG results organic?

DDG depends very heavily on Bing.

There are prolly only 3 global English language indexes. Google, Bing, Yandex. DDG uses vague language to make themselves look like they are actual search engines, which would make you think they actually index the web.

The truth is they rely largely on Bing and Google results and mostly just rerank them. That's why it's hard to take their PR seriously.

DDG's main selling point is that they respect your privacy. I could not care less where they get their results from. As such, I continue to take their "PR" seriously.

All I'm saying is it's pointless to consider them a realistic competitor of Google, Bing etc. when they basically punt over the most crucial infra aspect of being a search engine and when their service is more akin to querying the Bing API from a headless browser and mildly rearranging the results.

It's also clear that they try to obfuscate this fact by claiming "more than 500 signals / sources", which means they feel that being more upfront about the truth would work against them.

DDG primarily relies on Bing for its search ads.

I wouldn't be surprised if it is the primary source of their search results either.

Developing a new search engine from scratch that can compete with Google is almost impossible unless you are at the scale of Microsoft.

It is much easier to white label an existing search engine so you can have good search results on day one.

Absolutely love this quote from the article:

> Paraphrasing Monty Python: “What has Google ever done for us?”. They have done an awful lot for society, for the web and probably saved it a few times a long time ago. But it is time to also see them as the empire they have become. And empires must fall.

I think people generally do not understand well enough the far-fetched consequences of a monopoly in the search business. We really need more diversity and competition in this space, not just for quality of results and convenience of services; but for society as a whole.

The article rightly points out that Google is a new "Cambridge Analytica" (even on steroids), we should not passively wait for our democracies to break down because information is controlled by a single private entity. Let's step out of our comfort zones, try new products, support the ones we like!

That was an interesting choice of paraphrase. Does the author realize it's part of a gag where the characters go on to list all of the ways that the Roman government had materially improved quality of life for the people under its rule and thereby make their rebellion seem farcical? ;)

Clearly the author does realise this as is evident by the rest of the sentence, which was included in the quote.

That doesn't make sense. In the movie the joke is clearly on Reg, the guy who says the words "What have the Romans ever done for us?". I still think the author didn't really get the joke...

They're German, many Germans are great Python fans. I think you can safely assume they have seen the Life of Brian and analysed it carefully. And yes they got the joke.

> We really need more diversity and competition in this space, not just for quality of results and convenience of services; but for society as a whole.

There are plenty of alternatives. But people don't use them as much because they are worse.

Consider Bing ( http://www.bing.com/ ). It is backed by a company even bigger than Google. MSFT's market cap is 1.155T; Google's is 899B. MSFT is 30% bigger than Google. Why haven't they put more resources on their Bing search engine? Spend a few billion dollars, hire away some of the top search engine experts from Google, and build away!

I have spent a few billion dollars and they were not successful.

Bing is not a core part of Microsoft’s business.

Google receive over 100 Billion USD per year from ads revenue. If spending a few billion dollars can get you a new Google, then every one would have done it a long time ago.

Maybe the solution is not in the free market, but by taking the background of pubmed and create a search.gov.

No ads, no algorithms, no bullshit ranking and metrics, no tracking, just pure boolean search with a heavy handed spam filter and jam packed with features for the sake of utility and not engagement.

I'm sorry, but a heavy handed spam filter is the absolute last thing I want the government to have. This basically gives the government the best censorship tool they ever had.

Disclaimer: I work at Google, all opinions are my own.

>just pure boolean search

So if I just repeat the word Facebook a hundred times in my personal website, I'd rank higher than facebook.com when people search for "facebook"?

I think simple boolean search works for things like searching books in a library, where you can assume good faith from all the authors. But in the web you can't do that. It's just too easy to game.

> Absolutely love this quote from the article

Maybe you should've disclosed that you work at this company?

> I think people generally do not understand well enough the far-fetched consequences of a monopoly in the search business.

'far-fetched' (meaning implausible) should be 'far-reaching', right?

> The article rightly points out that Google is a new "Cambridge Analytica"...

One of the most amusing patterns, both in newspapers and on HN, is how often whenever Facebook does something wrong, the response is "this is why Facebook and Google should be regulated/destroyed/broken up".

How did such blatant advertisement make it to the top of HN? The article quality here has really been going downhill.

Cause people like the headline and talk about that? :D

Google bad.

Nah, two or more search engines all being the product of US big techs won't solve the problem.

EU should have its own search engine. They should subsidize this effort and have Google pay for it.

I've heard that rhetoric somewhere else, and I expect having Google pay for it will go about as well as having Mexico fund a wall across the border with the US.

But the European taxpayer should absolutely pay for it.

That's not really a comparison that makes sense. Mexico has very little leverage on the US. EU has loads of leverage on Google.

We shall see. It has thus far been in Google's best financial interests to make nice in the European markets because of the size of the user base that they would leave on the table if they responded to EU pressure by shutting down services.

I'm unsure if that continues to be true if the EU siphons money off of them to build a direct competitor.

The EU represents close to 50% of potential/projected revenue to Google in the next 10 years, is my guess. That's what I meant when I wrote "loads of leverage".

It's not that simple. In a scenario where google was shut out of the EU you'd have a lot of pissed off citizens, and google knows this.

I don't think it's in the basic values of the European Union to suddenly block Google. We're talking increasing taxation, over a 5-20 year period, that kind of thing. We're not North Korea or the PRC.

Or in some other other gradual way. Anyway: it's not a binary over-night thing based on a whim of some dictator.

Either way, I fully trust the competency of the people running these things; they have proven themselves in the past. I'm writing that as a Swedish citizen who is a little bit skeptic about whole the EU thing. This part though - I trust these EU bureaucrats. My impression is that there's a team of very sharp people working on this.

making a search engine is not that hard or expensive, getting users to switch to it after decades of relying on and trusting google is

I would actually argue that making a good search engine with quality and responsiveness worldwide rivaling Google web search turns out to be an architecturally hard problem.

A basic web crawler is not. A semantic web crawler and associated search engine capable of mapping real human input to useful information and dealing with the heterogeneous structure of data on the web, with low millisecond latency?

Bing has been working on that problem for years now, how come they haven't caught up more with Google?

I agree. Bing seems to have mostly solved the crawling problem by now, but not the query relevancy problem.

But then again, there are also a lot of other problems Microsoft haven't been able to solve...

It's a much more well understood problem now than it was when google was founded and the tools and platforms available to assist are light-years beyond what was available then. Many failed search startups have shown again and again that marginally better results are not the deciding factor in using a search engine, it's what's the installed default and how trusted the brand is.

Making a search engine was not hard 20 years ago. But now the internet is filled with so much more crap that it actually is.

EU can just fine Google and use that money as a foundation to build competitors.

Need not to be more complicated than that. It is not like Google is the same darling it once was at home either. The public now has an appetite to go after US big techs not vice versa.

And Google won't have a valid defense against it, unless the US government comes to play

The EU already puts a lot of money into search R&D exactly because of this.

That is exactly the point of the article, by the way. Cliqz is built in Munich, Europe. And it does not rely on any other US search engine to provide results (contrary to DDG, Qwant and others).

Yes, I'm really looking forward to an inferior, super-censored version of Google (hate speech laws, anti-piracy laws, right to be forgotten laws, etc) paid with my taxes.

I'll save you some future embarrassment and point out that Google complies with all applicable laws in the territories it provides search (including "hate speech laws, anti-piracy laws, right to be forgotten laws").

They should focus on building a search index and then license that to European search engine startups. That's the capital intensive part that requires you to build datacenters next to power plants, etc. and where government money will even out the competitive playing field.

It will also help insofar as site owners will be less inclined to block "EUROPEAN SEARCH INDEX CRAWLER", just as they're less inclined to block google, despite being inclined to block "small time search index".

The creative, non-capital intensive part where having a diverse ecosystem will help the most is building a layer on top of the index to actually find stuff - e.g. a bit like how duckduckgo is built atop bing's index.

There's https://www.qwant.com/ from France. They crawl the web on their own but it's a big job so they currently supplement with data from Bing.

https://money.cnn.com/2018/07/18/technology/google-eu-fine-a...

Google already payed a bit, now EU can begin :D

> There are only very few truly independent search engines in the world. Google, Bing, Yandex, and Baidu are the best known ones. Most other players only white label results from those four and are dependent on them and their business model. Or they are so localized and small that they do not offer a credible alternative. Nevertheless, we at Cliqz are also trying to build one: over the last five years, with around 100 people, we have built our own search engine beta.cliqz.com from scratch. It’s completely independent, it does not use anyone’s technology nor anyone’s business model. It’s built end-to-end from collecting, crawling, and analyzing data up to actually showing results all this with privacy by design. We’re quite proud of it. And each day until this Christmas, we will explain one core piece.

I look forward to reading the rest of the series.

I see no mention of their technology being open source, which makes it a lot less interesting as an alternative, to me.

Their search UI seems OK, once I turned off all of the noisy widgets, including the terrible pretty background image. I wish this trend would go away, but I'm glad they provided an option to turn it off.

Thanks but no thanks.

I don’t trust a browser nor a search engine if an ad imperium paid for it and gives it away for free.

Does the Cliqz browser still disallow plugins, such as ad blockers?

> Does the Cliqz browser still disallow plugins, such as ad blockers?

Hey, I work on Cliqz' adblocker (open-source here: https://github.com/cliqz-oss/adblocker). Cliqz browser comes built-in with adblocking, anti-tracking, anti-phishing and private search built-in so that users are protected by default.

You can also install another one if you prefer since all Firefox extension are also available from Cliqz! But the one built-in is pretty good already[^1] and we're happy to get more feedback about how we can improve it!

Finally, Cliqz search is also available standalone now so feel free to give it a try! https://beta.cliqz.com/

[^1]: https://whotracks.me/blog/adblockers_performance_study.html

So how does cliqz makes money?

Hi, Marc here, I work at Cliqz. We (unfortunately) don’t (at least not a reasonable amount). We have an advertising model, but it does not rely on any personal data. We also always think (and tested) a paid product, which would be the purest and (In my view) best form. Experience unfortunately suggests, that only very few people would pay. But, I would be happy to be convinced otherwise. Going back to the article though: This is exactly why we believe there should be a lot of competition and a variety of different alternatives. And one or the other might figure out a new approach towards financing search.

it doesn’t seem like a crazy idea to pay for a quality privacy focused search product. I’d better pay with actual money than with personal data.

Just an FYI: I wanted to add beta.cliqz.com to my Firefox search engine list to test it a bit over the next few days. Firefox doesn't seem to "recognize" it as a search engine so you cannot add it using the normal procedure that works on all the other search engines.

> I don’t trust a browser nor a search engine if an ad imperium paid for it and gives it away for free.

Yeah, that’s why I don’t trust chrome or google search.

Good point!

Still, I was referring to the Burda conglomerate. Why would we want to replace one ad giant’s tech with that of another (more regional) ad tycoon?

> Thanks but no thanks.

Lol, yeah. The HN headline is like "hell yeah, break up and/or nationalise Google/Alphabet," but the link is "get our product please think of us poor little oppressed advertising agencies" and, uh, no.

> “What has Google ever done for us?”. They have done an awful lot for society, for the web and probably saved it a few times a long time ago. But it is time to also see them as the empire they have become. And empires must fall.

They were definitely involved in several precedent setting legal decisions, such as the settlement that allowed for scanning of printed works.

Wow. You can't even do a search without enabling javascript. Surely no tracking implications there.

Yeah, I'm going to stick with DDG. Questionable ownership aside they at least have a functional website without running javascript.

This reminds me of an old quote by Wayne Gretzky.

“Skate to where the puck is going, not where it has been.”

Google will not be replaced by another search engine. It will be something else. Something we can’t imagine yet.

I agree with this statement, my default reaction would be to say “build one” but you have.

I’ll give Cliqz the old college try, but I hope you make it a priority to get into the next release of MobileSafari as an option if you’re intent with taking on Google and taking them on seriously.

Here’s a data point of one: my default method of “searching the web” is to start with Search on iOS, what you get when you pull down on the home screen. When the results I want aren’t there, and I can usually anticipate when that will happen, I shift it over to Safari which uses whatever my default is, DuckDuckGo but mainly because I can change my search engine on the fly with them.

I also hope you incorporate the same syntax as DuckDuckGo (if you haven’t already) because to be blunt, every search engine I’ve used in the last few years is terrible. I can’t use just one, and I end up doing a lot of fancy site searches. Not even Google today is as good as Google from 2007-2008.

Because you can inject your ad network (https://myoffrz.com/en/fuer-nutzer/) more easily if you have your own search engine, people like Google don't let you use your own ads on their results generally.

I don't disagree with the premise, after all I was part of a team that tried to do the same thing (well, without the ad network) and discovered that the adtech business has become so corrupt (not that it is all corrupt just that enough of it is) that the only way to cover the costs of running the thousand or so machines you need to hold a decent index and cache is hard to achieve.

Curious what you mean by "corrupt" here. I'm genuinely curious as I've never worked at a company in ad-tech before, so it's always fascinating got me to hear from insiders.

Ad networks that push adware or malware that facilitate click fraud. I would guess that anywhere from 2 to 3 billion dollars spent on Internet advertising each year is ending up in the hands of corrupt ad tech vendors.

What would a different search engine do to differentiate it from Google?

If the answer is privacy, that’s a niche already occupied by DDG.

What other ways could a web search engine differentiate itself to be worth customers using more than one?

I used to work for a vertical search engine (comparison shopping). For the most part, no one in that space was able to build a brand, and they all depended on either paid or organic traffic from Google to stay open. As the company I was at and competitors started to merge and die, the largest, Nextag, went on a PR campaign to tell people Google's a Monopoly[1].

These vertical search engines, at at least in the comparison shopping space, were never that good to begin with. Shopping (and possibly travel) is just the easiest vertical to monetize. All the sites were spammy, pages were optimized for clickouts, and results were optimized for yield, not relevancy, because everything was an ad. There's a reason none of them were recognizable brands and they all depended on Google for traffic.

So yes, not just what niche is there, but how can you make it profitable--or even break-even. Would you donate $10 per year to a non-profit search engine?

[1]: https://techcrunch.com/2012/06/08/nextag-ceo-google-is-a-mon...

allow the end user to tweak the search criterion easily. there are times when i dont want the "most relevant" results. i want things like historical order or alphabetic order. or even in order of ISBN number.

i really miss using an image itself for query terms.

I'm already degoogling myself, transferring all my emails to tutanota and using firefox with duckduckgo now.

I've been doing this too, but every time I check my open connections in my router there are a ton of open connections to 1e100.net

Best of luck to these guys. Is there a way that Cliqz (or any other search engine) will manage to break through when even Bing is unable to get significant traction despite Microsoft throwing its weight and its default Edge installations behind it?

Hopefully there will be even more stringent antitrust enforcement to prevent Google from buying favorable search placement everywhere, but Google has such a big edge both in consumer awareness and people adapting their searches to Google that I'm not sure how it can be dethroned.

Cliqz plans to make money by, in the future, "deliver[ing] sponsored offers to users based on their interests and browsing history...processed locally based on a remote repository of offers, with no personally identifiable data sent to remote servers"

This is the reason they're pushing their browser and their extension. Privacy wise, this method sounds OK if they really don't send the data (or what ads you see personally) out. I don't mind seeing ads if they are somewhat relevant, non-obtrusive, and not stalking me across the internet.

My first impression about Cliqz search is that it is somewhat viable, but the index is pretty shallow. There are actually a few other search engines with their own index not mentioned by the article (Google, Bing, Yandex, Baidu):

* Mojeek. Sometimes really good for obscure sites when you want to "grep the internet" but seems very vulnerable to blogspam and other SEO. Maybe I just haven't figured out how to query this one yet.

* Yippy. Pretty decent index. Cool feature: categorizes search results with a tree on the side, so if you search for "cobbler" you can just remove the pie or shoe results depending on what you're looking for.

* Gigablast / private.sh: private.sh is the new Gigablast proxy run by PIA. Shallow index as well, haven't used it as much.

The search engine market will be up for grabs if Google keeps getting worse at the same pace.

What happens to our data when this empire falls?

Ready for the subprime data to be bundled with AAA rated data causing a data industry micro-economic collapse? Because I am!

the same thing that happen to everything else

most it's saved, some is lost forever

It's sold off to the highest bidder. Because it isn't technically our data.

I've built a few different types of search engines over the years.

The first attempt was in 1999, i built a meta search engine using coldfusion.. I didn't have any server to run it on so it was on my dialup running locally. Worked pretty well, i was a big fan of metacrawler at the time.

The next time was back in around 2003.. I had a server running on my adsl connection at the time. I wrote a crawler of sorts that indexed word positions to search by proximity. At the time, Google didn't do this, it was a feature i was looking for. Eventually, the project died. If i had a 200k startup fund like google had, it could have probably turned into something.

I did build it on mysql and perl but it needed more servers.. Something i didn't really understand back then with only about 4 years web experience.

Then, i built another a meta search engine, ran it on a hosted server, gained some limited traction around 2011. Died as the sources for the api dried up or became too costly. It did make some money from advertising at the time, which is a pretty big deal. The coolest thing was the fact that i kept the searches open, each search had an rss feed that you could use to keep track of certain keyword sets.

Something i need to say about Google is this: it seems simple, but there is a lot going on in the background. Its doing a lot of things like checking for malware, spam, link farms, etc. but also its running the page in a real browser. Its pulling in data from google analytics and clickthrough/bounce rates, its always split-testing result sets, showing results based upon geographic location, language recognition, date recognition, etc. They use machine learning with real humans training those models, that's not to mention their internal systems for managing servers.

I think you shouldn't try and copy what they are doing, in the end it doesn't matter if you do.. Look at bing, its kind of like a joke compared to Google.

Facebook did the right thing, that was to bring the community and data onto the site itself. Identity is the future. Reddit is pretty good as well, you can talk about any link on the internet.

User-moderated content is the future, as AI/ML can always fool another AI/ML but not a human.

You only have to look at this site hacker news to tell that.

the problem is, I might have no reason to trust Google but I trust software like Cliqz even less, even though they tell me, that I should trust them (maybe because of this).

Also Cliqz belongs to Hubert Burda Media, a company which I trust even less.

Right. I was really feeling the call to action in the blog post until I tried to visit beta.cliqz.com and realized it was blocked by the filters I'm using in uBlock Origin and by the DNS-level adblocking I have on my network. Hmmm.

A competitive market with high barrier to entry seems to stabilize at 3 entrants.

Others that might seem to be in market are niche players.

Scale is good for consumers, benefit of scale is a thing, so 'trust busting' should happen if there are fewer than three.

In the us search market, we have google, ddg, and Bing.

Wanting more choice is very American, but it's hard to take seriously.

I don't think search really has a high barrier to entry. Certainly not compared to things I usually associate the term with like, say, airplane manufacture or deep sea oil extraction. Duck Duck Go only had three employees when they passed a million visitors, so I can't imagine they required huge amounts of capital.

> I don't think search really has a high barrier to entry.

Others disagree, including Google Chief Economist Hal Varian, who said (in 2008)[1]:

“Scale is pretty critical [because] search technology exhibits increasing returns to scale.”

Or Jonathan Rosenberg, Senior Vice President of Product Management and Marketing: “So more users more information, more information more users, more advertisers more users, more users more advertisers, it’s a beautiful thing."

Or Eric Schmidt: “Scale is key. We just have so much scale in terms of the data we can bring to bear.” “We are a company that operates at scale . . . trust me it is a scale company.” “We think search is about comprehensiveness, freshness, scale and size for what we do. It’s difficult for [Microsoft] to copy that.”

It seems obvious to me that internet search has both economies of scale and network effects, both classic barriers to entry.

[1] http://fairsearch.org/fact-checking-google-scale-is-a-barrie...

So, what's different about cliqz? You promise not to keep logs, skew results, and sell data?

Also,

> With 93% of the search market, Google’s algorithms decide what becomes truth. Can you think of a TV channel with a 93% audience? Would you find it acceptable if there were only one TV channel?

Seems to me like Google is more analogous to the TV remote.

As blog post mentions, more than half of Google searches do not result in an (out-)click, but stay on Google. So, the comparison is not completely outlandish.

The world needs more ways to navigate the internet is what it needs. An intuitive tool that helps me stay on track and/or helps me remember what the hell I did would be preferable.

The mystery-box-that-reads-my-mind-then-poops-out-links model feels increasingly clumsy as the years go by.

Perhaps the .dev tld isn't the best venue for this post.

For those who don't know, .dev is operated by Google.

Search engines are really a relic from back when everyone had their own webpage. Now everyone posts their content on a handful* of sites - Medium, Reddit, StackOverflow. I think what we need is an inversion of the search engine model... instead of a web crawler scouring the web for new pages, these handful of companies (Medium, Reddit, thousands more) should register with the search engines and push sitemaps regularly. This is the only way to level the playing field with Google, since the internet is so vast now that only Google/Microsoft/Yandex can afford the server/bandwidth fees to crawl it.

*- and by "handful" I mean tens of thousands of sites, which is relatively small.

> Now everyone posts their content on a handful of sites - Medium, Reddit, StackOverflow.

They do not and it would be a sad day if they did. Please don’t help push us towards that any further.

They do though. Old content will just gradually disappear into webarchive. Let's not pretend it didnt happen. (oh and btw we will probably change that again, but the content won't go into webpages-hosted-somewhere but probably will be served directly from the user's phne)

Only if it's common knowledge. I've legit started to add "blog" to my searches to find more obscure knowledge.

Don't forget that "it seems that Google is forgetting the old web": https://news.ycombinator.com/item?id=19604135

I know it is, but other search engines seem to be as well.

> Search engines are really a relic from back when everyone had their own webpage. Now everyone posts their content on a handful of sites

but:

> since the internet is so vast now that only Google/Microsoft/Yandex can afford the server/bandwidth fees to crawl it.

Is it a vast space that search engines need to crawl, or are there only a handful of sites that matter?

There are a handful of vast sites that matter.

Hmm, cliqz.com is in the blocklists of several DNS servers I'm using, so are they the underdogs we need to cheer for against Google et al, or are they some shady spammers as determined by the community of adblock list maintainers?

Hi Aorth,

Disclaimer: I work for Cliqz.

We are aware of this and it's unfortunate, although the lists are not the main ones from community like Easylist, uBlock Origin etc. Cliqz does need to collect data to build it's search engine and other features. Even though we do with by privacy-by-design in mind and without collecting any PII (we will be sharing more details on the process on data collection in the blogposts scheduled for Monday, Tuesday and Wednesday),still we do end up getting on such lists.

Of course, you don't need to trust what we say, all the code used for collecting data is open-source for transparency and auditing. Additionally, also happy to help setup network debugging incase you want check what exactly goes out. - https://github.com/cliqz-oss/browser-core

We are also willing to discuss with the maintainers of the list and explaining what and how data collection is done.

Side note:

> In the TV world, this would be the equivalent of 100 different channels, but they all show Fox News 24 hours a day and just replace the logo. This clearly cannot be good.

Unfortunately that's already happening, just look at Sinclair [0]. They operate 193 different TV news stations covering 40% of US households and syndicate very similar content between stations (with the same political biases, just different logos).

[0] https://en.wikipedia.org/wiki/Sinclair_Broadcast_Group

How much of that is syndication (using the same news stories in different local markets) vs whatever you would call having two channels in the same market with the same content, but trying to market themselves as different?

Can I please pay someone to let me customize search? I know what results I don't want, but there's no way for me to bundle a custom filter to exclude them.

93% of a search engine market. Ok.

But a search engine is only a smart part of the whole, where and how we get the information.

If you slice your market in convenient ways, you can make any company a monopoly.

This is extremely true. It's one of the reasons that Google+ came into existence. And with Google+ being folded up, Google has basically refrained from competing further in social indexing.

Google's business is advertising, not search. That's where their monopoly lies and that's where competition is needed. We probably don't need many other search engines. There is only so many improvements to be done in search. Over the years, information has been distilled. Google's main use is as a misspelling corrector, a wikipedia shortcut, a DNS shortcut, a keyword lookup for the news of the day and a product matcher in a few merchant sites. That's because most commercial sectors of the net are already in their post-competitive period, and only the winners in each category are still alive. Their number is not even enough to fit the first page of results. And the content that used to go in blogs, it's now behind twitter or facebook, unreachable to any search engine (except for corporate blogs which are used for SEO reasons). The recent changes in google search have infantilized it, burying anything but the most obvious sites.

SEO doesnt even count that much, instead it's curated "reputation" that matters.We might be at a point where DMOZ is probably viable, because there aren't now 1000 entries in each category, but 10. In fact a Dmoz would probably be a mightier competitor to their search.

But the key is , we need competition in advertising. After http referrer was removed from queries, we left all the information about the user's intent to google, and, unsurprisingly , they ate it all, and then some more, leaving a progressively diminishing piece of the advertising/intent processing pie to the rest of the internet. Is that ethical? Is it OK that a website doesn't know why the user visited it? After all, the user typed it in the browser's bar , not on google. Perhaps search queries should be initiated in the browser, and available to subsequent clicks for processing.

And as for tracking, is it OK that google is the only one who is tracking users? Shouldnt tracking be a user's decision, provided and mediated by the browser to any website who asks for it? Like it or not, advertising is not going away and it's healthier for the net if a large number of publishers share that (growing) pie rather than if google eats all of it. Tracking doesnt kill people, but anti-tracking hysteria causes many people to lose their jobs. Perhaps we should be pragmatic, because being irrationally anti-pragmatic is only serving Google's long-term interests

I think I get what you're saying but I'm not sure this is the right approach. It's accepting diminished rights in exchange for a stable compromise versus fighting for an absolutist solution at the risk of breaking the whole system.

And personally I'd rather have no internet at all than an internet that's a weird murky tracking soup where a hundred different parties are creepily looking over my shoulder, knowing more about me than what I would tell even my closest friends and family.

Feel free to go use Bing. It's been an option for 10 years.

I'm all for more search engines but I'm not a fan of the brand name Cliqz. I instantly think of the bogus/low quality search results on a parked webpage.

Quote: "There are only very few truly independent search engines in the world. Google, Bing, Yandex, and Baidu are the best known ones. Most other players only white label results from those four and are dependent on them and their business model"

Well, DuckDuckGo is pretty known and independent of Google. Not long time ago was an article here on HN about how DDG operates and they have their own web crawler, not relying on Google.

As others have pointed out, DDG appears to use Bing for the majority of their results, as such is not a "truly independent search engine[]".

>And empires must fall

Why?

because they’re often brittle and create closed systems

You have to see yourself that it is a pretty week argument.

This feels like tautological reasoning.

Brittle empires fall.

The Chinese empire has a history (including mutations to its government) spanning thousands of years.

And yet it fell.

> the last Chinese dynasty — the Qing dynasty — fell in 1911–1912, it marked the end of the nation's incredibly long imperial history.

It marked the end of that government, but the nation of China is still very much an empire, by definition of its ambition and cultural reach exceeding its borders. In the same sense that the US and UK are empires.

Of course, if you use a custom definition for everything and cherry pick informations, everything is possible.

But no, it wasn't just the end of the government, it also meant losing almost 30% of its territory.

https://imgur.com/r/mapporn/Cdi6KOL

> In the same sense that the US and UK are empires.

UK was an empire, and it's already fallen, it is now on the brink of losing even the Kingdom and becoming an Island hosting borders and customs.

U.S. is not really an empire, never was. It is more the mandator of many bad things that happened in the last 70 years, that turned many countries into U.S. colonies through "exporting democracy" which actually translated too many times as "planting dictatorships befriended with U.S.A.". But it was good for U.S. and only U.S.

That's why Cliqz has reasons to exist, U.S. is an unreliable ally, even more now and Europe needs to

> building the foundation for a sovereign digital future of Europe

> develop digital key technologies as a European alternative to the market dominating US platforms.

> If We Don’t Act Now, We Will Become a Digital Colony

I'm going by the Wikipedia definition, to wit: "An imperial political structure can be established and maintained in two ways: (i) as a territorial empire of direct conquest and control with force or (ii) as a coercive, hegemonic empire of indirect conquest and control with power." The US and China are the latter. The UK's power has slipped, but it's still in this category with extraterritorial colonies. A specific emperor is structurally optional.

But the Chinese Empire has already fallen and changed radically, U.S.S.R. is not the same thing as the Russian Empire, even though they shared most of the same territory.

The point was that every empire falls sooner or later and that it happened to China as well.

An empire crumbling under its own weight is different from deliberately destroying one because we dislike it.

I am not a historian, but I cannot think of any empire where there was no coercion. I read it in this context: Empires must fall, because "Absolute power corrupts absolutely".

Disclosure: I work at Cliqz.

In fairness, you'd also need to concede that there has never been a single independent village where there was no coercion. Mankind's history is mostly a story of how we have related to each other in forced groupings. (Forced by Mother Nature's many threats in the prehistoric past, but still, forced.)

So is it necessary that every grouping of man, "must fall", because there exists "coercion" in those groupings?

Just as a matter of full disclosure:

My own view is that any grouping must fall only when the grouping provides no, or very little, actual benefit to it's constituent members. Western Rome fell when people in Lutecia, Londonus, or even Ravenna were no longer deriving real benefit from being inside the Roman polity. Which arguably was long before Odoacer finally had enough and put an end to the charade.

Generally speaking, empires are negative entities. The work because they took advantage of being first (or first to be successful) and used that advantage to absorb the wealth of those they conquer. While a lot of productive things can come out of such an environment, at its heart the empire is sustained by consumption. When there is less to conquer, there is less wealth available for the wealthy, but the desires of the powerful is to always increase. The wealthy use power to to start pulling wealth from the empire. The common identity of the empire is destroyed and the wealth is no longer available to defend the empire, so the empire is ultimately absorbed by the next identity that comes along.

Wealth isn't some finite quantity that is merely taken or traded between a group of individuals. This is highly ignorant of economics and is a worldview that sadly what drives a lot of hysteria around wealth inequality (that when one person has more wealth it means they deprived it from another person who would have had it otherwise, combined with the idea it just sits there in a bank account not being used or it's all being spent like a consumer).

It is not a given that massive wealth accumulation involves depriving others of wealth but it's also very often the case.

While it's ridiculous to view wealth as a finite thing, it's just as ridiculous to imply hysteria over wealth inequality is based in ignorance. There is a long-standing trend of capitalists treating lower classes as exploitable resources undeserving of the means to have a healthy and happy existence beyond what keeps them productive inside the constraints of the current system.

Median salaries haven't been stagnant while the rich get richer because that's what's best for everyone, it's because that's what's best for the people with control.

Just for the record, many people who oppose and criticise massive wealth inequality are fully aware of the basics of economics (or even advanced economics).

Why? Because they hit a ceiling of the paradox of development. Development introduces problems that their current system cannot fix, and they face crisis: either breakthrough the ceiling or fail. Usually breakthrough is associated with figuring a new ways to extract more energy from the environment (industrial revolution).

I guess they don’t necessarily have to fall, but from a historical perspective I think it’s safe to say that empires will fall.

Because they only serve their own systemic survival and we have other systems of power and governance that can do more on top of this.

I don't think it's possible to convince people that viable alternatives exist. I've been aware of them for years, and never considered using them once. They have the stigma of just being not as good. Having more search engines does nothing if nobody will use them-- Google's hegemony remains if there's n or 100n other search engines, if none of them can provide quality results reliably.

Did that stigma not exist for IBM? Microsoft? Oracle? If the crux of Google is an outdated reputation they'll have to invest more and more to maintain it until it consumes them and somebody else steps up.

I love DDG. In my experience, Google still usually does a better job but DDG is good enough 90% of the time and bang searches make it trivial to handle that remaining 10%.

I recognize that there is still room to improve DDG's search quality. However, I wonder if DDG or any competitor can ever get as good as Google without collecting user data and using that to contextualize searches.

I’m not sure I agree that we need a new google, or to break up google.

But we shouldn’t trust google, what trust existed has been repeatedly damaged by their desire to grow and profit.

In my opinion we need transparency. Why is a result / snippet number 1, 2, 3, etc for a query?

“The algorithm did it” is not an answer.

Any new search engine should be accountable for the results they provide, and there should be a mechanism to dispute the results.

You complain about Google because you haven't use Baidu in China. If you have used baidu, you will know that Google is much better!

Do we need search ? It made sense when the web was wild. But now that it's very structured ... I'm wondering.

Kinda like road signs vs gps

The road system seems pretty structured and we still have road signs.

I want alternatives too - they are good for the consumer.

This article was negative to the point of being unreadable. This is not how you disrupt.

While I am curious about their search tech, it would be nice if they posted a blog post about their business model so we can understand their incentives up front.

While I'm wishing, it would be nice to hear about how they anonymize their logs. Their privacy notice only really mentions scrubbing associated IP addresses - which may not be sufficient.

LOL @ several accounts of people who work at this company selectively responding positively but ONLY to those who are already on the “Google = bad” bandwagon.

I’m extremely disappointed in the HN discussion here. The blog post is by someone wanting to compete with Google. Great. The entirety of the post makes ZERO compelling arguments about what features, guarantees, or outcomes their product will provide. In the last couple paragraphs, they mention the word privacy - okay but how so? What’s the business model? Why should I trust you?

The blog post is nothing more than an attack on Google (and other tech companies, but that comes later in the article). That’s the only “substance”.

There’s no question that key regulation is missing in the space. But what exactly is Google even doing that is anti consumer and killing competition? What anti trust laws have they broken? Should we identify them or maybe determine what is missing in our legal framework? If the author wants to compete with Google, perhaps they can share that insight. That be productive.

>Google is Cambridge Analytica on steroids.

That’s a pretty damning statement and unproductive. You want to make an emotional argument and have keyboard activists work for YOU instead of sticking to objective facts.

Disappointing this is #1 on the front page. We are turning HN into Reddit.

Google does everything it can to either have the first click or second click result in a paid event. Videos and related searches are clear examples of the abuse.

True content is buried or listed in a row or two out of 10 plus results.

Mobile is even more flagrant. Inswitch engines, just to get a less mainstream and digested view of my queries.

Google's US market share is 62.4% with Bing at 25.3% and rising [1]. That doesn't seem like a monopoly to me.

[1] https://www.comscore.com/Insights/Rankings#tab_search_share/

Not a fan of Google at all, but I'll pick them over "privacy focused search engine" owned by German publisher of Playboy any time of the day.

These guys call for banishment of "search monopoly" yet are in it for the money and business share. Privacy is just a currently sound competitive angle.

Cliqz gives me two choices for UI language, neither of which I speak, though I can read a little. It gives me three choices of search area: USA, Germany, France.

I would prefer the UI in English and I want hits from wherever they may be found not restricted to some small area.

Good try, but some way to go.

Submission title doesn't match article. Was it changed in the article or is this clickbait?

I think it's strange that companies like France (and Germany, UK, Japan, etc) don't pass a law saying "Only French companies may service web search requests originating in France". Web search is basically going to be a monopoly, so they might as well try to ensure that the vast economic rents created by the monopoly go to one of their one companies.

There is a very strong national security argument here: you don't want your citizens' web searches to be controlled by a foreign company, even if you're nominally allied with the country, because of the immense political power that control of web search entails.

This kind of policy would unite the populists on both the left and the right. The left would like the idea of redistributing money away from big American technobillionaires to local businesses, while the right would appreciate putting the interests of the nation first.

We'd get annoyed by being unable to use Google or Bing as there are not many such local services. The governments might be able to extract some more of the rents by taxing Google et al more.

I don't want google to fall. I want it to become better. The google results have moved from good (2004) to great(2006-2009) to good (2010-2011) and are on a slope to completely useless from there on .. right now are in shitty.

If you're looking for an alternative, dedicated tool to search US news give https://yetigogo.com a whirl. I'm indexing the top ~150 sites in real-time.

Was looking at that Cliqz company out of curiosity and while browsing their job page (https://cliqz.com/en/about/careers/campaign-manager) saw this:

"MyOffrz is opening a brand new marketing discipline: Browser Based Performance Marketing. This innovative technology makes it possible to show users discounts, special offers and valuable information with true added value within the browser. As a 100% subsidiary of Cliqz GmbH..."

"These include independent search engine, browser and privacy technologies as well as new techniques for responsible advertising and statistical data collection for the benefit of all users."

Seems a bit odd for a company that brands itself as a privacy search engine.

> Seems a bit odd for a company that brands itself as a privacy search engine.

Cliqz is nothing but a parasite disguised as a pro-privacy product, funded by Hubert Burda Media (https://en.wikipedia.org/wiki/Hubert_Burda_Media).

Their strategy to move into the couponing/voucher market comes as no surprise, since injecting ads (which this basically is) is a very lucrative business - and they have the customer base to do so. The line between malware and "legitimate" browser (extension) becomes increasingly blurred.

Their entire premise is that they can do an ad-financed product with less invasive tracking. Obviously, that involves advertising people.

Tracking and targeting are just two possible reasons why people dislike ads. Invasiveness is another one - think popups, popunders, layers, interstitials, pre-/postrolls etc.

So even if they don't do "tracking", injecting ads surely isn't something the average user "signed up for" when downloading the browser.

Sure, I totally agree. Doesn't change that that's the business Cliqz is trying to build. They're promising to be better on the privacy front than other ad-based products (we'll see how they actually do, I'm somewhat skeptical), not to be "distraction-free" or "non-targeted". That might not be the product you, I or many other people want.

Hey, I work at Cliqz. As mentioned in another thread; we have an advertising model, but it does not rely on any personal data. We have also experimented with paid products and we would love for it to be a viable option. Unfortunately, experience suggests that very few people are willing to pay (we would love to be wrong and will keep considering this possibility). So we are exploring many options; what they all have in common though is privacy by design. We do not, and do not want to collect personal data.

Hey, I work at Cliqz. We have an advertising model, but it does not rely on any personal data. We have also experimented with paid products and we would love for it to be a viable option. Unfortunately, experience suggests that very few people are willing to pay. So we are exploring many options; what they all have in common though is privacy by design. We do not, and do not want to collect personal data.

Proposed design is flawed. The same as Google. Check our proposal: https://news.ycombinator.com/item?id=21678527

It is already too late for Google to fall. Even if it falls something worse will take its place.

What are our options? Government? How will it be even remotely close? Data will be needed, How will the government get data? They will probably make it mandatory in schools of course. Why not? its for the good of the children.

Interesting times indeed.

“The individual is in a dilemma: either he decides to safeguard his freedom of choice, chooses to use traditional , personal, moral, or empirical means, thereby entering into competition with a power against which there is no efficacious defense and before which he must suffer defeat; or he decides to accept technical necessity, in which case he will himself by the victor, but only by submitting irreparably to technical slavery. In effect he has no freedom of choice.”

― Jacques Ellul, The Technological Society

You can't compete with search directly. It's entrenched. The best you can do is make it irrelevant by occluding it with something that provides better value.. voice-based assistants, for instance.

Wow, they managed to build a Google competitor that doesn't suck. This is actually pretty crazy. I never thought there would be new innovation in the search space. I'll be using this from now on.

There already are other search engines, but most people view Google as god and infallible in it's results. Even tin foil hat wearing people I know who hate Google the company refuse to not use Google.

Tried it out in German. Works definitely better for German language queries than Duckduckgo. Kudos to whomever is behind this & best of luck. The world needs alternatives to Google.

Perhaps what's needed is an antitrust breakup which forces Google to divest its ad business. They can then sell wholesale search to others, who can compete by monetizing it.

If someone wants to build a better search engine, start with a better code search and then expand to other areas. Github search is so bad.

Improved online code search and indexing feels like a very untapped market.

What do you think would happen if Google disappears? My bet is that Baidu would take over and then you would have even more problems.

How do I make this my default search engine as for when I type something into my address bar like I can with Google, Bing, or DDG?

One piece of feedback - currently your site governs language by region, rather than by browser language.

So let's say that google fall and you become the new king. Will you allow another competitor in?

Yes.

why?

Don't you have fiduciary duty for the owners of the company to maximize profits? Why do you want competition?

Lol, he's just saying that now. Everyone says that kind of stuff when they're not at the top. Then they get there and fully realize their constraints, and it dies down.

Yes I know. I just want to see how far he will go with these arguments.

[[Disclaimer I work at Cliqz]] Your question is spot on, many of us ask ourselves the same question.

Whether Cliqz would become a bully like Google if successful or not, I believe it's impossible to answer.

But at least, we know, that the data we collect from our users to build the search engine is totally anonymous and is used only to build search and the browser.

So, in the case that Cliqz would turn evil, I would stop using it, knowing for a fact that there are no sessions about me on their data, none.

Someone on the thread mentioned fiduciary duties, true that companies must maximize profit, but that does not mean becoming a gear on global surveillance conglomerate.

The unbundling of google is happening. There needs to be better document search.

I'd like a better (paid) way to search for medical advice/information

Is there an english-language version? The website is in German for me

There is - click the settings icon on the top-right corner. Then click on Sucheinstellungen, and under Sprache choose English.

Might make sense to make that more easily discoverable for international users, but I guess you're working on it.

Yep, we'll try to push an update in the coming days for a more meaningful interface and result language detection.

Somewhat suspicious that www.cliqz.com is hosted by Google..

Search engines are not just about quality of results. A reason to use Google is that it's what everyone else uses, so by using it you also get for free a picture of what kind of results everyone else is getting

The world doesn't want more search engines though. Like the world doesn't want another streaming service to pay for, it just wants everything in one place which costs $15 a month or preferably free.

use brave and switch you search engine to duckduckgo

is it just me or the site can't be reached?

Why is duckduckgo not mentioned at all?

hvae been using the CLITZ browser for some time and really enjoy it.

> CLITZ browser

Rather unfortunate typo, that.

The world had more search engines. A lot more. Google beat them all and they all folded.

Just Bing it :-)

It’s really sad to see the anti-success rhetoric.

I was around during the “search wars”, and what’s usually forgotten is that well, they were by far the best behaved as a corporate entity.

This “romantization” of this ideal state, devoid of historical context or acknowledgement of the world we live in is extremwly counter productive.

Should google, Facebook, Amazon, Apple do better? Absofuckinglutely!!!

Regulate them, hold them to a higher standard!

But Will breaking them up do anything except destroy them and just empower the next competitor ( who will have to learn the same lessons?

Even worse, likely in a country that won’t match your same values of “freedom”

Are imbalances in power great? Nope

Are monopolies awesome? Most never

Are billionaires awesome? Nope

But this is the stage we are at due to an interconnect global economy. We built this.

Knee jerk solutions don’t and have never fixed anything

I was around too. The problem is you think the Google of 2000 is the Google of 2019.

It is not. Apparently some people are still fooled by the doodles and dinosaur jump game.

It’s easy to say MORE REGULATIONS... just as after any tragedy people claim “we just need one more law”! The truth is government has proven toothless in tax collection and regulation of these giants.

I'm not sure applying antitrust laws (which have been in place for longer than anyone has been alive) to a monopoly (which has been acting monopolistic for years) qualifies as knee-jerk.

> I was around during the “search wars”, and what’s usually forgotten is that well, they were by far the best behaved as a corporate entity.

They were well behaved when they had competition to worry about. Once they effectively achieved a monopoly, ethics slowly got relegated to the back seat. They started leveraging the search monopoly to take over other industries, to undermine the open web, to engage in political censorship (hello Project Dragonfly) and much more.

Everything you say here is an excellent argument why search should not be a monopoly. We need competition to keep the search engines honest. Regulation won’t help, since orgs like the FCC are vulnerable to regulatory capture, and there’s no other way to "hold them to a higher standard" than taking your business elsewhere. As long as you continue using their products and thus earning them money, they don’t have any reason to care what you think about their business practises.

Isn't "project dragonfly" the Google search engine version for China? Google was trying to comply with the law. It's a law you might not agree with, but come on. The reason all these governments want Google split up is more censorship, not less. From torrents to negative reporting on EU commissars. From everyone's interpretation of hate speech to pages pointing out that sometimes social services are the abusers in children's lives, not so much their savior. Scihub, Schwartz, Snowden, Yellow vest demonstrations, Hong Kong student injuries and disappearances, Twitter making some tax agency officer's abuse of power a big deal, ...

That's why they want Google and Facebook broken. And of course, they're unwilling to spend any money, change any law anywhere, accommodate anyone, or put in any kind of effort whatsoever.

Fundamentally they want the internet to disappear. They want the control of information out of the hands of these idiotic American nerds that refuse to control information !

How would breaking up a company help censorship? Surely it's easier to censor one entity than many.

Divide and conquer. They can push for one of the companies that is pro-censorship.

One the other hand, Google is so big that it could just ignore demands from smaller countries, especially when they are a small drop on the bucket in terms of revenue. Multiple small, regional companies can't afford to go against the government in their area and aren't present in other areas.

Right now the only country that could actually do anything about Google is the US, but they won't because we benefit from having the number one internet search company be American.

Also, the rank and file employees in Google are largely liberal and tend to make a big fuss over things like censorship. If Google is broken up a company that is more friendly to state interference could emerge and take it's place.

You've nailed it. Back when tech was the underdog, it was trying to make things more open, accessible, free-as-in-speech. I remember when the internet was very pro Ron Paul, and against government regulating technology (remember the losing battle against DMCA and DRM and the PATRIOT act?). Now that tech has become successful, the losers (those who want the control) are trying their hardest to create new narratives about tech companies' abuse / monopoly. Can tech be improved? Or course, but with small iterations, not complete overhauls.

> Google was trying to comply with the law.

Google was trying to gain the Chinese market by implementing political censorship. And once they knelt and swore allegiance to the Chinese regime, every other totalitarian regime around the world would be lining up with demands that they do the same in their countries.

Expecting Google to stand up to EU on anything is naïve. They already got massive fines, they don’t have the spine to risk more. They will do as the commisars dictates, whether they’re split up or not.

> Google was trying to gain the Chinese market by implementing political censorship. And once they knelt and swore allegiance to the Chinese regime, every other totalitarian regime around the world would be lining up with demands that they do the same in their countries.

I'm super glad that google didn't go through with dragonfly and it's really sad that they even went as far as they did. However, going through with it would have just made them the same as basically every other company.

Apple, microsoft, and basically every other company all operate in china, following chinese "law". Ironically, it's two of the worst/most hated companies (google, facebook) in the US, and most frequently demanded to be broken up here on HN, are two of the very, very few companies that haven't bent over for china.

I think it’s pretty clear that Google would have gone through with Dragonfly if it wasn’t for the protests against it, both from the public and Google employees.

In other words, their refraining from Dragonfly wasn’t because of moral courage or a principled stance. It was just to avoid a PR disaster.

And it was only because Dragonfly was leaked that that we ever heard of it. Imagine what other interesting programs they’ve got going on that we haven’t heard of yet.

It's not so much the "anti-success" wars as that Google's story is a microcosm of the "tendency towards monopolies" that ultimately plague the upper end of capitalism.

Even as a Google Employee I agree that the concentration of marketshare and infrastructure that Google has distorts the market. But without radical intervention either by normal humans collectively or by governments using legislative and military power to shape corporations into their vision (again, probably collectively to be effective) the problem can't go away.

At the very least, we need to stop pretending that pointing our the problems of rent economies and monopoly tendencies in capitalism is not "an anti-success rhetoric" but rather a candid and informed discussion. Markets do not function well (defined as delivering utility to the largest number of people, we could of course define this in other ways) when you need to render the player's assets on a log scale.

> Even worse, likely in a country that won’t match your same values of “freedom”

For many people in the world, the US already is a country that matches this exact description - what with toothless antitrust laws, anti-union actions being tolerated, and what the ICE camps look like is something we don't even have to talk about.

Sure, China would be worse, but you should strive for more than just "the second worst developed country(tm)"

In fact, yours is the perfect argument why we in Europe should try to end the dominance of Google and break it up.

The US is far less restricted than europe for freedom of speech, the major principle that matters when you're talking about information technology.

In the case of search engine balkanisation (I'd be in favour of it), I'd think that eurosearch would, on average, be more censorious than amerifind.

Aside from Holocaust denial, hate speech and libel how does the US differ from "Europe" in terms of free speech?

The US has what we might call "better" libel and slander laws than some actors in the EU and with the UK being a popular example of the most ridiculously bad laws historically.

>> Aside from [...] libel

I didn't see your edit, sorry.

"Hate speech" is such a glaring flaw in "freedom of speech (EU version)" that it invalidates the concept. These laws differ by country, which makes it a nightmare to follow. Some of them have historically been quite silly. There was a case in 2008, where a protestor was fined because he wrote something along the lines of "get lost, jerk" on a sign directed at the president. This eventually got overruled, but it should never have been a case at all. Then there are also the obvious cases such as Count Dancula or whatever he's called.

Has Europe’s pro-union climate led to more competitive software companies, or better pay and benefits for its software engineers?

Every European engineer I talk to seems either amazed or resentful about American software engineers’ salaries and benefits. How have the unions helped you in this regard? How have they helped your tech companies be more competitive in the global economy?

> Has Europe’s pro-union climate led to more competitive software companies, or better pay and benefits for its software engineers?

E.g. we get health insurance even if we quit or switch jobs.

The thing about European software companies tho is that they are hardly unionized at all.

>Every European engineer I talk to seems either amazed or resentful about American software engineers’ salaries and benefits.

I am not, to be honest. Sure, at a first glance it might seem the US devs make tons of money. But once you account for cost of living (like the astronomical rents in every major tech hub city) things do not look that rosy anymore. I know US people who make twice or trice what I make, and yet they live in dumps that you can hardly call apartment and they say they cannot afford any better. That, combined with the at-will nature of US employment, really doesn't strike me as desirable.

I like living in a country where I have a pretty awesome standard of living, a nice apartment that I don't have to share (unless I want to) and a safety net that is (at least for now) able to catch me should I ever struggle or become sick, so I won't go homeless.

Those things are largely thanks to work the unions did in the past and keep doing.

Certainly, the salaries are a bit of an illusion when you take the insane housing costs of the Bay Area into account.

If breaking up the American tech companies helped disperse jobs outside of Silicon Valley, that would be a good thing.

Europe is really diverse in wealth and business climate. The wealth gap between different European nations is so large that the US probably seems like the pinnacle of equality. It's hard to talk about Europe in general in this sense. I've worked as a developer, where I made less than $600-800 a month. Although I was underpaid, but it probably wouldn't have been possible to make double that in my country. I think you might want to dial your question in slightly more than "Europe", perhaps "Western Europe".

> It’s really sad to see the anti-success rhetoric.

The title may be, but the article was not. Was a reasoned statement for why those folks bothered to build another search engine.

As Google has gotten bigger it seems to forgotten its motto of "Do No Evil," and done the total opposite for the sake of profits!

It was an inspirational company and story that many rooted for due to its culture and mission.

IIRC, they actually removed the "Don't be Evil" motto in may of 2018

They moved it to the end of the mission statement.

If Google were still as well behaved after success, I don't think posts like this would exist.

> they were by far the best behaved as a corporate entity

Were. When though? The founders said in their original university paper that selling advertising was wrong and would inherently corrupt a search engine. Of course once they realised how much money was being made in search, that fundamental tenet of don't be evil lapsed before the phrase was even coined.

History and Google's progress of updates seems to have borne their claim out. Google is worse than it was, in good measure as they have twisted results to push more sales. It's a shopping search first and foremost, and frankly not a very good search any more. With an obscene overreach of data gathering that is almost impossible to evade. Every opportunity to gather data they gather more. Remember the Google war driving guy who "accidentally" gathered all that extra wifi info from streetview cars? Entirely by accident. How many believe in that "accident" who isn't on Google payroll?

So no, I don't see it as anti-success, more that they became far too large, far too greedy, and far too abusive - e.g. demanding everyone train the self-driving fleet to pass a captcha. Scanning the world's books was a far more appealing proposition.

Given they have the data, and the holy trinity of search, advertising and monitoring, regulation is not enough. Split off advertising or analytics and monitoring, or break it up some other way. That's to serve the public interest, not as some anti-success crusade.

> Scanning the world's books was a far more appealing proposition.

That proposition failed precisely because it was regarded - rightly or wrongly, it's not clear that it matters - as a "far too large, far too greedy, and far too abusive" endeavor.

I thought it was more that the necessary licensing fell through, or was agreed and then fell apart. Added to which the Google Books interface was and is dire.

Are you reacting to the blog post, or something else? I didn't see any anti-success rhetoric, only anti-monopoly rhetoric. But for the record, success isn't sacred, especially if people succeed at something that sucks. I'd be decidedly anti-success where genocidal dictators were concerned, for example. I'd be explicitly against their ambition, their drive, and their personal fulfillment and growth journeys, sorry.

What behavior here is breaking anti trust laws? And please be specific. Amazing that you compare Google and genocidal dictators.

Did I mention anti-trust laws? And do you understand the concept of an example?

> Are billionaires awesome? Nope

I disagree here. Billionaires are often pretty awesome. Once someone gets way more money than the could ever need, they tend to pursue other activities that they enjoy more than making money, which is frequently giving it away (Bill Gates, Warren Buffet) or starting moonshot space companies like SpaceX, Blue Origin, or Virgin Galactic. Even Zuckerberg donates to projects like these.

Rich people take on projects that may not be viable yet, but are important nonetheless, essentially filling the role that government used to play in R&D.

Yeah. And even if their hobby is tearing up cash to mulch their back yard, I don't think it inherently harms anyone. There's a lot of talk about a wealth gap but it still feels like those caught on the wrong side are more prosperous than they used to be (eg they have smart phones and many other modern conveniences which didn't exist a few decades ago).

>Knee jerk solutions don’t and have never fixed anything

generalizations are false a lot of the times

It is not anti success. Tons of us got good value and watched Google do great things.

Now that they have that success, lacking threats, the value is evaporating.

Lots of people do not care how much others make. If the value is there, world better, game on!

But, getting very large amounts of money and those things aren't really happening and lots of people will definitely express negatives about it.

As they should.

I see links are back in my search results now. But my trust is not back with those links. I have found all the other search tools I need and am using them, encouraging greater success.

These companies with almost impossible to think about type large amounts of money can be doing better by people. Expecting that makes great sense.

And where that is not effective, seeking various remedies does too.

Competition, use of the State, protest, all on the table.

Anyone concerned about the potential impact of those things is completely free to avoid them. Just make sure the value to people is there and they won't really care about the money.

Knee jerk, break them up helped create this net we are seeing consolidated and increasingly devoid of value.

People calling for that again is understandable.

Finally, breakups are super expensive. A real, material threat of that happening makes for a real, material cost and risk assessment, which can justify hedging all of that with better value to the people too.

It's funny because Google had a very different perspective when MS was the Monopoly we were worried about.

> It’s really sad to see the anti-success rhetoric

Oh, come on!

Google is not doing its job well, other could do parts of it much better, but Google is too big to allow others to enter its same space, so we need alternatives that can put pressure on Google more than we need Google.

The "people hate success" rhetoric is so sad it can't even be measured.

i don't see anywhere in the linked article that talks about breaking up google, so it's hard to know what you're responding to. the article does make a central claim that google has an information monopoly, and that the search market needs competition (and oh, look at that, they made a new search engine!).

search isn't a naturally monopolistic market, because although there are some initial high fixed costs, much of those costs can be roughly scaled with size. there aren't really any other significant barriers to entry.

marginally but noticeably better search results (e.g., pagerank) did provide an early advantage for google, which is how it established its monopoly over time in the first place (combined with the ad auction model from overture, which gave it economic & political might), but that advantage has eroded with its lack of focus on search. google also has a giant suite of data gathering products but it's not clear that those provide a true competitive advantage in search (again, search results aren't much better than the competition).

the other interesting characteristic of search is that its competition isn't necessarily zero-sum. a user might do the same search on multiple search engines, providing each competitor with a full marginal revenue opportunity, rather than only one competitor winning the business (as in classic competition).

because of all this, i'm neutral on the breakup argument but am bullish on the "increasing competition" argument for search.

I think people should be responsible for their actions. If they only use google without even trying anything else then they are themselves to blame for whatever bias/lies google have fed them. And asking me to pay yet another tax for regulations so that dumb people are protected from the consequences of their actions is... it's like they are trying to build a kindergarten for adults. And I am not a fan of socialism.

Even if the consequences of the dumb people's actions indirectly (or directly) affect your life?

Yes. If I put myself in the position where I am heavily dependent on what they do then I have only myself to blame. And if it is a minor dependency then I probably won't even care. And if you are talking about them doing violence then bullets are cheap.

lot of facts

As far as I remember there's Google, there's Bing, something that only works with Chinese, something that only works with Russian, search engines that barely work at all and a bunch of meta search engines, that use Google or Bing or both at the same time under the hood, like DDG, Searx, Startpage, etc.

So yeah, I'm down for more search engines. If there wasn't only one dominant one, SEO wouldn't be such a problem, either, I imagine.

This is cool. But some criticism. UI seems a bit janky. Results are good but visually presented in a way it's hard to tell each result card from the next. Cliqz is a bad name, even tho you may want the world to be a cyberpunk dystopia where renegade hackers control the outposts of civilization, Google has not fallen yet, pick a name the 99% can relate to. And get Machiavellian ... if you're really gonna beat Google, you're not gonna do it with hippy tactics, no love flowers light or manifestos. You have to get evil and more evil than Google. You have to out evil the big evil.

Realistically if I were trying to beat Google I wouldn't be building a search engine. What can you offer that they can't? And if you even begin to compete with them there, what steps can they take that you can't recover from?

Instead, I would focus on beating them first, building search later (if search is what you actually care about). What's Google's weakness? Figure out "the cord that makes them run" (to paraphrase Meredith Vickers in Prometheus) and relentlessly attack that (legally, of course, this is business not cartel turf war). I have no idea what that might be, but Rome fell, IBM fell (well, stumbled, I guess?), so Google must have its Visigoths or Bill Gateses or whatever.

You built something cool, that replicates the functionality (80%). Cool. Good for you. You proved you can do it. Now decide if you really are serious about "beating Google", and figure out an effective way to do that. Search engine is not the way.

Anyway, Google built the (past of) search. If you really wanna do search, build the (future of) search. I dunno, like AR/VR search? Search for FBs AR platform? I have no idea but text search on WebPages is the past. Google already won. Get over it and move on. Build the future, or go build a better steam engine, you know, for like fun and stuff. Because it's cool, but it won't ever beat Google. Sorryz Cliqz

[disclaimer I work at Cliqz]

We are all ears on ideas on how to beat Google :-)

But on a serious note; i do not think that the aim is to beat Google because of Google but because on the monopoly they have on the access to the information. And that's why we are building a potential alternative there.

Attacking from a different domain, might beat Google on that area, or let's go wild here, perhaps even on revenue, but would not fix the problem we wanted to fix to the begin with.

"Replicating" 99% of the functionality with a quality to be good enough is a very valid option IMHO. Otherwise if one decides to leave Google, where do they go? If they do not want to advertise on Google, where do they go? There are too few alternatives, there are many names, but most of them are aggregators (full or partial). That said, if besides Bing, there would be 4 or 5 like them (or better), then yes, I concede that Cliqz as is, would make no sense.

well I like your crazy idea. why not offer me a job?

Google is getting bad, but I want a decent competitor to Windows/Android.

Apple is not competition. They serve a lax customer base.

No it does not. Google is one of the few things that need only need one of.

You guys are polishing the steam engine, while we need some real progress. Typing a search query isn't much better than typing a SQL query. This should be the last resort thing.

We need a service that would organize the information for us. Instead of us telling the service what we want to see, the service should tell us what's worth attention. Some people care about NFL, so they would be given the most valuable news about NFL. Others care about superconductors, so the service presents them a daily summary of most important advances there.

Reply on news.ycombinator.com