Google has added DuckDuckGo as a search engine option for Chrome users
techcrunch.com
techcrunch.com
It's also a solid choice for them to hedge against antitrust claims, if they can point to having just added them to their browser, regardless of the fact that Google is the default and they do not present a choice screen like Microsoft had to in the EU.
There's a lot to unpack in that statement... Is there any recent analysis on the usage stats that chrome is reporting back that someone could point to?
Isn't it well known that Google scoops up web history from the browser or have they stopped doing/never done this? In the latter case any pointers would be appreciated.
A default open browser history synced across devices seems like exactly the sort of thing that would show that DDG has increased its market share.
Source: I work in education - even in a highly educated area in a developed EU country, young and old alike think like this.
Not only is Google's indexing infrastructure not that fast, but they deliberately don't do that because some poorly designed sites have passwords or unique keys in the URL that should not be used to retrieve content for the public search index.
https://www.google.com/chrome/privacy/whitepaper.html#usages...
https://developers.google.com/web/tools/chrome-user-experien...
Good. 4 is a good number. It's on the low end of the number range people think of as "enough choice."
At its simplest, in the West we have a thing for threes. Three bits of God, three little pigs, three branches of government (in the USA at least), "things come in threes", three books/movies in a series (a trilogy), stories that have a "beginning/middle/end".
Bottom line, the West tends toward organizing and thinking of things in threes. Some might even be superstitious about threes (perhaps a Pythagorean influence).
In China the number 4 plays a similar role. I don't know much about 'numerology' in China, save to say that recently the number 4 (which apparently sounds like 'death' in Chinese) has been considered bad luck. Here's a better explanation than I could give: https://www.quora.com/In-Chinese-culture-why-is-the-number-4...
To add a complication, Chinese tends to use the US convention (with the ground floor being floor 1), while the English convention is the British one (with the ground floor being floor 0).
By happy coincidence, then, the 13th floor is also 十四樓, ie the 14th floor, so you only need to skip one floor, rather than two. That explains why HK skyscrapers are so high.
Say, frozen chicken, napkins, instant noodles, paper cups, etc. In some cases there is only 1 option offered, sometimes 2, rarely are there ever more than 4 options offered at Costco for a single type of item. When you trust that you are being offered the best choice or a top choice, well, we know what happens at Costco. People buy pallets in that warehouse.
Effectively, Costco shoppers are people who already have chosen, "the cheapest fairly good quality option."
Costco is the "I'm feeling lucky" button.
What percentage of Chrome users consented to the data collection? (Is consent even required?)
Does the data represent all Chrome users or only those who have consented?
Unwillingly consented, that's the vast majority of Chrome Sync users. Unless you enable the end-to-end-encryption (for which they require a second passphrase, so probably less than 0.1% actually use that), they will use your data for ad profiling etc.. Yes, that is on page 1312 of the Chrome Sync privacy statement. (They're only required to write it into there, if they do it, so it is quite certain that they didn't just want the bad PR for nothing.)
Is consent required? Assuming they actually do collect this data from their Chrome Sync data or through similar personally identifiable ways, consent would be required in many jurisdictions, especially the EU.
However, if they cared enough, it would be possible for them to collect this particular data point without personal identification. You could for example create a UUID per installation that's only associated with this one data point. Or you could have a time-based solution where each Chrome instance goes out to "vote" for their default search engine e.g. every 4 weeks. If you then look at the statistics on a weekly basis, you can just take these values times 4 to even roughly correct numbers. It's certainly going to be representative enough, you don't need every browser instance to have their vote in every week's statistic.
These metrics are from UMA stats. They are collected from everyone who ticks the box to report stats when installing Chrome.
They only get histograms of counts of visits to search engines, not the entire URL, and not search engines or other sites not in the list of things they track (which is at the bottom of the file).
chrome://chrome-urls/site-engagement
and other chrome-urls
These can provide useful data for me but not sure why I would want send the data to Google.
One way to make the pages I visit load faster is to disable Javascript. Another is to remove (or block) advertising. Another is to put DNS data for these sites into local hosts or zone files.
Those actions are how I prefer to approach the problem.
However as far as I can tell, those are not actions Google wants to take. They have their own preferred approach.
It is possible there are users who are aligned with Google in terms of how they want to approach the problems created by misuse/overuse of Javascript and advertising.
It is also possible there are some users who have no idea why pages are slow to load.
Those groups might want to send usage data to Google.
However I am not in either group. I dislike the web advertising business that Google depends on and therefore must nourish and support.
As such, there is no reason I can think of why I would want to send data to Google.
Also, I have not checked but I wonder if Google is restricted in how they can use the collected diagnostic data. Are they prohibited from using it for the purposes of selling advertising?
So if Chrome's ever made a UI change you disagreed with, then you're in a group that would have benefitted from sending Google usage data.
In terms of the restrictions on usage data, see https://www.google.com/chrome/privacy/whitepaper.html#usages... .
I care about command line programs, less-interactive and non-interactive use. Truly, the best interface is no interface.
The whitepaper.html appears to explain how usage data is utilised in ways that help Chrome improve but does not appear to contain any restrictions on use of the data to help further Google's ad sales business, whether directly or indirectly.
It is the business model that I do not wish to support.
Producing software such as Chrome is just something the company is doing in the course of selling advertising and collecting maximal amounts of data from users, whether the data is anonymised or not.
Reminds me of the sort of advantage Facebook had from its VPN app to identify competitors early to kill/acquire them.
* DDG in 2018 has served similar amount of search queries as Google in 2000.
* DDG growth rate is accelerating
* Google search growth rate is negative
* Google's share of global search is shrinking
DDG stats: https://duckduckgo.com/traffic
Google stats: http://www.internetlivestats.com/google-search-statistics/
Google search growth rate is always positive in that page. It just decelerated. Growth rate being negative means you're actively losing more users than you gain.
I know it was their first product, but I would imagine they get much of their revenue from other avenues, such as Android's built-in totally-not-antitrust web search app, and YouTube and Gmail and web ads...
I suspect people that actually care about privacy aren't using Chrome though.
I agree it would be nice to let people configure this (see comment 12 on that bug), but we're pretty careful about getting privacy right (despite wide-ranging internet claims to the contrary) and "falling back" to Google in that case would be a pretty major gaffe.
We haven't always been perfect. When we first launched, for example, we didn't exclude some cases from suggest querying that we should have, and that was my oversight. I can't remember the specifics (things like https:// URLs or input while in incognito mode, IIRC) but I landed a patch a couple days after the 2008 launch to clean it up.
The Chrome team as a whole is very privacy focused. There's a lot of people in public (including in this article's comments) who think Chrome is some sort of Google data collection device, but having seen things from the inside, I would trust Chrome with my data over any other browser. It makes reasonable tradeoffs by default (e.g. not enabling features like sync or server-side spellcheck that tend to send more data), and what stuff is there that people might not want (e.g. omnibox server-side suggestions) can all be easily disabled.
See also https://en.wikipedia.org/wiki/Google_Chrome#Listening_capabi... , which is a reasonable summary with more detail than what I've given above. For full details, see https://www.google.com/chrome/privacy/whitepaper.html .
Source: I am the former Chrome omnibox owner. You can find the relevant code for this starting at https://cs.chromium.org/chromium/src/components/omnibox/brow... ; look for how GetDefaultProviderURL() works and when that query is sent. You can also watch packets with your favorite network analyzer.
It's the complete opposite to that, and you said it yourself. Their aim is to quietly retain/recapture users while keeping antitrust at bay, and they did well precisely in not publicizing it.
The other day I searched for the website to check a restaurant gift card balance. All of DDGs results were obvious scam webpages. I often search for ElasticSearch documentation. DDG always returns very old versions for these docs, while google returns the most recent version.
I usually try in DDG first, and then in the small cases where it's not found, I just prefix "!g" and re-execute the query.
Full list of supported bang queries: https://duckduckgo.com/bang
I too just type the bang command somewhere, and usually it’s at the end of the search term or phrase.
EDIT: I tried to do just this, and both of them blocked it :(
Refused to display 'https://www.google.com/?q=cheese' in a frame because it set 'X-Frame-Options' to 'sameorigin'.
Refused to display 'https://duckduckgo.com/?q=cheese' in a frame because an ancestor violates the following Content Security Policy directive: "frame-ancestors 'self'".Also FWIW, iframes can still be useful on the regular web for third-party widgets, as well as same-domain pages.
If the web is a 'turing complete platform', then it should be possible to run a web browser in a web browser. So Chrome inside Chrome. That could be anywhere between the level of 'webassembly to run the whole thing', or it could be at the level of 'iframes give all the necessary functionality'. Today the first isn't viable because webpages can't make raw TCP sockets. The latter isn't viable because of the way sites can differentiate between iframes and the top level window.
If I have to use a fork/modified copy of Chrome for the outer copy with slightly different rules, then it isn't capable of implementing itself.
Imagine if gcc couldn't compile gcc - you needed to use a seperate compiler-compiler. It's the same thing.
2. I don't use ElasticSearch, but I can tell you that searching the python docs is quite simple in DDG, just throw a !py3 in there to directly search the latest Python 3 docs. Apparently, there's a comparable bang for ElasticSearch, !elastic. But I don't know how well it works (and it's a bit long, really).
This has happened because, firstly, I, too, can instantly recognise when results are garbage and so immediately type "!g". Secondly, I know when certain types of searches will be garbage - usually anything related to programming is useless using DDG. So, for work, my default search engine is just Google.
Sometimes, I just query with "!g" without even thinking about it, and at one point I realised I hadn't even been using DDG for several weeks except as a redirect.
I look forward in time to not having to bang Google a lot and being able to find...
- Distance info.. how far a drive is X point to Y point. DDG doesn’t offer this capability yet and it’s something I do Very frequently.
- nearest Movie showtimes
- nearby concert listings for today, tomorrow, weekend
- flight info and links to purchase flights
1 - I use Chrome + Google for work stuff 2 - I use Firefox + DuckDuckGo for personal stuff
I sync my passwords with Bitwarden.
I sync my passwords with Bitwarden.
I have started doing this as well, except I'm using KeePassXC and using Dropbox to distribute the file everywhere. Would Bitwarden work behind a company firewall at a company that doesn't allow Dropbox?
Bitwarden is pretty nice, using it at work. I do still like my control that KeePass gives, though.
Also, I currently use 1Password, but have been thinking about using Enpass[2] because you can sync with any cloud drive. I like the idea of syncing to a third party cloud drive in case my password service is compromised.
[1]: https://brave.com [2]: https://www.enpass.io/
I used Brave for awhile then switched to Firefox+uBlock Origin, hoping to do my teensy part in decreasing the market share of Chromium-based browsers while still being privacy-focused.
Often vulnerabilities go unpatched for days, which is pretty bad when the exact vulnerability exploit code is already visible in the chromium bug tracker!
Also slowly migrating to Bitwarden.
Or you could even use Chrome and DDG or Chrome and Startpage for work.
Anything where Chrome and/or Google are avoided is a good thing, IMO.
Over the last few years DuckDuckGo have become so good at handling my queries that I only occasionally use Google. That typically happens when DuckDuckGo doesn't find what I expect, but it always turns out that neither does Google.
I am so disappointed with DDG recently, it has adopted Google's strategy of returning searches that have nothing to do with your query if not enough results were found [0], and dialed it up to 11. If "I" "don't" "put" "each" "word" "in" "quotes," the results I get have nothing to do with my search... but if I do that (apart from the inconvenience of it all) it means (presumably?) that stemming isn't done on the search terms.
Maybe I'm old school, but I expect search results to match the search terms. Fuzzy matching (stemming, synonyms) is an added bonus, but silently dropping words which don't appear is decidedly not. Moreover, a search result returning "only" two results should be taken as a good thing for someone with confidence in their dataset (DDG naturally doesn't have that, because their coverage is far from 100% of the web) - it means the search terms were extremely precise and the results are highly relevant, with irrelevant results filtered out. Decreasing the signal-to-noise ratio by willfully ignoring my search terms may increase the quantity of search results but - and I don't know about you - for me I don't care about quantity and would choose relevance as the more appropriate metric to benchmark against.
(All that said, I still use DDG as my main search engine even if I am turning to appending !g far more than I ever used to because I firmly prefer DDG's respect for my privacy and person over Google's treatment of the same. But I'm disgruntled and, frankly, very disappointed. Sorry, @yegg!)
[0]: https://neosmart.net/blog/2016/on-the-growing-intentional-us...
Edit: actually the situation is even worse. DDG doesn't seem to even always respect "quoted" terms. Here's literally the first search I did after posting this [1]. The quoted term "CFF2" doesn't even appear in the majority of the results DDG pulls in - not just not in the page summary displayed, but literally not on the result page at all. For comparison, here's the Google equivalent:
[1]: https://duckduckgo.com/?q=windows+10+%22cff2%22&t=ffab&ia=we... [2]: https://www.google.com/search?hl=en&q=windows%2010%20%22cff2...
I completely agree with you here but in my experience it's not anything new with DDG, that's always been a problem as far as I'm concerned.
As a hobby I sometimes have to reverse engineer electronic circuits, when I'm not sure what a chip does I try to search the inscriptions on the package to see if I can find a datasheet online. Sometimes you end up with very cryptic strings like "xardc10-egh" or whatever. If you input this string on Google it gives you no results:
https://www.google.com/search?hl=en&q=xardc10%2Degh
If I do it on DDG I get pages of irrelevant results:
https://duckduckgo.com/?q=xardc10-egh&t=ffab&ia=web
That being said DDG improved slightly, when I did searches like those a couple of years ago I'd often end up with results containing completely broken encodings, binary dumps as ascii and other obviously erroneous content that got indexed by mistake. Here the results at least appear to link towards proper pages.
It comes up VERY times DuckDuckGo is mentioned, yet there's not a single source that suggests that DuckDuckGo is just a frontend for Bing.
It's clear that DuckDuckGo used Bing for some result, but not to what extend. Are all result Bing? Does Bing only provide results when DuckDuckGos own crawler fails? Are the results mixed? I very much get the impression that results are mixed, but that's not completely clear either.
I should check server logs at work, I don't run my own web servers, so I can't claim to have seen it.
> In fact, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from a variety of partners, including Oath (formerly Yahoo) and Bing.
Why should I care about where DDG gets its search results from? They promise a good web search and don't deliver.
Although the change has the awkward effect of splitting ddg reporting into the two groups based on date of traffic.
[1] Because in the world of the press everything should be announced so they can broadcast it and sell advertising by running stories. And not have to find it out by other more laborious methods.
This is a deliberate Orwell reference? Vernor Vinge speculated in Rainbows End that everything which couldn't be searched for in a search engine would effectively become invisible. In 2019, that manifests as, "everything which can't be searched for in a search engine, which is backed up by crosslinked mainstream news sites and which isn't warded by words meant to scare casual readers away."
Look, you know that Google in an act of benevolence gave them duck.com last year, that’s PR.
Antitrust angle is obvious. They want to appear they aren’t the only game in town. Esp when you have people like Warren making (hollow) antitrust campaign noise.
Microsoft did this with Apple.
> DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from a variety of partners, including Oath (formerly Yahoo) and Bing.
AFAIK Oath / Yahoo has switched to using Bing under the hood since 2009: https://www.nytimes.com/2009/07/30/technology/companies/30so...
Kind of disheartening regardless. I assumed they had their own scrappy, independent tech stack.
For one thing, loads of sites load all their content via Ajax, so at a minimum you're gonna need a browser engine as the base of your crawler...
Doing all these would allow you to surface better results than using just one. But it comes at a cost.
Yandex is Russian and has pretty poor results. And you’d be naive to think that those of us concerned about privacy would ever touch something built on top of it.
In other words there aren’t paid search engines that DuckDuckGo could turn to. Unless they build their own crawler, the only game in town is Bing.
And yes you can compete with Google (for a certain target group) by paying them for their search results. Results are just a distribution channel, it is what you do with them that matters. For example Google and DuckDuckGo both choose to show you ads and affiliate links but that is hardly the only option.
Sorry, but I don't trust DuckDuckGo that much and when technology fails, I'd rather use a search engine based in a country that has rule of law ;-)
They're desperate to collect user click data because they know that's the only way they'll have any chance of success. Even anonymized, that's very valuable data.
DDG also doesn’t have searching by time for longer than the last month, whereas many a times when I look for technical stuff I also tend to look for things in the last one year, which Google and Startpage provide.
https://www.google.com/search?q=bare+metal+arm-none-eabi+spr...
Would you think of Google as trustworthy because they only gave their backend two pieces of data? I myself would not, because I'm pretty sure the actual request and response messages are looked up by client ID (in their Google Analytics data store).
Chrome (and Chromium) creates at start and maintains SSL connections to Google. It is not easy to sniff what is being sent. Even if you MITM it, like in enterprise transparent proxies, Chrome will throw an error because of cert pinning. google domains should be whitelisted: "we recommend that you avoid the use of transparent proxies." https://support.google.com/chrome/a/answer/3504942?hl=en
But I doubt the Chrome extension that would do that patching could stay at Chrome store.
Yes the data. The whole request and the whole response. They need it in order to properly train their ad-network.
</speculation>
Given that they scoop up all this data I'd appreciate if their ad-network actually improved. Just the other day the dating site scams where back.
"We'll try not to show it again" they say. Well for vacuuming the market for the best and brightest they either don't try very hard or they are very dysfunctional because they fail as a group.
Everybody working for this company should be ashamed.
And anyway, it's “to improve your experience”, so you can't opt out. It's for your own good. Remember, Big Brother loves you.
Because Chrome keeps a log of all your activity, your DDG searches are easy to find here: https://myactivity.google.com/myactivity
So if you're web based (like me) then activities such as sending an email, checking out YT, reading HN, watching Twitch, and jerking off, all end up as entries in that log file.
Do you really think Google would have trained an AI to determine that last activity? How would they have trained the AI?
Sloppy. How do they know I'm not preparing for actual intercourse? How do they know I'm not downloading porn for later? I could also be watching gun videos, since they've been hosted there.
Granted, it's probably 95% accurate.
I think the causal order is flipped around here.
Yes!
>> How
How graphic do you want me to be? But isn't the real question: is there utility for an ad network to know about your preferences in porn? If there is such utility, you're best to believe Google implemented a way to get them.
If I was an ad network I would love to hear about your porn habits. I would absolutely love it.
If you know, you are obviously one of the people who has that data. If you don't know, you aren't.
If I was an ad network I would love to hear about your porn habits. I would absolutely love it.
Such networks don't have to know whether or not and the exact moment a given user jerks it. Though, it would actually be better if you were actively browsing around and not jerking it currently. I guess that's why discovery and AI are so bad on porn sites. It's actually better for their ad revenue!
Edit: And presumably you're using Incognito for... some of those activities, which wouldn't be captured regardless.
They would be captured by the ISP.
Did you know that Google happens to partner up with a lot of ISPs? Hmm, I wonder what for. What could they possibly have that Google needs?
;)
Ever heard of https?
https://www.upturn.org/reports/2016/what-isps-can-see/
EDIT: spelling
* Enabling Chrome Sync, which is opt-in
* Syncing history, which is on by default if you enable sync
* Not using a custom passphrase for sync data (not using one is the default)
* Having "My Activity" save "Web & App activity", which is opt-out
* Having synced Chrome history data sent to "Web & App activity", which is opt-out
For the last two bullets, the opt-outs are at https://myaccount.google.com/activitycontrols .
EDIT: Added italicized text for clarity.
If you're thinking of Google Safe Browsing (used by both Chrome and Firefox), you're wrong.
It works the other way around: Google sends you the list of undesired domains, and your client prevents you from visiting domains found on that list.
Nothing needs to be shared with a third party for that functionality.
A hash prefix list gets downloaded locally; Chrome checks locally against the prefix list. If a URL hits, Chrome will send the hash prefix (not the full hash and not the URL) to the server, the server will send back all full hashes that match that prefix, and then the client will complete the check locally.
In theory, if the server had a small number of matching full hashes, it could guess about what URL a client might be hitting, but in practice the system is designed as much as possible to avoid ever leaking data about what you're visiting to Google servers.
Clients download a database of partial hashes of malware URLs. If they get a hit on one of those partial hashes, they make a request for the full list of hashes with that prefix.
Google knows when a client makes one of those requests, but the exact URLs (or hashes) they're looking up are never revealed. The partial hash is 32 bits long, so there's enough collisions that making a request isn't especially revealing.
For Safe Browsing protection, here's how it works (in progress): https://chromium.googlesource.com/chromium/src/+/refs/change...
[Disclosure: I'm the Software Engineer on Chrome who wrote parts of this Safe Browsing code, and that incomplete documentation linked above.]
I might be wrong, but this is obe of the reasons I don't use Chrome so if anyone has links that proves something else I'm interested.
> Another pro-privacy search rivals, French search engine Qwant, has also been added as a new option — though only in its home market, France.
"Duckduckgo is one of our main rivals." Is a bit of a self fulfilling prophecy for Google. They need to amp up DDG's legitimacy to ward off accusations of antitrust. Credibility, legitimacy and awareness are really the only things DDG needs to reach a wider audience and gain greater adoption.
[1] https:/duckduckgo.com/traffic
Maybe the creator really enjoyed that game as kid!
Weinberg explained the beginnings of the name with respect to the children's game duck, duck, goose. He said of the origin of the name: "Really it just popped in my head one day and I just liked it. It is certainly influenced/derived from duck duck goose, but other than that there is no relation, e.g., a metaphor."
(I know ... for business reasons ... but isn't Chrome open source? How is this in practice prevented?)
I guess the only difference is that with this change, DDG is available as a search engine by default with a blank install, even before you've actually used it.
You go to Settings > Manage search engines > Add or pick one of the auto discovered ones. This also works in the mobile versions of Chrome.
You can find more instructions here: https://support.google.com/chrome/answer/95426
just like Apple lets Pandora on Apple Watch and not Spotify.
Nothing new to see here.
I keep a running list of "alternative" search engines -- not directly Google or Bing, not Yahoo -- that work without JS or session cookies. There are thirteen in the list at the moment.
Takes user a few seconds to type https://www.google.com/?q=%s or whatever into settings.
Would that produce more meaningful "choice"?
Would users choose a search engine based on advertising or word-of-mouth?
Looks like Qwant uses Bing for ads. No ads if Javascript is turned off.
e.g.
curl -A "superagent/1.0" https://lite.qwant.com/?q=example
If get no results, try this first curl -A "superagent/1.0" https://lite.qwant.com/verify
Can get search suggestions in JSON via Opensearch: curl https://api.qwant.com/api/suggest/?q=exampleAlso quite interesting that they changed the default search engine for China to be Baidu.
It seems to be a logical move since I believe Google is blocked in China.