Use DuckDuckGo to improve your privacy online (2018)
spreadprivacy.com
spreadprivacy.com
From: https://support.startpage.com/index.php?/Knowledgebase/Artic...
More: https://support.startpage.com/index.php?/Knowledgebase/List/...
> Why does Google let Startpage access their search results?
> Startpage.com has a contract with Google that allows us to use their official "Syndicated Web Search" feed, so we have to pay them to get those results.
So what is their own business model? Do they show ads? (I disabled adblock but didn't see any.)
There's thousands of them
!w Wikipedia !hn Here :) !auk Amazon UK !so Stack Overflow
For me personally, Google is a less pleasant experience these days.
The DuckDuckGo privacy extension is currently the only type of "ad-blocker" I currently use. I really like the look and feel, as well as the functionality. The automatic downgrade from an A rating for the lack of an entry on https://tosdr.org/ is a little over the top for me though.
I'm very upset because this hinders my daily work and research. I'm starting to rely on Bing which is definitely worse than old Google but may be better than current Google.
I want to like DDG but it's normally missing article dates or other small features I depend on. I'll keep checking in to see if this changes.
Results are decent but I just hate that.
I tried to switch to DDG on mobile recently (using the Brave browser) and I found myself really missing those inline results.
Our search result pages may include a small number of
clearly labeled "sponsored links", which generate revenue
and cover our operational costs. Those links are retrieved
from platforms such as Google Adwords. In order to enable
the prevention of click fraud, some non-identifying system
information is shared, but because we never share personal
information or information that could uniquely identify
you, the ads we display are not connected to any
individual user."> Our search result pages may include a small number of clearly labeled "sponsored links", which generate revenue and cover our operational costs. Those links are retrieved from platforms such as Google Adwords. In order to enable the prevention of click fraud, some non-identifying system information is shared, but because we never share personal information or information that could uniquely identify you, the ads we display are not connected to any individual user."
I still find this weird because it's the exact opposite for me. I've posted this here several times, but to say it again: Everytime I use google, they honor all my forced terms. DDG, as soon as they don't have a lot of results, says "As we can't show you a lot of results, we decided you want something completely unrelated instead. Have fun."
Google also shows you which terms they excluded, DDG quietly drops them.
I like DDG, but when they made that change, it made me use !g more often.
edit: Because people tend to downvote me when I say this, here are examples:
Query: google "amp" modal
DDG: Google Maps SO issue, Google Sheets landing page, more google maps results. Only 7 is related by simply showing the AMP homepage.
Google: Only google amp results related to "modals" in some way.
And yes, the results google shows are actually in DDGs index, it's just that they decided they know what I want even though I forced the amp term.
I've found myself using double quotes more often now than ever before. Google just doesn't seem to understand what I'm looking for.
The growth of content marketing has ruined Google search results.
Ex: `libwacom huion gaomon`, missing libwacom in first result.
I tried switching to Brave and DuckDuckGo over the last six months, but found myself opening google and re-running the search so often that I recently gave up, and, with misgivings, reset my browser and search preferences back to google.
Duckduckgo just never found what I was looking for, and, in broader searches, would return relatively fewer results.
What Google does is to have context and to use that context to make the queries more specific. They use for example your location and your history of searches.
However being privacy conscious, I deleted my Google history and disabled the collecting of history in my Google account. And then the results became visibly worse.
This is important because when people complain loudly about the difference, they usually have a search history in Google going back a decade, reflecting a trail of embarrassing moments of course that most couldn’t make public. Not many people are curious to look at that history, although to Google’s credit, they do expose it in full detail and a nice interface.
With DDG you just have to be a little more explicit. For example searching for “ruby” won’t yield results related to programming (except for ruby-lang.org), but searching for “ruby programming” does. So in general you just have to be a little more specific and we’re talking about a word or two.
Nowadays whenever I can’t find something on DDG on my first try, I’m confident that I won’t find it with Google either. And lately I feel like DDG is better, maybe due to my changing search patterns.
The real gain is privacy and this reflects in the searches you’re doing. I can’t convince myself to search for medical conditions on Google anymore. What if I’d get classified as a diabetic (I’m not)? What if that data leaks and this affects my credit score? No thank you.
my guess as to the reasons for this particular failure of !G is that, because i have been gradually shifting to fastmail for the past 1.2years and the chain of messages between myself and the company i was looking up never managed to make it from linkedin --> gmail.
Moreover, I'm not convinced it's entirely due to my search history, location or other online history. It's definitely not search skills, or inspecificity of search terms that's the problem.
IMHO, it's mostly poor coverage. In my casual experience, it's particularly evident in academic journal articles, but there's definitely poorer coverage in general. Even when I know the web page I'm looking for, I can't get it to show up in the search. Or, as I mentioned, a blanket search will bring up a page or so of relevant links, compared to many pages from google.
The elephant in the room here is Bing. Bing is an objectively worse search. There's no way around that, and no ideological dressing that can change the fact.
I've brought this discussion up because at that time (4 months ago) the results on DuckDuckGo were worse than Bing's: https://www.reddit.com/r/duckduckgo/comments/9j5f1p/duckduck...
But nowadays they are better for that same keyword. Whether somebody on their side noticed my issue on Reddit and manipulated this particular result, I don't know. But they definitely aren't just a shell over Bing.
Google provides the best results in this case, without doubt. For a software company of their size, hiring some of the best engineers and specialized in ML, I wouldn't have expected otherwise.
I still use Google's Search for double checking whenever I don't find something, but as I said, I increasingly find that for my search queries Google can't help me in finding anything that DuckDuckGo can't.
I don't search academic papers so of course we have a different experience.
---
Btw, I don't think caring about privacy is an ideology. I've worked in the advertising industry and I still do, although I'm on the publisher's side now.
Companies are carelessly trading user data and if that happens to me (1) I need to be informed and (2) I need a substantial reward in return to outweigh the risks and as long as there are cost effective alternatives that protect privacy, I'd rather use those, voting with my wallet and all that.
Money quote:
> We also of course have more traditional links in the search results, which we also source from a variety of partners, including Oath (formerly Yahoo) and Bing.
The other 400 sources are just used to widget stuff while the organic links are mostly from Bing and Oath. So it's a mash-up.
For example, just now, I came across "ED survivor". No idea what "ED" means here (it's eating disorder).
I just put it into DDG and got nothing on the first page that looked like it could be pertinent: A user name "ED survivor" on Instagram posting about vegetarianism, Ed Stafford on Wikipedia, "ED: The Survivor" (a mod for a computer game evidently), Devil Survivor Full HD (a film obviously, no idea what ED means here), a few more in this vein, until the page ends with the TV series Survivor on Wikipedia.
Google: First hit "Template:ED survivor" in Wikipedia (and that page contains the words "eating disorder". Second hit "Eating Disorder Survivors Wall", third hit "Surviving ED" from HealthyPlace (so it's health-related – and the snippet shown on the result page starts with "Eating Disorders recovery is a long road.).
The rest of the results on Google are all similar, all of them(!) clearly tell me what ED means in this context.
DDG gave me crap, none of the results(!) came even close to answering my question.
This is not a singular occurence, just a thing I wanted to know a few minutes ago. I'd love to take DDG's search results, but when not narrowed down to specific origins with bang commands, the results are usually worthless.
Edit: and before people tell me that I should have used quotation marks around my search terms, great, let's try that: Two results with "We would like to show you a description here but the site won't allow us.". From the top three results, one times out, one redirects to a sports betting site, and one to a product in an online apothecary (nothing to do with eating disorder). Then penis extension. Erectile dysfunction, something general about mental disorder, and again two results about penis enlargement.
Whatever the reasons may be for this decision, it allows DDG to track all links that the user follows from the results page. That is no better than Google. It is not a privacy improvement in that regard.
If the user submits the request via POST, then the results are not prefixed. For example, if the request is submitted from an html form with method set to "post".
I submit that allowing users to GET (clean) results without using Javascript would be a privacy improvement over Google. Not to mention that it would please many users who get frustrated when trying to copy links from a Google results page only to find they have been prefixed and loaded with Google url parameters.
Users deserve clean result links.
To see the difference between prefixed and non-prefixed result links:
curl -o a.htm https://duckduckgo.com/lite/?q=example.com
grep -m1 .=result-link a.htm
curl -o b.htm -d "q=example.com" https://duckduckgo.com/lite/
grep -m1 .=result-link b.htmIt's meant so that the sites you click on can't see your search queries via the Referer HTTP header.
You can turn that off in DDG's settings[1] under the privacy tab. It's the "Redirect" switch. They explain more about it[2]. Basically, it's a privacy choice they give you between preventing sites you visit from seeing your search queries or preventing DDG from seeing what you clicked on.
If you want, you can prevent both by enabling their settings option to use POST instead of GET in their search. Then you can disable the redirect and the sites won't be able to see your search query from Referer, because it would have been passed through the request body instead of URL parameters. However, that's going to prevent you from being able to have your searches in your browser history.
EDIT: I now note why you mentioned javascript and why the Referer option talks about being needed on old browsers. It seems that when javascript is enabled, they do something else to ensure that Referer doesn't leak search queries without passing info on what you clicked on to DDG. That means that, by default on a typical browser, both leaks are prevented. That's pretty neat.
1.
Privacy conscious users can control whether a Referer header is sent and if so, the content of that header.
They might choose to use a browser that does not make it easy to control the Referer header.
But DDG does not account for those privacy conscious users who may be using browsers that do allow this control. (Bias discosure: I use one and not for privacy reasons.)
The DDG decision seems to encourage users to choose browsers that have privacy-unfriendly default settings or that do not make it easy or even possible to control what if any Referer is sent.
At the very least, it tries to resolve an issue that is a clearly client-side one and is best resolved there.
2.
Correct me if I am wrong but storing "settings" with DDG requires a cookie and/or Javascript to be enabled.
If this is true, then again this is not an improvement in privacy. Privacy users may prefer to keep these browser "features" disabled wherever possible.
Users can of course control whether they send GET or POST by using a client such as curl or the like, or sending requests from a local copy of a DDG html search page with the method changed from GET to POST.
One might wonder why DDG cannot provide an alternative search page with its form method set to POST. Users can then choose which search page they prefer. No cookie needed.
Instead it sets the default method on its search pages to GET (perhaps leaking the query in the URL).
How many users will bother to change the default? A minority of them, no doubt.
There is an opportunity to let users turn off cookies and Javascript and let them perform searches using POST, the method usuaully advocated by those advancing internet privacy.
I cannot see that it would cost DDG anything substantial to provide it, except not promoting GET as a method by default, not requiring users who want to change defaults to enable cookies or javascript to "keep state" and reliquinshing the ability to easily track which result links users follow.
To reiterate, control over the Referer header is easily done by the http client. Clients that do not allow easy user control over this are not improving internet privacy.
In my experience sending Referer headers is almost never necessary. It is very rare that I find a site that requires a Referer header, let alone a specific one.
Sending gratuitous information seems like a clear departure from the internet privacy playbook. Then again, I am not a privacy expert and perhaps the experts have decided there must be a carve-out for the Referer header and users should err on the side of sending too much information rather than too little.
The only reason I am aware of this stuff is because I use the web heavily without javascript or cookies and I take an active role in what headers I send. I generally do not do this for privacy, but for speed and control. I can strip out the prefix client-side if I must use GET.
> The DDG decision seems to encourage users to choose browsers that have privacy-unfriendly default settings or that do not make it easy or even possible to control what if any Referer is sent.
I can't follow your logic. That DDG provides their own method of circumvention for information leakage through the Referer header does not mean that it encourages browsers that don't have such features. That's like saying that a building that provides facilities for the disabled is encouraging people to become disabled.
> At the very least, it tries to resolve an issue that is a clearly client-side one and is best resolved there.
And you believe that that's going to happen? In a broad sense, DDG's goal is to promote privacy. The more it can help people in keeping their privacy, the better. I for one, don't control the Referer header in my browser even though I could because I'm lazy, and I appreciate DDG helping me a bit in that regard. However, I imagine that many users of DDG don't even know what a Referer header is, why complicate their search for privacy when DDG can help them?
> Correct me if I am wrong but storing "settings" with DDG requires a cookie and/or Javascript to be enabled.
> If this is true, then again this is not an improvement in privacy.
Why? You give me the impression that you think that any use of cookies requires the ability to track a user, but that's not so. A cookie is merely data that is passed with each request. If the server receives a request "please give me search results for foo and take the following settings into account...", that's not really privacy invasive. The server is not tracking you and remembering you, you're simply providing your preferences in each request as if it were the first time that the server had ever met you.
> One might wonder why DDG cannot provide an alternative search page with its form method set to POST. Users can then choose which search page they prefer. No cookie needed.
It does. It's in the settings. It probably does use a cookie, but there's no privacy reason to avoid it.
> Instead it sets the default method on its search pages to GET (perhaps leaking the query in the URL).
> How many users will bother to change the default? A minority of them, no doubt.
Like I said in the comment you replied to, the default doesn't leak to either. In the second link I shared, DDG explains how it uses a meta tag to set the URL that the browser should use for Referer. As I understand it, it doesn't even require javascript to work.
curl -s https://duckduckgo.com/lite/?q=example.com | grep -m1 "'result-link'"
curl -sd "q=example.com" https://duckduckgo.com/lite/ | grep -m1 "'result-link'" diff a.htm b.htm
My working directory is always mounted tmpfs. This is how I keep things fast and clean.
curl -sd "q=example.com" https://duckduckgo.com/lite/ | xmllint -html -xpath '//a[@class = "result-link"]/@href' -
curl -s https://duckduckgo.com/lite/?q=example.com | xmllint -html -xpath '//a[@class = "result-link"]/@href' -I don't think that DDG is trying to have independently auditable level of privacy guarantees. What they are selling is trust. The trust that major tech companies have squandered over the last decade. I trust a couple guys sitting in an office in Pennsylvania somewhere way more than the faceless behemoth that is Google at this point. Who knows if that trust is worthwhile, or if things won't end up going the same way, but I'm on their side for now.
For example:
For example, searching: "word x" "sentence a" would prompt Google to show only pages that include both the word and the sentence in the same page.
Its actually part of the syntax of search queries for google. That said, it still isn't always the best results.
- they use link clicks to determine coherence between query and result. More users => more clicks => greater coherence
- they fund themselves using advertising, so their revenue is roughly proportional to their number of users. Increasing users => increasing money => more money to devote towards improving results
If one is interested in improving the results of DDG, then providing them with free advertising is an easy way to do it.
Do you have a source on that?
I think products that tend to align with hacker values (transparency, privacy, etc.) are fair game.
Didn’t DuckDuckGo switch off of OpenStreetMap to Apple Maps recently?
you might want to try occasionally using !sp instead of !g -> same results, no google.
Google knowledge graph is years ahead of DDG. Average person does not want to click through results, s/he just wants the answer. Compare these queries in Google vs. ddg:
- nba scores
- 2018 super bowl
- lakers roster
- movies in theatres
- game of thrones cast
- $aapl
- where is my phone
- 2019 thanksgiving
Scrolling a bit or even clicking on "More results" is not that big of an inconvenience for me to completely drop them. I will admit though that when a search fails for me, I may at times check the same search query at Google, but I now use Google for like 10% of my searches or so.
As for just "getting the answer", I suspect you're right most of the time, but I really don't mind clicking through the first few results links to see the answer. Figuring out the best answer and putting it in-line with the search results is a nice-to-have for me, but certainly not essential.
Also note that I'm making a trade-off here: lower search result quality in return for not having my search history tracked. For many people, I imagine they don't care about their privacy as much and will make the opposite trade-off. That's fine; I'm just glad we have good enough options so people can make that decision and not be forced one way or the other.
If I'm searching for a place to eat near my house I use google, if I'm searching for WSUS error code 0X8024401C I use DDG
Even shorter to type. ;)
I don't use or follow Facebook that much, but every time I hear Mark Zuckerberg defending Facebook's business model, he always says something like "we want to make sure the ads you see are actually useful/relevant to you". Is there a way to achieve this outcome without taking into account more personal information?
If you make a query on a search engine I'm sure we can find a way to have better results without the search engine retaining your private information. Either there is some kind of third party you can temporarily give your info to and trust it to forget it immediatly after giving you the results. Or the search engine could give standard results and your data broker then filters it using your info (which doesn't leave your pc).
I used to think of this data broker as some sort of AI but now that I think about it maybe a simple browser extension that filters Google search results or augments your DDG queries might do the trick.
AFAIK, no one has conclusively shown that highly targeted ads perform better than interest-based ones (as in, putting ads for cars in car magazines). In fact, I'm under the impression that "we don't really know what we are doing" is the ad industry's secret.
The Atlantic calls this Wanamaker's Paradox[1], in honor of a retailer that said "Half the money I spend on advertising is wasted, the trouble is I don't know which half". If DDG is right and showing ads based on your search terms is enough, then you'd expect them to get ahead of Google, because they would have similar profits with less overhead.
Of course, there is much more to Google's dominance than "because they show better ads", so I don't think "efficiency gap" is the best way to look at it anyway.
[1] https://www.theatlantic.com/business/archive/2014/06/a-dange...
So DDG is now all around the better choice.
I love DDG for search but not sure this browser competes with 1Blocker X Safari integration on iOS.
Also it has "bangs" which give you shortcuts to lots of other services. https://duckduckgo.com/bang
I was just curious what benefits you get from switching to this browser vs. using a content blocker with Safari.
But here[0] are the examples of sport and movies/tv shows that i was talking about.
Only recently learned this from another HN commenter around here.
"weather: toronto" instead
People saying they respect your privacy, but... how do you really know? Also because "improved privacy" != "no targeted ads". They can still do massive amounts of money from the data we provide.
At some point:
1) they would have to comply with some law somewhere, handling user data (logs). 2) they may become the number 2 search engine. Spammers will target them and they would have to start doing some behavioral analysis on "what links worked well to those kind of users" to fix that.
Are they so transparent and well intentioned? Is their system so simple, so innocuous and filled only with anonymous data? Fine, they should Open-source the engine and servers, prove that they are using the same code we see, give us RO access to the DBs, and the equivalent of a MacDonalds kitchen tour.
Otherwise, they are just relying on nice words.
You don't really know. However, when your entire USP is being privacy-friendly (and really, that's the primary feature that Google can't copy), it would be incredibly damaging if word got out that that wasn't true. And we can be reasonably certain that, if it wasn't true, word would get out.
Also note that DDG explicitly says they don't do user tracking. So while ads are targeted in the sense that you'll see ads relevant to the search term you're using (and your location and other data attached to the query), they're not keeping a profile of you and targeting ads based on that. In other words: there should be no significant user data for them to hand over.
I'm not sure why they would have to become the number 2 search engine at some point, although regular search engine manipulation does already seem to affect them sometimes. I don't see why they would have to track users to combat that.
Sure, open source would be nice. But right now, we do have reason to assume that they're at least significantly better than Google, without having to sacrifice a lot in search result quality.
> word would get out
> no significant user data
These, as you say, are assumptions. That's why I don't (still) buy it.
They are in "sell" mode right now. They would say anything to gain traction, and it's working. That's why they are pushing hard and investing a lot. Remember it's about making BIG money through an advertisement-based business, what could possibly go wrong?
Imagine that you are magically handled the DB for the searches done today. What would you can possible learn from that information? Perhaps you can't track what searches did HN user Vinnl, but the info has surely a value. They may not be honest about what are they doing with that information.
Another assumption I make is that I don't expect then to seriously displace Google any time soon,but rather remain a player that fills the niche of private search. If that ever appears to change, we can always adapt - let's not make perfect the enemy of the good :-)
It claims to block trackers like Google and FB, and provides other privacy related features.
Does not say what it's built on... Probably chromium.
The search results from DDG just feel “off” compared to google search. I want to like DDG and I keep attempting to use it; but it’s failing at making relevant information immediately available at my fingertips.
- Privacy
- It doesn't try to be smarter than me
More often when I use Google to search (e.g. for research), it decides to ignore one or multiple words, yielding irrelevant results. It's fixable, I can quote them, it even offers to do that, but it drives me nuts.
EDIT to add: I think nobody disputes or cares that most results come from (or equal) Bing results, as long as they're good enough.
[1] https://i0.wp.com/fourweekmba.com/wp-content/uploads/2018/02...
It's possible to serve ads without being creepy if that's what you're asking.
I thought they "Do No Evil," were all about corporate and social good? That's the image I once had!
I love how commercials "follow you around". If I search for diamond rings in a browser on one computer, they suddenly materialize everywhere.
I tried it with Duck-Duck-go, searching for something I nearly never search for, something such that the ads follow you around. I just searched in DDG, but did not go to the end website. I then resumed non-DDG browsing, and those came up as ads. I don't know if they are looking at image caches, or loading javascript libraries.
They have a way around it already, and it runs.
This is all speculation (though informed by the Snowden revelations), but if the NSA asks DDG for data for some legitimate reason, it is likely that the NSA would just demand "all" the data, and not a scoped version of it. Meaning your information could get caught up in it.
I agree that for most people this may not be a concern, but if you would like to learn more, privacytools.io has some more information about the "14 Eyes" and other nation state information gathering efforts.
Edit: Okay, you can't really have evidence of absence, but good reasons to believe they don't log?
Though I have no way of verifying they don't keep logs, if I discover they do: at least I can sue them for it.
Yandex has a pretty decent search engine, and also helpfully provides a link to Google / Bing too, in case one isn't satisfied with their search results. This is smart of them as they still get data to improve their search engine, but also don't leave their users disgruntled when the search results aren't relevant.
Two reasons:
1. The US is a functional democracy, with a bill of rights and privacy laws which aren’t perfect, but exist; California even has its own version of the GDPR
2. They have strong trade agreements with the EU and if you’re an EU citizen this matters, as companies found to violate the GDPR will get screwed; GDPR which also makes the export of data very complicated, so US companies are increasingly keeping EU data on EU servers
Speaking as an EU citizen, the NSA is not my enemy. But companies like Facebook are. So for me a company like DuckDuckGo is still acceptable because without privacy, they offer nothing of interest, so they won’t do anything to jeopardize their reputation.
Speaking of which, DuckDuckGo can exist in the US, but not in China or in Russia, both being jurisdictions that are ... challenging, to say the least.
If you insist on not using US services, then the only alternatives worth looking at are in Europe, because we have strong privacy laws and we aren’t being led by dictators. And you’d better not look at a “five eyes country” ;-)
Here’s one: https://www.qwant.com/
No, it is not. By using non-american alternatives, you ensure fragmentation of your personal data, thus making it harder to build a better / more accurate profile of you.
Moreover, I personally don't like the fact that using DuckDuckGo still helps the other major search engines - they still get data from DDG that helps them improve their search engine, thus contributing to their monopoly.
> The US is a functional democracy ...
Good for the Americans, then. Not for us who live elsewhere. The US government and courts have repeatedly applied different legal yardsticks when it comes to a citizen vis a foreigner.
> as companies found to violate the GDPR will get screwed; GDPR which also makes the export of data very complicated, so US companies are increasingly keeping EU data on EU servers
I don't see why you think this doesn't apply to Chinese / Russian businesses too. Ultimately if they want to do business in EU or elsewhere, they will have to follow the law of the land. Right?
> Speaking as an EU citizen, the NSA is not my enemy.
I disagree. The Five Eyes spying program is designed to indirectly spy on the citizens of their country using each others government agency, and sharing that data.
> Speaking of which, DuckDuckGo can exist in the US, but not in China or in Russia,
And this brings another important aspect on this debate - we shouldn't be trusting private companies to protect our privacy rights and instead should be pressurizing our own government to legislate laws on this.
Most Americans somehow seem to naively believe that Apple and DuckDuckGo (for example) are working more to protecting your privacy, whereas all they are doing is slowly trying to gain your trust to get you to share more data with them. And ultimately, like Google betrayed all of us and went fully "evil", so will Apple and DuckDuckGo.
(And notice that they have already started being more invasive).
for the rest you're only one !g away from the usual