DuckDuckGo’s quest to prove online privacy is possible
wired.com
wired.com
- Local search is naturally very poor, to the point that 100% of my local search traffic is still on Google.
- Image search is nothing to write home about.
- I'm not pleased that looking up something contentious/political more often than not brings up reactionary or false content. It's either bad news sources or low quality blogs.
The last one might not be a problem to people whose opinions the results align with better but it is what it is. It's hard to say whether this is what the web would be without Google "censorship"/filter bubble stuff, or whether these are just the kinds of content the DDG (bing + other sources) algorithms have decided to rank ahead of them.
Either way, it is an invaluable resource and an important one to have. Search is already extremely concentrated towards Google but ideally there should always be usable alternatives.
Google Images has been deteriorating every year for over a decade now.
Another advantage of DDG/Bing is the ability to really turn off safe search. Unlike Google, it will not try to guess if you want nudity or not based on your search query, resulting in weird situations where you have to look for porn in order to get "anatomically correct" pictures.
That forces it off no matter what your query is.
I, err, only know this because of anatomy class homework.
https://www.microsoft.com/en-us/bing/apis/bing-web-search-ap...
(Can't find a source indicating that they're using this, but I'd be surprised if they weren't.)
https://chrome.google.com/webstore/detail/view-image/jpcmhce...
(not arguing for GI over DDG or any other image search, just pointing out that those UI issues -- which were coerced by publisher lawsuits, not Google's own preference -- can be trivially fixed with extensions)
Sadly, your link is a Chrome extension, and as a Firefox user, it doesn't really resolve my problem at all.
In my experience I had to add g! to 90% of my search to get useful results, except for the most trivial searches. It was so much so that I would often add one in the beginning and one at the end, resulting in my google query ending up as "search term g!".
I know not everyone shares my viewpoint but I wish people on reddit and HN would stop hyping up DDG. It will likely never be real competitor to Google. I don't mean to blame them in any way. Even Microsoft, with all their might, still fails to match Google's capabilities.
Some of it isn’t Google’s fault or course, all the hobby stuff that happens in the blood bowl community group on Facebook never ends up in a search engine, but some of it is certainly Google’s fault. I’m just not sure how much privacy helps though, sure DDG might not try to sell me diapers because their advertisers might not know I have a two year old, but it still tries to sell me honey.
Having saying that, yes, Google does work for me more than any of the alternatives and most of the times I can find even the most arcane and niche stuff I'm looking for, after investing few hours. Investing this level of efforts in DDG have never been fruitful and I have given up long ago trying anything but the most trivial stuff.
For example, it’s not using your location behind the scenes so “restaurants near me” doesn’t work. But swap to say “restaurants near ZIPCODE” and it’s fine.
I bring this up because oddly enough I had the opposite problem yesterday. When using a friends phone I just couldn’t find what I was looking for until I swapped from Google to DDG. From 15 useless queries to first try third result.
If it’s a frequent issue for you perhaps you could provide some examples? I only ask because I hear people agreeing with you and yet I have never seen an actual example.
On my machine, it's the !bangs that keep me hooked. I realize the results aren't quite as good as Google's, but it's usually just the top few results that seem to be key ones missing from DDG. I'm really surprised at the lack of popularity of SP, it's been my Google reference for a long time now.
DDG has my favorite image search.
Generally, I haven't used Google in years. If DDG isn't finding something, I use some other privacy-oriented search engine.
Has DDG published a statement about the whole deal? I've only seen the Microsoft's response ("oops something went wrong we only intended to censor a little bit") but I don't think DDG even said anything.
I do something both odd and inefficient for local business search: I open the Apple Map app, and restaurants and other businesses show up, especially when I zoom in. I live in a small town, and seeing a map (with satellite imagery enabled, of course) and zooming in on different areas reminds me of where I might want to eat, etc. Fun.
* Some spammy links slip thru-- these are dumb sites with lot of stuffed keywords and zero actual content. I dont get these with Google. I've usually seen these when searching for questions related to email list providers, like Aweber or Activecampaign
* Last year, I started getting random porn results for harmless queries, like "Monster videogame 2010". Again, didnt get these on Google.
For these reasons, Im hesitant to recommend DDG to non-technical users.
Why I stick with DDG-- I like to search in private windows, and Google harasses you everytime to accept their terms and conditions. On my iPhone, if I have 10 tabs open, I'll have to do this 10 times. DDG saves me a lot of this hassle.
For me, the privacy is a bonus, but like other commenters have said, there is no way to verify these claims.
so if I searched for "How to move my email list to Aweber" I would get a page that was a copy paste of the Aweber home page, with my search term randomly sprinkled in.
And it wasnt just 1-2 sites. Sometimes, the top 3-4 would be spam, and even Aweber's own help page would be pushed down.
Seemed to me someone was doing some sort of dynamic search stuffing which was fooling DDG, but not Google. But the dark SEO was so obvious, I'm surprised DDG (or Bing or whoever they get their results from) fell for it.
I've tried it a few times, this latest time I've been using it exclusively for a month or two and the results are almost completely worthless 95% of the time and I end up using the !g operator. In many cases the results have nothing to do with that I typed at all.
I've de-googled in every other way possible but Google's search engine is so much better than everything else I've tried that the rest are unusable.
I've heard it said before that Google's effectiveness is mostly due to the vast amount of information they have about me but that's not it. I've used lots of computers all over the East coast that I had never used before without logging into an account and Google gives good results every time.
I use incognito too for almost everything. However, I thought that the terms & conditions popup was only a Youtube thing (it drives me insane too)... I guess they haven't rolled it out to everyone yet.
I do much of my web browsing in private browser tabs. Why not? Even for things like HN, auto fill for login makes it easy and then when I follow links I don’t worry so much about tracking cookies.
It only happened for a brief time, so I guess it was a bug in DDG. But when it did happen, it was serious enough I stopped using DDG at work completely.
Bootstrapping themselves with different information sources as they've been doing is good for the very beginning as then they serve as a privacy-respecting aggregator to those sources, but as the tank man incident proved, you run the risk you'll be made complicit to censorship, whether intentionally or not.
Additionally, people engage with the first thing on the page about twice as much as the second thing, and that second thing about twice as much as third thing, and so on. That means if instant answers are near the top (increasingly the case) it means that those sub-indexes are increasingly more important as they are engaged with relatively more than the traditional “10 blue links."
A lot of that traditional web index is also used for just navigational purposes, which we can easily do ourselves. We are already crawling for our tracker blocking data set, encryption data set, and several other things.
In other words, we do make use of our own crawlers, and also have the ability to change some things around for indexes we use, which we do routinely. However, you need all of these indexes to make search work well, and have them all come up at the right times (a lot of what we do in search ourselves).
In this context, I think people elevate the long-tail “web index” too much. Should we have satellites in space to make good maps? It is also necessary for search. Should we be collecting sports and stocks data ourselves? Also necessary for search, but no one really calls on us to do that.
Our approach has been to partner with the best data providers in each vertical so we can provide the best alternative to Google search. And when nothing exists or it isn’t good enough, we do it ourselves or add layers on top. To that end, it isn’t binary — there is a lot in between and in fact that’s the way it works today.
Some specific things that would help me a bit (I don't know if they exist!) -- these may well be pipe dreams, but...
-- A decent search engine for mathematics in the form of LaTeX would be _amazing_...
-- I'd enjoy being able to pass parameters to bangs specifically. An example would be Google Scholar (alas still the best and the last part of their ecosystem I use -- doesn't seem to be as tied in to ads as the rest though) such as 'date from' and 'date to' parameters, with a sensible calling syntax, e.g. `!gsc;f:2015;t:- Convolutional Neural Nets`
-- Likewise, a customisable, ideally API-able scientific/technical search interface would be very useful (for me).
-- No big provider has tried to index the onionweb (I don't know if you're feeling that brave) -- I'd be interested to know
Finally, are you considering being a privacy-first cloud provider in your own right, out of curiosity?
EDIT: Also, back to the point: any plans when it comes to only relying on Bing for images? That was the issue in the end- least how I understood it.
Why did DDG shutdown the community effort to extend IA when you say yourself that IAs are DDGs leverage?
edit: DuckDuckGo did not buy Yippy. Yippy.com seems to be defunct and just redirects to DuckDuckGo.com
It's kind of sad that they shut it down. They actually had a decent index and a unique "category" thing on the sidebar that you could use to narrow down search results.
Ideally it would be continually audited by multiple trusted third parties.. something maybe like the EFF.
Even then there'd be no guarantees, but it'd be way better than what users have now, which is just DDG's claims.
For a for profit company, the third parties such as EFF would be good enough for most users. Would highly recommend that EFF offer some sort of certificate (PGP signed key would be fine) to authenticate such audit.
While there is no privacy audit that I'm aware of, the theory is that there is no motivation to sell my searching habits to anyone.
DDG plans such a feature, but it will likely never be their primary revenue driver.
They have almost infinite ways of identifying my right to use their engine given that they're the one emitting the Open Search snippet from within an authenticated window; uuid hostname or uuid path/query param are the two which spring to mind
They could sell your data for even more of a profit. Why couldn't that motivate them?
Or they could do it to comply with some jurisdiction's laws.
Or they could do it accidentally or as a leak from within.
The best protection is that no data is collected in the first place, but we need auditing to make the claim of no collection credible.
Two examples:
- While Neeva has no ads, they prioritize sites with tons of ads in their results (search for 'best laptop'). If your main premise is that ads are bad, why not penalize/remove second order sites that are heavy in ads?
- They claim they respect user privacy, yet make technical choices like using raw Google maps in their product as Maps module, which sends requests from the client directly to Google
Bonus; you won't leak your actual client IP to any of them. Comes with morty[1], which is a decent "anonymizing" HTTP proxy. Supposed privacy wins completely aside, it's the best search user experience I've had in many years. You do run the risk of getting CAPTCHA'd but so far I've had 0 of that with the above-mentioned.
* Google is very much better at finding solutions to programming errors. Duck Duck Go sometimes works fine, but sometimes it sends me on a wild goose chase (if you pardon the ... almost pun). This has always been a problem since I started using DDG.
* Duck Duck Go is poisoned by useless autogenerated results. If I want to compare two options, I don't want an autogenerated comparison. I want actual analyses and anecdotes from users. In that case, I normally add site:reddit.com rather than !g so I can't positively say Google is better but I believe it is. This started bothering me at some point, but it might be my fault.
In both cases, I think the issue is the same: Google is still better at prioritising conversational content over content produced predominately to get eyeballs near ads. This is slightly ironic.
Sometimes google gives a larger number of irrelevant results, so maybe they have more index reach, or maybe they’re worse at filtering garbage. I can’t tell.
I don’t use ddg for local search. I usually use yelp or (preferably) a privacy respecting mapping app instead. Google’s local results were garbage and often somehow more gamed than yelp’s last time I checked. Maybe they’re decent now. Don’t know; don’t care.
For that other 5%, it’s usually programming-related queries where everyday words take on new meanings. Since DDG doesn’t tailor search results, it has a harder time with some of my uncommon word usage.
For those, it takes two seconds to append “!g” at the end of the query to go to Google; it’s barely disruptive to my workflow. I’ve noticed myself doing that less and less since I switched in January. Perhaps a combination of DDG getting better, and me getting better at tailoring my queries for it.
http://techrights.org/2020/07/02/ddg-privacy-abuser-in-disgu...
Long time DDG user. Barely use bangs at this point. Haven't switched back to Google in ages.
I don't really do much local searching, but it seems like that's an issue. When I do local searches I just quantify things "Best Thai food in <My Neighborhood>", but I live in area heavily traversed by food critics (I tend to avoid group reviews like Yelp, and Amazon which I tend to think of as easily gamed in favor of newspaper critics and Consumer Reports or America's Test Kitchen, an approach with its own pros and cons).
Most things seem fine although the other day I got a ton of porn when I did an image search for an artist I was discovering "Rico Nasty". Thank god for work from home!
As an aside, I find Google's search result interface extremely off putting at this point. I'm not exactly sure what it is? It seems like there's barely any results above the fold, it's much noisier with widgets everywhere, and why are the description strings only a sentence long at max?
I downloaded a new copy of Chrome (or maybe it was the one that popped up from Cypress tests or something and I errantly used it as main browser window), and yeagh.
I try to use these alternatives to lessen the absolute power of google and Microsoft, not quietly boost them.
My fault, really.
I'm glad people are coming around to the fact that information is now controlled by a few giant tech companies.
As much as I hate corporations who resort to smearing their competitors instead of elevating their own offerings it looks like it does change public opinion and essentially works in practice.
I've had the idea for a while of writing for myself such a tool (probably can't be a browser extension?), but I haven't gotten around to it yet. Does one already exist, or is trivial to write?
As I type this comment, I found that at the commandline on macOS one can do, say:
open -a "Google Chrome" "https://duckduckgo.com/?q=umbrella"
open -a "Google Chrome" "https://www.google.com/search?q=umbrella"
function search() {
open -a "Google Chrome" "https://duckduckgo.com/?q=$*"
open -a "Google Chrome" "https://www.google.com/search?q=$*"
}
function search {
local query="${@}"
osascript -l JavaScript -e "
browser = Application('Google Chrome')
google_window = browser.Window().make()
ddg_window = browser.Window().make()
google_window.activeTab.url = 'https://duckduckgo.com/?q=${query}'
ddg_window.activeTab.url = 'https://www.google.com/search?q=${query}'
google_window.bounds = { 'x': 0, 'y': 0, 'width': 600, 'height': 600 }
ddg_window.bounds = { 'x': 600, 'y': 0, 'width': 600, 'height': 600 }
"
}
* Not opening the extra window if Chrome isn’t already running.
* URL-encoding your query.
* Auto-detecting the screen bounds (and Dock).
* Auto-detecting the active browser.
But I didn’t want to turn this comment into a full-fledged gist. As is, this works in Chromium-based browsers (replace the name in the `browser = Application('Google Chrome')` line). Adding support for all of them and Safari is simple: https://gist.github.com/vitorgalvao/5392178
Like most people I know, I switched to Google directly when I realised I just scrolled down to Google and clicked the first link.
At least on Linux it's likely doable with xdotool, it should let you identify the windows and put them where you want them.
Two iframes side by side?
https://stackoverflow.com/questions/23737427/how-to-put-two-...
DDG's results haven't gotten any better for that specific search.
"China banned DuckDuckGo in 2014 and we have no plans to change that. This image belongs in search results and never disappeared from our main page. For the image tab we mainly use Bing and they fixed the error. We’re looking to supplement w/ more sources."
I still say (by looking at the results for random queries) that ddg is is mostly bing, do you have any actual data to refute this?
Not opposed to it, just doesn't add up to the claim
Happy it bit them in the ass for the tank man image because they're constantly lying without lying about their complete reliance on Bing. Put differently, if Bing shuts down tomorrow, so does DDG.
>Please do a quick search here on HN and read up on all the previous conversations about privacy and security implications of that before selling the narrative that DDG is somehow more awesome than more established search engines.
I'm not sure I understand your argument. What are the privacy and security implications? If you submit a query and DDG turns around and gathers the results elsewhere, those other sources would have no idea it was you submitting the query; all they see is a request from DDG. Sounds like a better deal than directly exposing yourself to the sources you mention.
That isn't at all what I said nor implied. The gist of my comment was DDG is the lesser evil of all parties involved.
Your statement:
> DDG uses Google's and other search engines' data. DDG does not do their own crawling. Please do a quick search here on HN and read up on all the previous conversations about privacy and security implications of that [...] (emphasis mine)
You draw a connection between the potential third parties involved and the adverse affect to the individual's privacy and security stemming directly from their involvement. I was outlining that, if DDG was sending the request to those parties -- not the individual making the query -- the other parties wouldn't be aware of the individual unless DDG explicitly tells them (which I don't believe they do).
It's all about aligning your goals with the interests of the company. DuckDuckGo has built their business around being a privacy-focused search company. It is not in their interest to ruin that brand (and their company). Google is built on showing you ads and giving you free stuff in return (which requires they collect a lot of information). So yes, I think it's better (and quite natural) to trust DuckDuckGo a lot more than Google when it comes to privacy.
This would require allowlists and easy disabling for targeted searches but would generally stay enabled.
Whatever happened to Brave? Aren't they working on a search engine, too. Anyone know if that similarly uses Bing, Yandex, etc, for the search results or if they have their own crawler? It's been a while since I've looked in depth at user agent logs to see if Brave is making an appearance.
Overall, I've been finding it pretty good. It has information cards for various entities on the right-hand side, similar to all other search engines. You can additionally search for images, news, and videos.
I'd like to see DDG innovate more as the product seems stale at this point. 10 years have gone by and the product hasn't evolved much, feels pretty much the same. It seems they are in some kind of contractual Bing Search/Ads dependency lockdown preventing them from innovation in search. Also, are ad-based business models really what we want from the web?
On the other hand I can see them saying 'what we are doing has worked pretty well in the past, lets continue doing that".
Having said that, my main concern is that someday in the future DDG will be sold to some less ethical company and I'll have to abandon it (having to start the search for a new search engine of choice all over again).
That is a very easy trade off for more privacy.
Search options
[ ] No ads
[ ] Contextual ads
[ ] Targeted ads
Too bad they get so much money from Google. Otherwise this might be cool.
Your browser automatically crawls the pages you visit and adds the info to the distributed index that is also stored on all participating machines (perhaps with a copy on the cloud for query performance). Though a single browser may visit only a few pages, we ought to get a pretty good WWW index from the combined crawling of millions of browsers.
Main advantage would be resistance to censorship.
At the beginnning I would still go back to google for technical queries, but by now I have accepted that DDG is actually better at excluding spam/SEO stuff than google. Also the fact that it actually honours keywords like 'site:' 'domain:' is a huge time saver.
The difference may be regional or specific to my areas of interest so perhaps your mileage may vary but to me DDG has improved substantially in the past 2 years whereas google just seems to increasingly just push for this natural language paradigm and discard my specific query, showing instead whatever popular approximation it chooses (I can only assume this probably saves them a tonne of resources and caters to >90% of users).
The only feature I still feel google is unmatched is 'regional' online shopping discoverability. The 'Google Shopping' service is still the go to place for me to find specialist physical stores near me that also have an online retail front.