DuckDuckGo on CNBC: We’ve grown 600% since NSA surveillance news broke
technical.ly
technical.ly
Add in the !bang feature for searching most websites (classics like !w - Wikipedia, !g - Google, and stuff like !gh - GitHub, !aur - Arch User Repository) and my favorite "define X" keyword that links straight to Wordnik, and my search experience is better than Google.
The !bangs also function as bookmarks, so if I ever want to go to GitHub, I can just search !gh and it'll take me there. It's like having a set of search engines stored universally, accessible from any device with web access.
And of course if I need Google, say for word etymologies, it's just a !g away.
Down side to that feature is when you search a random blog once and then two years later it's suggesting you search there again.
Both Firefox and Chrome use OpenSearch (http://www.opensearch.org/Home) to achieve this.
I can directly search Audible.com (a site I've never used before) with an intuitive !audible. Then I never have to think about this shortcut ever again, because it isn't saved in my browser somewhere.
Although pointless, it's fun to use my keyword "ddg" to use their bang syntax to search another site. Even if I already have that site as its own keyword.
ie:
There is no difference between me searching "y this searches youtube" and "ddg !y this also searches youtube" or having to actually visit ddg to search Youtube. There's no reason for me to not just use a feature already built into my browser.
Plus I make ddg my default browser so I do just go to the address bar put `!yt cats` and get youtube cat videos.
!h searches Hoogle; I would prefer it to search Hacker News. To search Hacker News I have to use !hnsearch or !hn
!hn isn't much larger than !h but is still something I would prefer to set myself.
!m goes to Google Maps but I would prefer to use another map service. I could use !mapquest but would rather it just be !m
!sd goes to Slickdeals instead of Science Daily (which is !sciencedaily , talk about lengthy)
E:
I just compared my keywords and then searched the DDG Bang list for where I would be taken if I had used DDG instead of ones I set myself.
E:
>Plus I make ddg my default browser so I do just go to the address bar put `!yt cats` and get youtube cat videos.
I just type "y cats" and get youtube cat videos. ;)
>!hn isn't much larger than !h but is still something I would prefer to set myself.
>!m goes to Google Maps but I would prefer to use another map service. I could use !mapquest but would rather it just be !m
>!sd goes to Slickdeals instead of Science Daily (which is !sciencedaily , talk about lengthy)
It sounds like this is a prime use case for text shortcuts such as those built into OS X or Textexpander etc…
Just create a shortcut to transform '!h' to '!hn' or '!sd' to '!sciencedaily'.
>Good thing neither of us have to agree on what keywords to use.
Precisely! :)
What do they think that icon to the right of the tabs is? Just some bit of abstract art? :)
It's a lot like Pressing F11 without the issues that arise when using Full Screen browsing.
>There's no reason for me to not just use a feature already built into my browser.
In Firefox you have a dedicated search bar (ctrl-k) which remembers your search across locations. So you search for "heisenbug oracle ipv6" press enter and are not immediately happy about the precision of your search.
So you press ctrl-k again and append "!so". Bang, your previous search is now applied to stack overflow and you have your answer straight at spot 1.
It's a very good flow. It will never work in retarded browsers who insist on removing the search bar though (like Chrome, Safari, IE).
In a deeply misguided act of Chromeism, Firefox was considering going in that direction too, but the outrage in the userbase hopefully caused them to never venture that line of thought again.
And copying the search text in the URL bar actually copies the link too, which is nice.
Uh huh. Because they'll lose their ability to tell if something is a search or a URL because of bang parsing, right?
Yes.
In the past that page would even discriminate searches made with DuckDuckGo. They recently redesigned the History page to fit their Material Design look.
I would hope that when I'm logged out, Chrome would not track me... And what about Chromium?
There's several more settings under Advanced Settings -> Privacy: "Learn more" links to https://support.google.com/chrome/?p=settings_privacy
-Use a web service to help resolve navigation errors
-Use a prediction service to help complete searches and URLs typed in the address bar or the app launcher search box
-Prefetch resources to load pages more quickly
-Automatically report details of possible security incidents to Google
-Enable phishing and malware protection
-Use a web service to help resolve spelling errors
-Automatically send usage statistics and crash reports to Google
All of these send info back to Google and can be used for tracking even when logged out. Not to mention the usual tracking in any browser. And this is just what they are up-front about.
See: https://support.mozilla.org/en-US/kb/how-does-phishing-and-m...
Prefetch makes local predictions and connects to any site, not Google in particular AFAIK.
The omnibar prediction service uses your default search engine, so Google is only involved if you've left it as such.
So if you take the obvious steps of changing your search engine and signing out of Google, that leaves only the navigation error service. As with safe browsing, this appears to send URLs to Google only if certain local conditions are met.
Google still seems to be lightyears ahead of everybody else as far as understanding actual content meaning and intent though. It's fun to try to give google a query it can't return results instantly for.
So far I'm pretty happy with DDG results.
[1]: https://addons.mozilla.org/en-US/firefox/addon/duckduckbang/
It took me a while of using DDG and looking at results side-by-side before I could reliably distinguish between "results aren't as good" and "results don't feel like Google results".
For example, I kept track for a little while of how many times I had to use !g. I changed my DDG theme for a while to use colors similar to Google, and found myself using !g less often. That has disturbing implications for just how conditioned I was towards Google search results. (These days I use the default DDG theme.)
Apart from that, I reported various minor cases where DDG's rankings needed fixing (many of which got fixed, with actual email from a human about the issue).
These days, though, I almost never use !g. In the rare cases where I get annoyed with a set of search results not showing me anything useful, I tend to find that Google's results aren't any better.
One area I do wish DDG would fix: showing calculations directly without having to use !wa. For instance, just today I searched for "2^64 picoseconds in years" on DDG, and that didn't instantly give me the answer; I had to repeat the search with !wa. Google gives the answer at the top of the search results.
This is a big problem when I'm doing some research. I don't want links from sites I frequent. I want links that actually match my query.
It's not my primary search engine, but I do find myself turning to DDG from time to time
It's maybe once every couple days that I have to use "!g" to get google results, for everything else DDG works excellently. Even the times when I have to use "!g", it's often a hint that I'm searching for an unpopular phrase, and find that if I rephrase my search results I get much better results out of both search engines.
I remember there was a story on HN once a few months back where a kind soul from DDG posted an email address that one could submit notes to wrt highlighting poor search results so that they could address them, I don't recall the email and haven't been able to find it. If this is still available with DDG could someone please re-post that email here? I would very much like to help improve the quality of DDG to make it better for everyone but I can't find anywhere to suggest improvements on their website.
Edit: I was able to find their Feedback page, but I much prefer email personally: https://duckduckgo.com/feedback
I have—it's about 80 – 90% accurate for me, which is pretty good but not always perfect. The search-within-site feature (i.e. "search string site:jakeseliger.com" is definitely not as good as Google, though I'm not sure why. Searching within Reddit or HN, for example, is not nearly as good.
That being said the !g button is great: if DDG doesn't work, the move to Google is fast and painless. I've definitely evangelized DDG some to friends.
Again, if ddg doesn't give me what I want I use !g to try google.
As a user I don't just want to find it, I want to find it easy. For example, the most (/more) relevant answer should be ranked 1st, not 12th on the second page. Also, a lot of times I'm trying to solve a difficult question - so the 'closer' it takes me to the real solution by combining more than one 'top' search results, the betters is the search engine quality. Also, I love how Google answers a lot of questions without visiting any website at all (on the top with its cards). I know it might be bad for the website whose information is probably being scraped, but it's better for me as a user.
Google's answers are hardly black and white--part of being a developer, an engineer, or even just a high-functioning human being, is being able to synthesize an answer from several data sources of varying utility.
One could easily replace the word "Google" with "DuckDuckGo" and be facing the same quandary. In fact, having just realized that (I was in the same boat as you earlier today) I'm going to give DDG a shot and see how it holds up.
My main concern is that it's still a free service and I really don't see how it'll be sustainable in the long term without compromising privacy in their current model. If you're not the customer you're the product etc...
I'd gladly pay $10 a month for a "premium" search engine with strong privacy garantees. I'm definitely not going to enable ads in ddg and I can't imagine that the average duckduckgo user thinks differently.
Right now it seems they're still growing really fast. What if tomorrow it's not the case anymore? What if their userbase reaches a plateau? Will they really still be morally strong enough to commit to user privacy or will they slowly start to give in more and more in order to increase their revenue? Does "do no evil" ring a bell?
On the other hand if they get money from the users themselves then the incentives are different, they might try to get more money by offering new services and they probably won't risk doing that that'll piss off their paying users.
Maybe I'm too cynical but in my mind even if ddg's intentions are pure now eventually they'll get bought or something and someone, some day will want to cash in. And for a free service cashing in means selling the userbase to the highest bidder.
But maybe I'm a bit naive, I guess that would still be possible even with paying users but I think the risk would be greater for them.
Also, at least on their settings page, they encourage donations or the purchase of their gear in order to deal with costs. Nevertheless, I can see how keeping up a free service could be a problem.
Most have servers in data centers in 10+ different countries, so I doubt they are all closely monitored nor capable of keeping their keys closely guarded.
Most VPNs like to advertise the 'military grade encryption' or 'no logs' nonsense, I'd rather see them post results of security audits, ideally conducted by outside firms.
It probably doesn't matter because 99.999% only want to access Netflix in another country. Out of the remaining 0.001%, most are just people from repressive countries dodging their country's porn filters.
If they are giving people the service they want then how is it a scam?
There are loads of trash VPN providers, but there are a lot of genuine ones that really care about privacy activism and provide an honest service.
I started out thinking "theory and reality", but, even in theory, adding a ton of very slow, latent, and unreliable data sources to your query doesn't produce a fast result.
I encourage people interested to look at "Peer-to-Peer Information Retrieval: An Overview" in order to get a better understanding on what technical challenges distributed search engines like Yacy have yet to overcome and the paths researchers are taking to overcome them.
However, an adversary can also fingerprint your OS/patch level based on packet structure[1][2] - which can't be easily changed. This will add more identifying bits.
1.http://nmap.org/book/osdetect-fingerprint-format.html
2.http://www.windowsecurity.com/articles-tutorials/intrusion_d...
https://panopticlick.eff.org/browser-uniqueness.pdf
All in all it's pretty scary though. My browser is unique among 5.3 million or something. And then there's the algorithm they mention that can detect fingerprint changes with 99.1% accuracy, and be used to do things like recreate cookies that you deleted.
On the other hand, there is a lot we can do to protect our privacy from corporations. I do simple things like using duckduckgo, using a private email service that I trust, only using chrome for social media access, and setting Firefox to be as secure as possible and drop all cookies every time I close it.
I read the comment prior to the username and thought your description of browser use sounded familiar, then checked the name and wasn't surprised that it was your blog where I saw this recommendation.
Do you still use Ghostery with Firefox and Fastmail for email?
I really enjoy your blog btw.
I should probably have tried more but I guess I'm too pragmatic for my own good.
Google still wins when you don't know the word/term/name etc for something and are describing it. The reason for that is mostly that every time someone searches on Google, they help make Google smarter. Effectively every time you search, the results are largely based on what someone searched for historically and what they clicked (or even subsequent searches and clicks).
e.g. "movie about kidnapped daughter" (I'm looking for the 2008 movie Taken, but have "forgotten" the name).
Google: 3rd result. Bing: 8th result. DDG: 7th result.
Why is Google 3rd and DDG/Bing 7th and 8th respectively? They likely have all indexed the Wikipedia and IMDB pages for the movie Taken. However Google likely ranks those links closer to the top because historically that's what users want when they searched for that phase.
By itself, one data point like that is extremely weak evidence; the later queries might actually have been completely unrelated. But that's a source of error that tends to disappear when averaged across a large number of different users. In the aggregate, the data can be extremely valuable. But doing that kind of analysis requires correlating multiple searches from the same user, and storing the resulting profile for a long enough period of time to do useful aggregation.
This is a search problem. I honestly don't see why DDG can't develop an AI that understands cultural and language contexts. Currently it doesn't but innovating this will take a lot of brain power, something DDG is lacking (when comparing the sheer #s of personnel focused on search).
1) Smaller index. Some stuff just seems not to be in DDG at all. Even after finding a page I'm looking for via !g, I'll go back and try to see whether I could've gotten DDG to return it with some more specific search term (even verbatim phrases from the page) with no luck.
2) Languages other than English. The Google/DDG search quality gap is much, much larger in Greek than in English, for example.
I know that the interviewer has to ask at least the first part to get the interview going, but it also highlight everything that wrong in the thinking of online ads/marketing.
If you need to collect user information to make money, them perhaps your product isn't that great to begin with (unless you're an ad company like Google, but the we get into the argument who the user is).
Also collection information, so you can grow to become a big brand, I would argue that you've thrown trust out the window a long time ago.
Being based in the US is a dealbreaker for privacy.
If you want your searches to not be used for targeted advertising, DDG sounds like a sensible option, actually. If you don't want governments to know what you're searching for, that sounds really hard to strongly guarantee. (I'd think they're logging your communication way before it reaches the server, in particular.)
Being based in the US is a dealbreaker because of US Surveillance.
Being based in the Canada is a dealbreaker because of Canadian/US/UK/Austrailain/New Zealand Surveillance.
Being based in the France is a dealbreaker because of French Surveillance.
Being based in the Germany is a dealbreaker because of German Surveillance.
Being based in the Japan is a dealbreaker because of Japanese Surveillance.
Being based in the Austrailia is a dealbreaker because of Canadian/US/UK/Austrailain/New Zealand Surveillance.
Being based in the Spain is a dealbreaker because of Spanish Surveillance.
The question is again, where exactly is government survailence not a deal breaker? Every country that is capable of it is doing it, and every country that is not capable has agreements with those that do.
[0] http://3g2upl4pq6kufc4m.onionI know they have sort by date, but this just sorts by date without taking into account how relevant the result is.
What's the estimate market share of DuckDuckGo today, that's the real question.
Do they dominate a niche? I think they have significant market share among HackerNews users.
If they had gone up millions then they would have used real numbers. Therefore the actual numbers are still pretty low.
(1)http://www.internetlivestats.com/google-search-statistics/
I feel like "duck duck go" is too long for the avg american to grasp or use in an ongoing convo when compared to "google", "bing" or "yahoo."
I can't see people saying "just duck duck go it." Maybe something like DDG or "duckduck"instead?
Maybe that's just me...
$1.80 / 1000 queries:
Bing through Yahoo BOSS API (which isn't free anymore) is the main source (and Yandex is used instead of Bing for certain countries in Europe&Asia). All other sources are smaller sources that augment the main search results. DDG is at the moment a meta search engine.
There was a recent search engine startup with its own large scale crawler, Blekko. Though, recently IBM bought it for their Watson AI, as a knowledge base.
I can't watch the video, about which the text reads "The news anchor just can’t resist a little jab about DuckDuckGo’s location choice".
Deciding before my query to use DDG is a hard habit/practice to employ. However, when shown results from Google, if DDG results were just a click away (maybe already rendered in another "tab") it would make A/B testing easier and seemless - which would be essential to eventually moving away and changing my "default" search engine.
Just my $0.02
It sure would, but you'd also lose quite some time doing A/B testing. I know because I tried, and eventually decided it's not worth spending any time on. So I just put DDG as default, and now - when the results aren't satisfactory - google is just one !g away. That is imo a much better and easier way of getting to know DDG.
For things not in English, it's quite bad.
To be fair what really made me move away was that their page load times were much slower. Now that I live in a country that is closer to DDG's servers I may give them another go.
I like the idea of less tracking and less personalization. But it turns out I'm not willing to sacrifice almost anything for it.
The internet is so unkind to close seconds.
I often bang to google for longer queries or almost anything that's local to Germany.
Then just type "gh <query>" on your location bar and it'll search directly in that site.
Also, you don't have to configure anything. DDG bangs are intuitive. I find myself searching sites I wouldn't have otherwise.
If you compare a query like "static webserver", you'll see why Google's results are much more diverse, than DDG's:
It seems to be decent at first but after a while, I get the feeling that I'm not finding good matches that comes with google and loosing precious time on searches instead of studying the resulting pages.
One big problem I find with DuckDuckGo is the search suggestions. I found that with DDG they are very poor and that I used them with Google far more often than I thought.
That said - give DDG a go! I think that for many people, the results are more than good enough. They have improved dramatically over the past few years. Even if you're not ready to use them full-time, I'd recommend trying them out on your phone first, perhaps.
I have ddg as my start page on my main computer at home, I was looking for a military paint code so typed "military paint code A6" and all I got was Audi A6 sites (non military and I wan't looking for anything to do with Audi). Scoot over to google and my expected results where at the top.
I continue to use dgg because they won't improve otherwise. Google need the competition, frankly.
Maybe ? Maybe ?!
Come on, that's the whole point of duckduckgo.
http://dontbubble.us/ https://news.ycombinator.com/item?id=2673898
The result seemed based off of only one of the search terms, not a grouping.
If I'm searching something code-specific sometimes I might need to defer to searching elsewhere however.
Overall I would highly recommend DDG.
For easy queries, DDG is great, but so is Google. When we reach just mildly confusing queries, the differences are very noticeable. One example from the top of my head: Searching for "devise", a popular ruby gem, yields several unrelated results before the actual Gem on DDG, while Google has the Github page as first result. Due to the popularity of devise I wouldn't rate this as a difficult query, yet problems already emerge on DDG. Multiply that by the number of queries you input daily and it becomes annoying.
I've long given up searching for trendily-named JavaScript libraries without adding "js" to the search term.
(Edit- clarification re: ads)
Or, even better: "!rubygems devise".
IIRC, the incognito/private mode doesn't matter here either, at least for chrome's omnibox
That's probably because of the lack of tracking. "Devise" has meanings outside of the Ruby world that are - arguably - more significant than the name of some authentication/session library. Google knows this, but - since it's able to profile you and stick you in the "Ruby programmer" pigeonhole - it knows to promote Ruby gem stuff. DDG has no such information, so it focuses on dictionary definitions first, since those are more commonly-useful than the Ruby gem.
When I want to look up something factual I know nothing about, DDG is the right searchengine. I finde that DDG is best at giving short, concise results.
When I'm searching for something within the (very) broad "entertainment" category, I go with Google.
[0] https://duckduckgo.com/?q=duckduckgo+shortcuts&t=canonical&i...
Without having much proof I would claim that DuckDuckGo is get better at about the same rate I feel Google as been declining in quality.
Is there absolutely no chance that even 90's era Lycos could ruin this search? Use the blank ! and skip the list.
Do I need a brief overview before I do more refined research? Use !wiki
Do I have a bug in my code? Use !so
Did Stack Overflow have nothing about my bug? Use !scholar
For any other search, I just wind up putting in !g and letting Google handle it because, if I haven't found it by this point, the regular DDG search probably isn't going to help.
Any NSL can order them to hand over the logs in certain regimes (bulk or per IP? We know what happened), but they cannot force them to write logs in the first hand. Without logging it will also be ~10% faster.
That doesn't really encapsulate the range, sure if I'm logging into facebook from my home connection off a stock linux desktop then that's way more trackable than Tails off a memory stick in a laptop with no hard drive using the wifi of the McDonalds across the street from Starbucks.
The issue is that the bar for anonymity online gets pushed higher and higher.
For Google results, in the very rare case that I need to see what they have, I use "searchthis !sp" to get anonymous Google results from Startpage. I used SP fulltime previously but they've had reliability issues and an odd issue with the back button.
Most !bangs that I use are for !w, !a and !gm. Firefox's dedicated search bar helps a lot with editing a long search string with another bang. Doubtful I'll be moving off DDG due to that feature.
Lately, DDG has also been quite slow for me and doesn't load at all for several seconds. Overall, I love the privacy part, but it's not as useful as a search engine ought to be for my usage. So I'm unable to quit the other alternatives, even though I badly want to.
Something this got me thinking though; what about other less popular search engines such as Lycos, do they ALL track usage? Or are there other search engines that are anonymized in a similar fashion to DuckDuckGo?
And is it just me or is it actually faster when you click on one of the results? Whenever I do that with google it seems like there is a delay while google does some analytics or something.
http://yro.slashdot.org/story/13/07/21/1853212/when-the-nsa-...
Every time I've tried to switch to DuckDuckGo, this has been the primary stumbling point.
But, I tried again just a couple months ago (I went whole hog and changed the search engine in chrome to ddg) and have been very impressed with it. It's been continuously worked on enough that it now serves about 90-95% of my daily search needs without any fuss and I actually prefer the way it presents images, semantic search and videos in search results over google's. It does a much better job at returning results for what I'm actually searching for and that's awesome.
For example in Google, if I search for "Mad Max" I get showtimes for "Mad Max: Fury Road" at the top and and imdb-like bit of information for "Mad Max: Fury Road" on the right (neither of which I searched for) and then a list of search results which these days are increasingly just links to Wikipedia's take on whatever I'm searching for (this time for "Mad Max" and "Mad Max (franchise)") followed by news on "Fury Road" the "Fury Road" video game, IMDB links to "Mad Max (1979)" and "Mad Max:Fury Road (2015)", etc. then trailers on youtube for both movies and links to the movie sites etc.
It's okay, I suppose, but Google first assumes I'm looking for "Mad Max:Fury Road" and fills the results for that, then I get links to WP and IMDB on the same topic (I could have just gone to those), except for WP it's not for Fury Road. And why no love for Thunderdome?
Guess what happens when I search ddg? I get a list of possible meanings, the first of which is "Mad Max" not Fury Road (that's #2), then a list of other possible meaning (which include Fury Road, the videogame, the Franchise, etc.). This is awesome, it's not assuming which meaning I want, and thereby getting it wrong like Google and the list of possible meanings is better ordered. Then the search results are better too, of course the prerequisite IMDB and WP links are there, but the top 4 results are for "Mad Max" (or the franchise) and not for "Fury Road"...I'm actually getting results for what I searched for, not for what it thinks I searched for. The mix of results after that is also "better" to my eyes, it includes a large fan site, which Google doesn't ever seem to get around to, Amazon, ebay, games, non-WP fan wikis, reviews, and so on.
Google seems intent on shoving the latest thing that the film studio marketing departments are currently pushing, while DDG provide links to information on what I actually searched for.
I've found this to be true for most of my searches. DDG is actually finding what I want instead of what Google wants.
About the only times I'm finding I'm using Google any more is in two cases:
* I've exhausted DDG's results and want to see if Google's bigger index has something else.
* Google's more sophisticated time constraints on searches. DDG just lets me order results, but Google let's me slice out results between time ranges, which I often find more useful for research purposes.
Bonus: Privacy, again not my main interest, but it's nice that it's there. !Bang syntax. I don't use many of them, but I find them useful (it's also how I execute google searches, just put a !g before my search in ddg).
Wishes: time-slice for search, someway to make it my default in mobile chrome on my android devices
No less impressive, and so much more informative!
Now you can search Youtube using your browser as a (prefixed) bang.
Also, you don't have to configure anything. DDG bangs are intuitive. I find myself searching sites I wouldn't have otherwise.
https://news.ycombinator.com/item?id=9733726
If I want to update the beginning of a string I ctrl+l -> home -> ctrl+shift+right arrow.
If it is at the end of the string I ctrl+l -> ctrl+shift+left arrow.
I don't mind having to press the home key on occasion that I'd try a different source (which is almost never).
That whole process sounds clunkier than just using DDG and a browser with a dedicated search bar. I don't think I'll be switching to your method, but it sounds like something I could see some people doing prior to DDG's existence.
So the difference between the nav bar and a dedicated search bar is almost nil. They serve the same functionality, but one allows you to search domains without having to prefix a keyword.
It might sound clunky because I primarily use keyboard shortcuts instead of reaching over to my mouse.
been using DDG since almost the beginning so i'm kind of conflicted...
And yes, it is a noticeable difference in growth rate.
(It's at 1:05)
http://www.wired.co.uk/news/archive/2015-05/18/gchq-hacking-...
But the early adopters do, and these are the high technologists, the users of HackerNews, and it appears that these are the ones driving the change. The other replies to your comment sadly seem to equate the "rest of the world" as non US early adopters / users on HN.
This may or may not lead to the late adopters coming along to DDG in a few years. If they come, it will not be because of privacy matters, or the bang syntax but because a) it would be better and/or b) everyone else is doing it.