Inside DuckDuckGo, Google's Tiniest, Fiercest Competitor | Hacker News Reader

Inside DuckDuckGo, Google's Tiniest, Fiercest Competitor

fastcolabs.com

83 pointsASquare40 comments

I see duckduckgo's traffic page referenced a lot: https://duckduckgo.com/traffic.html

Anyone have any idea what Google or Bing's traffic graphs might look like over the same period? Would be very interesting to do a comparison.

I run a small subreddit, and was quite pleased with how it's grown. Then I noticed I actually FELL in subreddit rankings. turned out there's just a huge rising tide.

My intuition is that DDG is growing faster than search is growing, but that the growth rate may be less impressive than it appears from simply looking at the chart.

Either way, I'm really, really happy they're challenging google (and also glad Bing is still in the game). Monopolies are no good for anyone.

Google's statistics http://www.statisticbrain.com/google-searches/

DDG is a drop in the bucket.

For Google, but not for DDG. It is very well possible to compete with a large entity in a niche. DDG is doing very well if you compare it with other (much higher profile) attempts at launching a new search engine. Remember Cuil?

Exactly what I was looking for, thanks. The total isn't as important as the rate. Google has grown about 40% in the past four years. DDG has grown many times that.

It's very, very dangerous to extrapolate growth rates into the future. But I doubt Google's will grow faster, whereas DDG still has the potential to keep up high rates of growth.

If I did the math right (not certain), DDG has had 3096% average annual growth. Keeping that up for even four more years would make them large.

So it's not ridiculous to pay attention to DDG's progress. Unless things level off.

I made the switch to DDG a couple of months ago, as I like their search results page (long listing, not short pages like Google), but after repeatedly being frustrated by not finding the results I need (and surprised each time I hit Google only to find the results right at the top), I switched back to Google yesterday. DDG as awesome as they are, simply are not Google. :(

The reason Google can give you good results is because of the tracking they do at tailoring the search results to you. DuckDuckGo doesn't do tracking like Google, so it's not going to be tailored to you.

Then what's the point of using DDG, when it can't give the results, tailored as per my needs!

that's not the only reason, google's results are better because they have vastly more resources both in employee talent and data. It's not just something as trivial as keeping track of your search history.

Where do you think those vast resources come from? Those resouces come from selling every ounce of information on you they can, they are even reading your email before you even open it. http://www.usatoday.com/story/opinion/2014/09/04/google-spyi...

"they" aren't reading email and what does it even mean "before I open it" - they host it!!! It's software that "reads" gmail. If you don't want google reading your email, then simply host it elsewhere. Nobody is forcing you to use gmail. It's patently obvious that they use gmail and the usage of their other free services to garner more information to maximize their revenue. Don't like it, don't use it. It's a simple as that.

It frustrates me no end to read stupid comments like yours. Of course they use what you freely give them to their advantage. There are plenty of email hosting options, viable search alternatives, viable mapping alternatives etc etc. Google don't have a monopoly on anything. They make you a simple trade; compelling products in return for profiling you, what you say, where you go and what your search intents might be.

I know perfectly well where google's profits come from. Who reading this site doesn't?

I don't use it. I was just pointing out that their 'vast resources' are coming from the sale of your info, nothing more. Isn't that the point of these posts? Your comparing one search engine that is 'trying' to keep your searches private yet complaining they don't have the resources, when most of those resources for Google are coming from not keeping your searches private. I'm not sure where the complaint and or comparison is taking place. I freely give DDG my search data and they aren't using it to their advantage in the same manner google is, so whats your point again about ddg not being as good? I don't get your argument at all.

Its not just about tracking and collecting information on your habits that makes their searches relevant to you? You really think so? https://www.youtube.com/watch?v=mzZzf6PoyC4

What hnriot says is pretty true. However, one of the points of using DDG is right on their front page below the search bar: "The search engine that doesn't track you."

As I said, the results can't be tailored to your needs. For instance: when I search Python how does DDG know to bring up? Python can refer to the snake, the programming language, the horror film, the missile, the sports car, the turboprop engine, etc.. DDG doesn't know what I intend. It can try to go off of popularity though, or I have to feed it more keywords for what I'm actually looking for.

Granted other engines can have this issue, but I think DDG more susceptible to this problem.

https://news.ycombinator.com/item?id=8266839

You may be in a trap of selection bias here. True, sometimes !g reveals much better results, but one uses it only when not satisfied with DDG. However try to Google some queries that DDG does fine and you will see Google's results can be worse for some queries.

I'm not sold that are a competitor. I'm also not sold on the privacy niche that they are trying to fill.

Without funding, they will have to rely on advertising. It's fact. I doubt companies with serious marketing budgets will use their platform.

Look at AppNexus for example. Pretty advanced technology.

You, as an advertiser would be a total idiot to spend the money on a 'static' ad placement within DDG.

Statistics matter. If this was 1990, maybe. Not in todays world. People don't just throw money at stuff like this.

Now from privacy angle. Has anybody actually verified their claims? Or do we just take their word for it?

Now, they offer Chrome extension and serve advertising. Both of those things leak data. So slowly, they are starting to have those 'exceptions'.

And do you really think they will be able to tell FBI 'We don't track stuff sorry guys!' when FBI really needs something from them?

They will either be shut down or forced to log data once they get any useful traction. That's the way it works.

I bet that once they reach that point, they will pivot as a 'regular' search engine or look for a buyer (re: Google).

Don't worry, you will get a warm notice of how they fought for your privacy but at the end they had no choice.

Holy defeatest attitude. Your not sold? LOL no your being sold ever single day. Those things leak data? Find some proof besides your own mind. Talk about stockholm syndrome.

> Find some proof besides your own mind

Proof of what? Referral tracking?

The DuckDuckGo extension is to provide duckduckgo searches using chrome, DDG's ads on 'their' site doesn't track you while your browsing the search results or when you click, however once you reach the page of the link you click on, then and only then are you probably catalogued and or recorded. But thats the independent site you visited not DDG. DDG can't force the websites you visit that you found using their service to not record or track you once you get there. In otherwords, ads can and do track, but the ads displayed on DDG do not...

"We do not use any third parties to do the code insertion, and we do not work with any sites that share personally identifiable information (e.g. name, address, etc.) via their affiliate programs. This means that no information is shared from DuckDuckGo to the sites, and the only information that is collected from this process is product information, which is not tied to any particular user and which we do not save or store on our end. It is completely analogous to the search result case from the previous paragraph--we can see anonymous product info such that we cannot tie them to any particular person (or even tie multiple purchases together). This whole affiliate process is an attempt to keep advertising to a minimal level on DuckDuckGo."

For those who think only Google can find what your looking for there is two reasons for that, one of course is they are tracking the crap out of you and tailoring search results for you in your own little search bubble world you've created, but an even bigger reason why some have trouble switching is because whether you know it or not, you've trained your brain to search in a Google like fashion, in otherwords, every search engine takes a little finessing when choosing what terms your use in your search, I've been using DuckDuckGo so long now, that my brain is now trained to use DuckDuckGo style search terms, so much so that those same terms don't always work on Google when they work just as I expected on DuckDuckGo. If anyone wants some pointers on how to make the change, simply try adding one extra search parameter to your regular query that you would use on google and wala, duckduckgo will more than likely find it, which to me is a small price to pay for a bit more privacy. Yes at times I also !g bang google, but its gets more or more less frequent.

previous submission, with lots of comments:

https://news.ycombinator.com/item?id=7270973

Yes, the current post is a dupe. Thanks.

I've tried switching to DDG a few times but since so many other things are integrated into Google/Bing it never sticks.

Have the search results been gamed like they have on Google? That's my fear of DDG getting too big. Google is next to useless for so many searches that just return result after result of useless sites who have farmed questions/answers from other sites.

Alright, so this is probably the place to ask this.

Why is there a juxtaposition between what people want and between what people are willing to give?

Specifically: why do so many people ban web crawlers other than the "large" ones, and then wonder why the quality of other search engines are lacking?

> Specifically: why do so many people ban web crawlers other than the "large" ones, and then wonder why the quality of other search engines are lacking?

Not to be difficult but is this a real problem? Do a lot of sites ban DDG's crawler and then complain about the low quality experience? Can you name a single example?

As far as I know, DDG doesn't have a crawler and just uses other search APIs (Google, Bing).

edit: OK, I was wrong - https://duckduckgo.com/duckduckbot

2nd wrong, DDG doesn't use Google for search results at all. Google essentially killed Scroogle that did exactly what startpage is doing now by banning their search proxy, which makes one wonder, what is the special relationship that startpage has with google that scroogle did not.

Oh. In that case I got DDG and StartPage confused.

Maybe StartPage still displays Google's ads? They display some ads.

I wonder if there's really much overlap between the groups. Maybe the difference is that you're just describing different people who prioritize different things?

> Specifically: why do so many people ban web crawlers other than the "large" ones,

Never heard of this. Third party crawlers usually get banned because they are poorly made (get stuck) or disobey robots.txt.

Gabriel Weinberg also has a super blog (and now a book) on traction that every startup founder should read.

Blog: http://www.gabrielweinberg.com/blog/

Book: http://tractionbook.com/

+1 to the book, it's chock full of more ideas to try than anything else I've read. (I'm sure the blog is great too, but I'm not familiar with it)

Just got my copy of traction in the mail, about to start reading it. :)

Looks like from the query graph in the article, that DuckDuckGo enjoyed a huge surge in users after the Snowden leaks (more than doubling).

I've only toyed with the idea of switching off google for searching... mostly because I'm so tied into their ecosystem with my Android... but, I must say, DuckDuckGo looks impressive.

Just change your default search engine for a week and see what you think. 95% of the time I get the same results, and I just hop to google.com for the other 5%.

I've tried several times, and it has never stuck. But other people have clearly had different results.

Out of curiosity, did you make use of the g! keyword to cause a search to use Google instead? That was what made the transition easy for me. My search then consists of doing an initial search, and if the results look poor at first glance, toss a g! in front and see if it's better.

Thanks: I'll have to remember to make use of that.

DDG lets you use bang to indicate where you want to search. For example adding !g will send the search to Google. I use DDG for searching and in cases where I don't like the results and think Google could do better add the !g. That way you get the best of both worlds.

I've found that for regular searches with good obvious results then DDG does just fine. When the combination of words matter then it doesn't do as well, giving more generic results than the specifics you were after.

Reply on news.ycombinator.com