Duck Duck Go = Perfect search engine for programmers | Hacker News Reader

Duck Duck Go = Perfect search engine for programmers

fseek.me

70 pointsfseek48 comments

I like Duck Duck Go. Mostly because, there is someone out there who is courageous enough to give it a go, single-handedly, into the search engine market. So props to him, I can support someone like him.

Having said that. There can be too much of a good thing. I feel that HN is being used as an advertising platform to promote DDG, which in itself is not such a bad thing when there are major developments and exiting features and news about it once in a while. So can we try to go easy on this promotional bandwagon?

I'd rather see 10 more articles about DDG than one more about the ipad or fakestevejobs.

"I'd rather see 10 more articles about DDG than one more about the ipad or fakestevejobs."

...because clearly, those are the only choices.

As long as we're throwing around false dichotomies, I'd rather see 10 more articles about the internals of a search engine than one more rah-rah post about DDG.

That's really good, when will you post one?

Gabriel could do one too if he feels like it.

Of course those are not the only two choices (there were three of them, buy hey, who is counting), but since they seem to be the most prevalent posts that irritate me I figured I'd put my personal preference out there. If you want a longer list I can provide one but I don't think it will make the point much clearer.

"That's really good, when will you post one?"

Now you're just being petty. There are plenty of technical articles that languish in "new" every day -- I've submitted more than than a few, and I vote for many others.

An argument against other types of fanboy spam is not an argument for DDG fanboy spam.

I don't think the DDG articles are 'fanboy spam', they constitute an indepth look at one of the more promising start-ups and it is happening here on HN as it unfolds.

Just like patio11's stuff is a case study in how it is done, so is this, with one huge difference, I can't see myself ever buying a set of bingo cards but I can definitely see myself using a search engine, if only for comparison purposes.

I could say with just as much justification that I'd not like to ever hear a word about bingo cards again, but I don't think that is right because the bingo cards serve as an illustration on how to run a successful startup.

The other stuff is the fanboy stuff. And that I could do without, hence my preference for DDG articles.

But I promise I will write an article about the guts of a search engine, I built a small one about 5 years ago and maybe there are some interesting bits that warrant dissection.

What would you like to know? For starters: http://www.gabrielweinberg.com/blog/2009/03/duck-duck-go-arc...

Do you maintain full text indices for parts of the web and which parts, or does it all fall back to external services like Wikipedia search, Yahoo BOSS, etc?

If you have some indices on your servers, how do you build and search them? Is it your own technology, or open source (Xapian, Lucene-based, etc)? In what percent of searches do you display results from your own indices?

I would like to know what was the motivation for starting the project. I don't think it's necessarily a bad idea for David to tackle Goliath but there has to be a unique value proposition.

Based on the information I have found about the project the only area I can see this competing is in the user experience (scrolling results, encrypted search etc). Maybe someone who knows more about the project can enlighten us.

The original motivation was getting fed up with spam in Google. Here's an interview where I explained: http://www.snugd.com/2009/04/24/3361/

It's not perfect yet, but I'd really like to get there: http://www.gabrielweinberg.com/blog/2010/03/hack-hack-go.htm...

Here's what's coming:

--Stack Overflow, man pages & programming documentation in 0-click.

--More programming !bangs; I've already added a bunch but would love more suggestions: http://duckduckgo.com/bang.html

--More goodies; I've added a lot recently (color codes, regexp, more advanced math to wolfram alpha, today unicode); again, suggestions welcome: http://duckduckgo.com/goodies.html

I use !bangs a lot, but I wish they worked at the end of your search terms instead of just at the beginning.

I often don't think about the specifics of what I want until I've started typing, then I realize, "Oh, I actually want the wikipedia page of this." So I have to Ctrl-A to the beginning and type !w [space].

It's not a big hassle, but it'd be great if I could skip that step.

Easy fix--will do tomorrow as it's getting late :)

This is now fixed. I also made them case insensitive.

"I use !bangs a lot, but I wish they worked at the end of your search terms instead of just at the beginning."

In addition to, not instead of. (And why not just allow it anywhere in the terms?)

I like seeing that my previous search was a ! search in my ff search bar. If it's a long search term I wouldn't see it at the end.

ddg is quite nice when you're searching in languages with a relatively simple morphology. For English, I'm not using google any more unless ddg tells me to do so :)

It is very easy to perform shallow parsing operations on English, because of its relatively simple morphology. However, for agglutinative languages like Turkish, (Finnish, Hungarian and Japanese are also in the same family) where stems can appear under too many forms to enumerate, basic shallow parsing algorithms would not produce as interesting search results.

My anecdotal experience with DDG in Turkish seems to go in line with that assumption. So, I think DDG has a lot room for improvement in processing languages with complex morphology.

==================

A famous illustrative Turkish word is: uygarlaştıramadıklarımızdanmışsınızcasına

...which decomposes to suffixes as follows: uygar+laş+tır+ama+dık+lar+ımız+dan+mış+sınız+casına

...and translates to english as follows: behaving as if you(plural) are among those whom we could not cause to become civilized

I am current taking the search engine for a spin and it has made a great first impression. I particularly like that instead of paging the search results it displays the results on demand using ajax. I prefer this interface because with less effort I see more results. I've found that with Google I tend to not to move past the first page and by doing that I probably miss out on information that may have been useful to me.

I wonder if Google has ever considered a similar interface for their search results.

I tried to use DDG exclusively for a month. Unfortunately, I had to drop back to google to find a lot of info. I still use DDG sometimes but for the standard stuff I search for, it doesn't fully do it for me.

Google will not do that because it would cost them a fortune in missed ad impressions and clicks.

The big question is how Gabriel will scale DDG with the design decisions already made.

It's quite possible that DDG can get to profitability with the layout the way it is today, maybe not as profitable as google but I don't think that would matter much.

Making a step back in income is a lot harder than foregoing a certain amount of income from the start.

Google will not do that because it would cost them a fortune in missed ad impressions and clicks.

This seems unnecessarily constraining. They already allow you change your preferences to see 100 results per search instead of the default. How does this fit with your logic?

Certainly Google is very concerned with click-through on ads, but I'm not sure they'd be losing much if they allowed continuous scrolling. What percentage of their revenue do you think they derive from ads placed on 'next' pages?

And certainly there would be some way that they could integrate more ads with the Ajax loads: adding them to the sidebar as you scroll, or otherwise integrating them with the incrementally added results.

> They already allow you change your preferences to see 100 results per search instead of the default. How does this fit with your logic?

That nobody except for a few geeks do it. Just like the 'I feel lucky' button.

> Certainly Google is very concerned with click-through on ads, but I'm not sure they'd be losing much if they allowed continuous scrolling. What percentage of their revenue do you think they derive from ads placed on 'next' pages?

About 32% according to their own statistics. 68% of the clicks (ads or results) are on page 1, the rest on subsequent pages.

Google is working hard to get to the point where 100% of the clicks are done on the first page, then it is a moot point, but a 32% impact to their bottom line is not something they'll do if they don't have to.

Also, yes, they could load that ad again, but I suspect that may not be as effective and would lead to a lot of complaints. A 'quiet' (as in non-animated) page creates a lot of goodwill, switching ads would likely lead to a distraction and therefore diminished user experience.

Great statistics, thanks! That's much higher than I would have guessed. While I agree that very few people change the default number of results, I would have also guessed that few click through to a second page.

Personally, I don't like the AJAX update approach. I'd much rather that a site just give me a long list of results, and allow me to scroll through them without loading delays. If I could set Google to return 1000 results, it would feel about right.

> I'd much rather that a site just give me a long list of results, and allow me to scroll through them without loading delays.

I think that is a balance between server load and user convenience.

On ddg it is already pretty fast, if you aren't in a real hurry it is hardly noticeable and faster than clicking through to a second page.

Well, that, and the fact that users really hate pages that are slow to load. (Although most of the loading would happen off-screen, this can still slow down users/other requests/...)

> > They already allow you change your preferences to see 100 results per search instead of the default. How does this fit with your logic?

> That nobody except for a few geeks do it. Just like the 'I feel lucky' button.

Why don't they offer this as an option then, by the same logic?

They've talked about that at some length, it is a relic from the early days and even if it costs them money (60 million bucks annually iirc) they feel they should keep it not to upset the users that have gotten used to it being there and occasionally using it.

edit: so much for my memory, it costs them almost twice that, 110 million per year, source:

http://hubpages.com/hub/Im-feeling-lucky-button-costs-Google...

Can't they just load up more ads with the additional search results at the bottom?

I don't think he needs to do much scaling - DDG needs some machines, and adding a couple of people may be useful, but it seems to do well enough so far, and expenses can't be very high. It certainly doesn't need Google's profitability to keep going...

Just in case you don't know, this plugin does that for Google, HN and a bunch of other sites: http://autopagerize.net/

It's great, except it would slow down Firefox notably for me. I haven't tried out the Chrome version.

Bing Image Search has infinite scroll without paging.

http://www.bing.com/images/search?q=san+francisco

When I tried out DuckDuckGo about a year ago, I noticed it was really good for general queries, but always had to go back to Google when I searched for compiler error messages or other programming-related things. Because of this, I eventually slipped back into full-time Googleage, despite giving DDG a try for about a month and a half.

That's why I like the !g option. If for whatever reason DDG does not seem to "get" what I want (probably because of me being trained to "google it") I can easily search with Google while leaving DDG as default search. Granted, I'm using it as default only since about a week, but this is quite alright when considering that I came from 100% Google. My previous attempts ended very similar to yours..

A lot has changed in a year. I'd love if you'd try it again and email me your detailed feedback, so in another year maybe you'll switch :)

(To grandparent comment[er]) I also had a similar experience a while ago; but I gave DDG a second change recently, and indeed, the results have improved enough for me to keep DDG as my primary search engine, resorting to Google only once every 50 searches or so.

Perfect for programmers? I was skeptical. Then I searched for A* (which doesn't work on Google). Perfect indeed!

Nice. I will switch when it handles mathematical expressions well.

What kind? It might already or might be an easy fix.

personally I often use google to do stuff like

http://duckduckgo.com/?q=120mph+in+km%2Fh

http://duckduckgo.com/?q=1%24+in+€

If I search for x^2 I want to see results related to (x squared), not "X-2" or "X2", etc. The ^ is deliberate and should not be ignored.

When I first heard about DDG it was because of it's privacy. I couldn't care less for that.

Now, saying that it is good for programming questions made me set it as the default search engine without even testing.

Zero-click info is amazing for quick doc checks, the auto-extending page is sweet and actually displaying the page link and favicon is a lot more informative.

Two things though:

- It's slow. Maybe because I'm in South America, or the server is not that good, but speed is essential for a search engine.

- Special characters are not always handled correctly. A* work perfectly, but the situation is inverted when it comes to "@". Google's results for @override (http://www.google.com/search?q=@override) are better than DDG's (http://duckduckgo.com/?q=@override).

Thx. Servers are currently in the US; hence, the slowness for some people outside of the US. Eventually I hope to improve the infrastructure there.

Special character handling will be much better as I index programming documentation and Stack Overflow with the special characters.

I like DDG, but the last time I used it I had to fall back to Google to find an answer to my question. I was trying to find the date at which the central bank of Canada was going to make its next rate announcement. I knew it was June, used the most obvious keywords, but didn't find a good answer with DDG. With Google it was in the top 10.

Not a huge problem. It works most times when I use it. But it's not quite Google yet.

Is DDG completely independent or does it pull in results from other search engines, like a meta search engine? Does it do its own spidering?

Both. Gabriel has disussed this before (I'm on my iPhone so no links sorry). But basically he does his own spodering and also uses yahoo boss and the bing api.

As a win32 programmer, I always try things like createwindowex and I expect to get msdn as it is the most informative page. I can get it on DDG (doesn't even, show up ). It's the first result on google, even if I mistype it like createwindowe or createwindoaexaze. But may be I'm a dinosaur.

The first result at http://duckduckgo.com/?q=createwindowex is not what you want? It goes to http://msdn.microsoft.com/en-us/library/ms632680(VS.85).aspx

And FYI in Chrome you can set it to use DDG for search instead of Google (they actually have quite a number of choices).

"Privacy -Your searches are private! I don’t need to say more"

Well ... they are now

Reply on news.ycombinator.com