Hacker News Reader

Comment by luhn | Hacker News Reader

Microsoft offers a Bing API [1]. So Microsoft is getting paid by DDG for using Bing. Given Google's domination of the search market, I think it makes sense for Microsoft to sell their search API and let other companies help chip away at Google's user base.

[1] https://azure.microsoft.com/en-us/services/cognitive-service...

View on HN

Correct. The list price for the Bing web search API is $30/10,000 searches. I wouldn't be surprised if DDG was also able to negotiate a volume discount.

In addition, DDG serves ads from the Microsoft ad network, so they will also make money there.

Also, worth noting that Google provides the same type of service through API. I built a search engine [0] where Google web results are one of many sources used.

[0] https://www.runnaroo.com

For those curious, "[Google] Custom Search JSON API provides 100 search queries per day for free. ...Additional requests cost $5 per 1000 queries, up to 10k queries per day."

It seems runnaroo would therefore be limited to 10k searches/day?

Not necessarily. Startpage uses Google for all of its results.

> You can’t beat Google when it comes to online search. So we’re paying them to use their brilliant search results in order to remove all trackers and logs. - https://startpage.com

Does anyone know how one can get whatever deal Startpage has?

Startpage is descended from Ixquick (launched 1998). I believe they have an old, uniquely favorable contract with Google as a result.

If I am willing to write a wrapper for this API, I can avoid all the ads (but not tracking)?

Does this API still return paid results near the top?

That sentence could also mean "$5/1000 applies up to 10k queries per day (and you get to talk to a sales rep if you do more than 10k queries per day)".

Like your search engine!

If DDG gains sufficient marketshare, MS may just buy them out, for a steal perhaps since they have the ability to tighten the screws. So DDG is little more than an MS proxy aimed at Google’s search business. It’s not really an underdog, just two behemoths fighting for control of the search business. Sounds more like “Meet the new boss, same as the old boss.”

This is only true if DDG isn't investing into its own backend as it grows.

If all the users go to DDG.com, it won't matter if Bing has a great backend or not.

I've recently lived this experience in another context. We built a great backend for a partner that was good at publicity. When they redirected users to their internal backend, we were dead in the water. It didn't matter that the partner struggled with their backend and ours was better. We hadn't invested resources in independent publicity, so users couldn't find us. It was a hard lesson to learn.

And to that point, DDG does have its own crawler, and I believe leverages other third-parties as well (I know Yandex was historically in the mix, for example). So if Microsoft did try to strongarm DuckDuckGo, DDG would have other options, even if less-than-ideal.

There’s no one close to Google in the US. After a big drop, there’s no one REMOTELY close to Bing for 2nd best results. Bing dropping DDG in 2020 or 2021 would be a death kneel for them for US search results. Probably many other countries too. Yandex is too far off right now to be a replacement. I haven’t checked Yandex in a bit of time though. Perhaps they drastically improved.

People in tech circles always say DDG has a crawler. I know they do. No one ever says how big it is. I have a hard time believing it’s crawling more than a couple percent what Bing is crawling.

> No one ever says how big [DuckDuckBot] is

One complicating matter is that it specializes in DDG's "Instant Answers"; apparently¹ the "source from other search engines" approach really only applies to the traditional results list, while DuckDuckBot is focused specifically on the Instant Answers, effectively optimizing for the absolute most relevant first result possible. So in its current state DuckDuckBot ain't exactly comparable in purpose.

Still, I don't see much that'd be stopping DDG from ramping this up and having it generate the traditional list of results, too. Yeah, obviously the results for quite a few queries will be far from ideal, but if the query already hits one of DDG's curated data sources, I suspect the results might already be better if anything.

----

¹: https://help.duckduckgo.com/duckduckgo-help-pages/results/so...

DuckDuckGo is ddg.gg not ddg.com

However, I believe they are also duck.com

I believe DDG used other solutions pre-Bing; it’s just that, after they switched to Bing, it seemed to work well enough that they kept it. Could they afford the hit in result quality resulting from switching away, if MS started playing rough? That’s a good question, but I’m sure the DDG folks have thought about it.

Depends on the user I imagine. Most of my searches start on DDG and I just got used to expecting certain types of "human phrase searches" to be !g'd, or more recently for me, !s'd.

So if DDG degraded in quality I'd be fine, because I don't expect perfection. Though I suppose it depends on how much, of course.

I believe DDG used other solutions pre-Bing...

DDG used the Yahoo Boss API [0] originally. It was pretty good, but Yahoo shut it down and Bing now also powers their search engine.

[0] https://en.wikipedia.org/wiki/Yahoo!_Search_BOSS

Wasn't Yahoo! Bing powered at that time? :)

DDG launched in 2008, and according to the below, the deal between Yahoo and Microsoft didn't happen until 2009 at the earliest.

In July 2009, Microsoft and Yahoo! announced a deal in which Bing would power Yahoo! Search. All Yahoo! Search global customers and partners made the transition by early 2012. The deal was altered in 2015, meaning Yahoo! was only required to use Bing for a "majority" of searches.

Source: https://en.wikipedia.org/?curid=4325491

That’s not clear if DDG’s US and European results were using Bing or not in late 2009.

I know for sure people have said DDG is better over time. I can’t imagine it was very good in 2008 or 2009.

DDG was still working significantly with Yahoo! as late as 2016: https://news.ycombinator.com/item?id=12013264 . The links are dead but you can desume from the conversation what it was about.

Note: I know Yahoo is Bing behind the scene - I'm just saying the technical implementation at the time was not tied exclusively to "bing.com".

Good point. Though my bigger point was that DDG has essentially always been dependent on Bing/Microsoft.

2016 is the year Yahoo was sold to Verizon. 2 years later, Verizon writes down $4.6B of the AOL Yahoo subsidiary. A year after that they sell Tumblr in a fire sale. Verizon stopped caring about their web media empire when they did the write down/sale too. Effectively throwing in the towel as a major web advertising hub even though they are likely still top 5 in the west after Google, Facebook, Microsoft, Amazon. That’s peanuts to Verizon though.

I can’t imagine Verizon cares much about their deal with DDG. If Microsoft cared, it makes sense that they could have Verizon and Yahoo do what they want over lawsuit worries from DDG.

I’m being pedantic at this point and totally guessing with my assumptions. I admit the continuous praising of DDG on the web has gotten to me and likely making me feel and react this way.

> for a steal perhaps since they have the ability to tighten the screws.

You think they don't have some tight contractual relationship with built in renewals and extensions? This is really similar to how corporations lock up retail real estate locations. They don't sign a 1 or 2 or even 5 year lease and then hope the landlord doeesn't rake them over the coals when there is no where else to go to. (Which could often be the case..) You build in renewal options as part of the deal. Those can last a long long time. May even have (probably does) a 'right of first refusal' and so on.

Maybe DDG is hoarding the results of every API call to Bing to build its own database. Then one day they can turn off the Bing integration and nobody will notice.

You can't hoard search results since they change. The same search in March may be different in April.

1) search results need to be dynamic over time

2) MSFT has clauses in its contract to address this. If a commenter on HN thinks of this one minute after reading the article, you can be sure that their legal team has thought about it too.

So Bing results plus Bing ads. Why does no one use Bing then? Just the branding? Microsoft uncool?

DDG doesn't collect personal info, and the ads aren't based on your ad profile, they're based on just the search terms. This is DDG's differentiator.

Plus hashbangs and the results are not exactly bing for every term.

Some people do. I do. On places with vocal minority (or in general sometimes) like HN, reddit, Twitter, you’ll see many more people tout the trendy/cool DDG. As you said Bing/Microsoft aren’t cool.

I’m talking about the majority of these people who outside of basic ad/tracker blocking and using DDG, don’t stick to caring about privacy above most other things.

A lot of what I’m saying doesn’t count for HN’s crowd as much as other bigger sites.

I don’t have any backing for this statement. Just what i believe from common sense looking around.

https://www.dogpile.com/

I remember that being my favorite search engine over 20 years ago. . . Crazy it's still around.

$30/10,000 Bing searches seems like a lot of money for DDG to pay, and from what I read below, Google is even more so.

Would ad revenue for 10,000 interactions actually offset this cost for DDG?

Can't answer that question, but it's worth mentioning they also add their affiliate tag to Amazon product links and possibly others to get some revenue.

Now I'm curious, how many search queries does the average person make per month? I've been more and more dissatisfied with results I'm getting and would happily pay $10-20 per month for search. Especially if that means I'm not getting tracked and not seeing ads. Or maybe just seeing ads that aren't being in-lined.

That's news to me, wouldn't Microsoft's hacky JavaScript ads be able to snoop on DDG users browsing habits?

Just wondering, are you a one man show running a search engine. That is extremely impressive. I did some relevant local searches and was able to find what I was looking for in the first page.

Right now, it is just me. Thank you. For the local search it is pulling from Yelp.

It's a pretty great deal for Microsoft, because they get paid twice. Once in dollar value for access to their API, and once in user queries. Even anonymized user queries have value in terms of learning what users are interested in for search purposes in general.

Reply on news.ycombinator.com