Hacker News Reader

Comment by npteljes | Hacker News Reader

I love the acknowledgement that DDG put on their page.

    We're currently experiencing an issue with DuckDuckGo Search. Thanks for your patience while we get our ducks in a row.

    In the meantime, you can use other search engines right here by using "bangs"

So fun and straightforward.

View on HN

Interesting how they insist that they use "so much more" than just Bing for their results, but the moment Bing goes down their search functionality is down entirely, unable to show a single result.

They probably start with Bing results and do custom filtering, ranking and presentation, so without the initial Bing results they can't do anything.

Someone please tell them they can wrap that code inside an "if" condition, and continue with the rest of the search.

What would be the rest if they don't have their own index?

So basically DDG is the wrapper for Bing Search, not an "independent" search company as they claim.

Probably. But DDG at least offers more privacy, right?

Right?

DDG has a history of breaching privacy: https://techcrunch.com/2022/05/24/ddg-microsoft-tracking-blo...

Meanwhile Brave Search or Mojeek provide more privacy, being independent at the same time.

> Brave Search or Mojeek provide more privacy

Do they? Or have they just not had that "history" yet? I don't see anything fundamentally different in Brave that protects your and my privacy better than on DDG. I don't know Mojeek enough.

Brave explicitly claims not to.[1] Brave also does not rely on Bing for results.

[1] https://search.brave.com/help/privacy-policy

Edit: mojeek claims the same.[2]

[2] https://www.mojeek.com/about/privacy/

Last time i checked, Brave was insisting on convinving me to use certain cryptocurrency platforms and it was more intrusive with it than typical web ads, which seemed really twisted, as the same Brave claimed to give ad-free experience.

Are they still doing that?

They don't. The cryptocurrency options are all disabled by default and are opt-in.

It literally never did that. There's some crypto wallet button that you get in the address bar that you can disable with a right click, and that's pretty much the extent of it. All crypto and other related functionality is completely opt-in.

Yes, those ads are still part of Brave.

What ads? I'm writing this on Brave and I've never seen any ads. Are we talking about the same browser?

I was talking about Brave Search (https://search.brave.com), not Brave browser. Seaech doesn't promote crypto.

No, they're not. I've been using Brave almost since its inception and I still have no idea what its cryptocurrency features are. Nor have I ever seen an intrusive ad from Brave, or anywhere else since Brave has an ad blocker built in.

Why does this same conversation happen every time Brave is mentioned on HN? Have any of the people complaining about Brave's crypto-whatever "problem" ever actually used it?

That only concern the browser not the search engine ?

HN discussion at the time: https://news.ycombinator.com/item?id=31490515

Cause Brave CEO, Brendan Eich, is such a swell guy.

That is true, but it's referring to their android browser app not their search engine, if I've read correctly.

Yes it is one of many metasearch sites and not actually a search engine. Other engines include Kagi, Yandex, Brave, Mojeek, Quant, and something called Google.

Quant also relies on Bing API.

Quant or Qwant? Qwant does not rely on the Bing api but has its own index.

Are you sure about that? It doesn't completely rely on it but didn't it use bing as one of the search providers? I'm probably completely wrong and my info is outdated but I'm asking because it would be pretty special (in a good way!) if they use their own index exclusively.

Qwant was not usable yesterday during the Bing outage so that pretty much shows they cannot function without Bing (even if they do have some auxiliary indexes of some sort)

Qwant, sorry. On Wikiledia they mention Bing multiple times: https://en.m.wikipedia.org/wiki/Qwant

Qwant was also out of service today due to Bing API outage: https://www.gamingdeputy.com/bing-outage-exposes-qwant-and-d...

Pretty sure kagi is a bing wrapper too, although they blend in some other datasets.

Kagi's main results come from Google actually. (edit: see https://help.kagi.com/kagi/why-kagi/kagi-vs-google.html - Kagi really shows how good Google could be, since it's mostly using the Google index and then doing user-friendly things instead of user-unfriendly things on top)

While I'm not disagreeing that Kagi's main results from Google, the source you linked doesn't specifically say that. Just says:

> "Heck, it even enables Kagi to exist!"

> "We’re grateful to have access to Google's search technology and infrastructure for Kagi."

So I wonder what the actual results mix is. Like I said, could be mainly from Google.

Edit: Added additional quote.

I could be wrong, but I think they're playing to their early adopter audience, that being tech workers.

Google is terrible at tech searches, without verbatim and now "web" search. Kagi spending work in this area, and indexing this area, might bear good fruit.

I firmly believe Google will release Premium Search at some point. They did it quite successfully with YouTube...

One wonders if the popular narrative around crappification of Search is partially self-induced to prime consumers for premiumization.

More countries have nukes than companies have actual indexes

But do more countries actually build them?

As someone who uses Kagi, I think it is absolutely not accurate to say they are a search engine rather than a meta search engine. Most of their results come from the other engines you listed.

https://help.kagi.com/kagi/search-details/search-sources.htm...

Interesting, I always had the impression they had their own index. Shame.

As per the first sentence on that page, they do. It's just not their only source, and from my experience it is far from the main source. You can see "% of unique Kagi results" on each search; these are the results from their own index.

Mojeek has its very own index.

Never heard of Mojeek and tested it right now, surprised how fast it feels! Going to use it as my default for a while.

Just tried it and it led me to a 403 error page: “Sorry your network appears to be sending automated queries so we can't process your search at this time.”

Qwant is also a meta engine, and was also down yesterday.

Qwant also relies on Bing (and was not usable during yesterday's incident)

The industry term is "whitelabeling"

Sometimes it pays to pretend

The legal term is "deception".

Taking someone else's product and making it better is a useful service.

It's not deception if you tell people you're doing it.

DDG massively oversells their own contribution to the search results.

If you look at their marketing material, they claim to "use a variety of sources, 500+ to bring you your search results". This is of course technically correct, but the truth is they are a relatively thin wrapper on Bing results.

So yes, I'd claim it is a form of deception.

And Bing lifts their results from Google (https://www.wired.com/2011/02/bing-copies-google/) so in the end it seems like the internet only has one search engine.

There's also Kagi

That article is from 2011. I can imagine that in 13 years this may no longer be true.

What would lead you to believe that in the last 13 years MS would make the titanic investment of reimplementing their (alleged) Google-based search backend?

Given the trajectory of Bing this seems unlikely.

For me it’s because DDG (aka Bing) results are vastly different than Google results. I use the !g and !s bangs often.

I think you’re making a huge assumption that nothing would change in 13 years!

Makes me wonder, does this mean Bing is the Chromium or Firefox of search?

Brave Search [1] is 100% independent. There's also Yandex [2] which also works excellently, but is biased towards more Russian language results. The image search is second to none though.

[1] - https://search.brave.com/

[2] - https://yandex.com/

Brave search is only very recently independent, and not, as far as I can tell, 100% of the way there. While they are building out their own index, they only recently stopped augmenting their own results with Bing, and are still augmenting them with Google.

That is not fully accurate. Brave Search is fully independent - the last dependency on Bing was dropped more than a year ago, but even at the time the dependency would not have prevented Brave Search from functioning during an event like yesterday’s Bing outage - it would have resulted in a drop of quality on a subset of complicated queries, though.

The “augmentation with Google” that you mention is an optional feature that only works in Brave browser and doesn’t mean Brave Search is less independent than in other browsers (but it might mean that the quality is increased for some queries; or at least closer to Google results in these cases).

I think when talking about independence it is important to make a distinction between “dependent” which means “cannot work at all if the provider goes down” (e.g. yesterday’s outage) and “dependent” meaning that the service would continue to operate but with some amount of degradation of service. It’s a spectrum. Yesterday we saw that some services could not operate at all when Bing was down.

Disclaimer: I work at Brave.

Wait till you find out who makes some store-brand food.

If they branded themselves as what it is, anonymized bing search with better features, I would be totally fine with that. As is, even if they don’t go with that branding, I have no problem using the service under those terms.

"Bing but without copilot" alone would be a product worth having. But I think DDG has a few more things on top of that, like the `!bang` stuff and the bit where they don't send tracking data on you to Microsoft.

I really don't understand why any technically literate person would be impressed by the "!bang" stuff on DDG. You know your browser already provides this feature, right? Just configure some custom search keywords in your search settings. What does DDG offer that's better than this?

How do I set that up on iOS Safari?

Exactly. You can't except by setting DDG as the default search engine. Chrome, Brave, and FF for iOS also don't directly support this sort of feature either.

They already set them up for a ton of sites, though, and often you can guess your way to a bang on the first try. I have only looked at the actual list once or twice. Plus it seems to match the closest one in cases of using one that doesn't exist. I can also go to a machine that isn't my own and take advantage of them there right away. I heavily use the quickmarks in qutebrowser, but also the ddg bangs. I just hit O for open in new tab and can put in a bang right there.

Just to emphasize: When opan says ton, they mean a TON. Around 13 thousand, if I remember correctly.

They proxy the !bang things so that whoever provides the search gets less info about who you are. Less isn't nothing, but it is still better than your browser can do without the proxy.

>Just configure some custom search keywords in your search settings. What does DDG offer that's better than this?

Pre-configured keywords could be considered better than manually-configured keywords.

>Just configure

I might not be impressed, but as I get older, I skew toward out of the box experience over configuration. I rather learn, and then apply that knowledge everywhere the thing is used, than configure, and lug my configuration around. Therefore, I value an opinionated feature like the bangs in DDG, even though as a professional, I know that there are many alternatives to it. If I know how to do it with DDG, I can use it everywhere I load DDG: my personal computer, my work computer, my phone's browser, my phone's other browser, the browser in the VM, the browser in the VM in the cloud, on the freshly installed computer with its default settings, everywhere.

https://duckduckgo.com/duckduckgo-help-pages/results/sources...

> # Where do DuckDuckGo search results come from? // Most of our search result pages feature one or more Instant Answers. To deliver Instant Answers on specific topics, DuckDuckGo leverages many sources, including specialized sources like Sportradar and crowd-sourced sites like Wikipedia. We also maintain our own crawler (DuckDuckBot) and many indexes to support our results. Of course, we have more traditional links and images in our search results too, which we largely source from Bing. Our focus is synthesizing all these sources to create a superior search experience

Their previous answer: https://news.ycombinator.com/item?id=40454640

How can a poor peasant start using Bing APIs? They seem to cost a ton. Do I have to raise money before I even attempt a metasearch engine.

the most valuable service they provide is being a proxy for bing. they will never own up to it, but that in and of itself is an awesome service, as long as they're bing honest about your privacy.

I wonder if there are any duckduckgo/other search engine devs here to answer; why not keep a limited own index? Like, wikipedia, stack overflow, build up a cache of own-scraped sites for the top hits to popular search queries?

Am a being naive with how much is possible without going all-in and building an index of the whole web, or is it possible to have some kind of in-house fallback?

I believe that generally refers to all their custom answers, e.g. when you search for "weather <place name>". Nevertheless, those don't seem to work either.

I bet their engineers are currently discussing how to make any such answers possible without Bing working, assuming the upstream data for them is not in fact also Bing.

If the upstream data isn't Bing then why did they stop working in the first place?

No they always downplayed and tried to obscure the fact that it is basically a bing wrapper. Smoke and mirrors seems to be the main business strategy.

Absurd. They have been doing search for so long and couldn't build their own search index? I understand it's a huge investment, and clearly for some data sources you need to create partnerships and depend on others (e.g., youtube videos). But you would think that a search engine company would invest more in this area.

Do you really understand the scale of this (ongoing) investment? I'm not sure I'd classify this as "Absurd" - the value of DDG is not yet-another (tm) search index but the specific values they add on top. When you have limited resources I'd say they're making the right choice.

Brave rolled their own completely independent search [1] on what I assume is a relatively limited budget. It seems that regularly grabbing the data would be pretty easy. The harder part would be searching/ordering it in an efficient and meaningful way while avoiding SEO, but that seems more like a fun problem than a difficult one (if not both).

[1] - https://search.brave.com/

There are two things the general HN sentiment genuinely believes is impossible to build unless you have billions of dollars:

- Search Engines

- Browser Engines

I don't quite understand how we got there because neither of those things are impossible. Both are achievable with a small team and a couple years of runway. As proved by Brave and Ladybird.

It’s easy to make a crappy search and crappy browser, I think the sentiment is aimed at producing a useful alternative to google and chrome

In my honest opinion, Brave is a bit better than a crappy browser. They have their little crypto widgets, which you can disable very quickly if you’d like. Which depending on how hot blooded you are on the subject of crypto can be a turn off. But “crappy” seems a bit on the harsh end of the spectrum, no?

As I understand it, Brave shipped a white-labelled Chromium, and piggybacked their search results on Bing+Google until they had enough cached results to start using their own index.

These are both orders of magnitude easier than building a browser or a search engine from scratch (although they may require similar capital investment)

If you have a web browser that isn’t Firefox, it’s probably Chromium these days. Also irrelevant to the search product.

Regarding the search product, you’re suggesting they used to use Bing/Google but now don’t. … Does that not mean they run their own indexers? So you’re just diminishing their value today based on what they used to do?

Not as irrelevant as it might seem. They do run their own indexers now, but my understanding is that they didn't actually build the index that way. They used the Brave browser traffic to guide which pages to index, which was primarily driven by the Google/Bing search results the browser was vending at that time. it's more of an embrace-extend strategy than a build-from-scratch

The point of my previous response stands... What is the difference in how one reaches a destination, if they in fact did make it to that destination? Granted, this does read a bit like an ethical dilemma where people with different values may come to different conclusions. But if you use Brave search today, you are getting results sourced from Brave's index, and not Microsoft/Google's, so in my mind, how they got to this point is now irrelevant. It's just smarter to do it how they did instead of starting from square 1.

As proved by Mojeek; a full web scale international search engine built from the ground up with £3 million in funding, and no dependencies. Admittedly longer than a couple of years; 5 years as a hobby project, 10 years building, and more recently GTM [0].

I declare a bias as CEO, but the determination of our founder Marc Smith deserves more recognition, so here is the story up to 2021 [1]

[0] https://www.mojeek.com/about/technology.html

[1] https://blog.mojeek.com/2021/03/to-track-or-not-to-track.htm...

I wonder if there's anything in their contract with Bing that prevents this.

Exactly. See Brave Search, they are much newer but did it right.

Brave Search feels like Google from 2008. Just works, no BS.

Building an organic dataset with decent results is the expensive and hard part. Weather, wiki etc are the kind of things you can develop on a $20/m dedicated server.

I definitely felt misled, when I first learned about this, back when they censored Tank Man the same time Bing did[0]. I did remain a user though, mostly because I haven't felt like keeping up who the current good guys are. Lately I have been considering Kagi, but I don't like it that I need to log in on all my devices, and then I have to have a fallback, for when I'm not on my own devices.

So yeah, for my intents and purposes, DDG is a frontend to Bing. I do appreciate how uncluttered it is though, in comparison.

https://en.wikipedia.org/wiki/Tank_Man#Censorship

I do wish Kagi had less friction getting logged in on each device, but they have gotten it as streamlined as possible without patching the browsers. Your key goes in a query string so you can put it in any browser's new/default search engine config[1].

[1] https://help.kagi.com/kagi/getting-started/setting-default.h...

I just left Kagi. At first, it looked very nice, but for some reason, in a few weeks both Google and ddg gave me much better results. The new g web filter is actually quite good as well.

Yeah, exactly. They're mostly a front end for Bing. I know they have their fans, and this remark will probably anger them, but I have never understood the point of DDG.

I'm not angry but: privacy? Isn't that the point they're blasting all over their page and apps?

DDG UI is a lot calmer. Opening Bing feels like entering a Middle-Eastern market.

I created this account just to say thank you for this comment.

There are only 3 planet scale search indices I think. Google, Bing and Yandex. Everyone else is just rehashing those results.

Mojeek has its own index as well.

Data point: Bing was not down for me.

This. At some point in the past I was giving a "test drive" to DuckDuckGo. And the moment when I realised that they are a Bing wrapper was the end of it.

comparing their search result with bing for at least the first 3 or 4 pages you'll see a very high correlation to bing's result (as long as you're logged out of bing). The main thing is they proxy your search for you and give half way decent results like bing.

I keep meaning to make a Chrome extension called DDGoogle that would reskin Google results in the style of DuckDuckGo. This way people who insist on using DDG wouldn't have to endure Bing-quality results.

Firefox users can create their own "bangs" with bookmark keywords. Just bookmark https://example.com/%s and then assign a keyword to it from the Library window (full bookmarks manager).

Or right-click in any search box on any website, choose "Add keyword for this search".

Took them several hours to put that message there though.

Often the case with US-based companies that have outages during "office hours in Europe". We, europeans see it going down. Communication on why, how and ETAs only appear at the start of the US day.

This take implies there is no on-call rotation at DDG, which I find suspicious

the most important function of on call rotation is there to fix problems, not to make announcements to the public. so they surely have the first, but maybe not the second.

Well, maybe that perspective explains why it takes 6+ hours to update any status page then: "welp, shit's broke, better wait for the management team to wake up to tell anyone"

more like: wait for them to decide exactly how to present this to the public.

This is incorrect, the primary job of an on-call rotation is to satisfy customers.

It’s usually much better ROI to publicly acknowledge an issue if the resolution is not going to be single digit minutes, as it massively reduces the incoming query/support burden

yes, but the users of the search engine are not the customers. actually, who would the customers of ddg be?

ad agencies? are they going to get upset if their ads are not visible for a few hours on of many sites where they post them? are they even going to notice?

DDG has a global remote workforce so this wouldn't seem to apply in their case.

(just guessing) the directors might be in US though

My company has a global remote workforce. The only team of which that owns any infrastructure being the team I am on, and all of my peers on that team being based in various parts of the US. So it’s not as simple maybe as you might think.

DDG has relatively large pool of European employees.

They should put an if results = 0 then post helpful message instead of try broken search again?

Nice, but they totally missed the opportunity to say 'Thanks for your patience while we get our ducks in a go'.

To be fair, DDG uptime seems about 5-6 9's. P0 outages happen but DDG seems to have them maybe once every couple of years or so. That's pretty excellent effort.

I thought I was easily entertained.

I'd love to see the DDG engineering team at work - I'm guessing no nonsense, smart folks who can poke fun at themselves but still get things done. Contrasting to what I've personally seen at a certain other search giant.

The notice helped me realize that Yahoo! Search still exists xD

Doesn't yahoo also use bing under the hood?

get it because ducks

Reply on news.ycombinator.com