Bing contract prohibits DuckDuckGo from completely blocking Microsoft tracking
twitter.com
twitter.com
This is not about search. To be clear, when you load our search results, you are completely anonymous, including ads. For ads, we actually worked with Microsoft to make ad clicks privacy protected as well. From our public ads page, "Microsoft Advertising does not associate your ad-click behavior with a user profile." This page is linked to next to every Microsoft ad that is served on our search engine (duckduckgo.com). https://help.duckduckgo.com/company/ads-by-microsoft-on-duck....
In all our browsing apps (iOS/Android/Mac) we also block third-party cookies, including those from Microsoft-owned properties like LinkedIn and Bing. That is, the privacy thing most people talk about on the web (blocking 3rd party cookies) applies here to MSFT. We also have a lot of other web protections that also apply to MSFT-owned properties as well, e.g., GPC, first-party cookie expiration, fingerprinting protection, referrer header trimming, cookie consent handling, fire button data clearing, etc.
This is just about non-DuckDuckGo and non-Microsoft sites in our browsers, where our search syndication agreement currently prevents us from stopping Microsoft-owned scripts from loading, though we can still apply our browser's protections post-load (like 3rd party cookie blocking and others mentioned above, and do). We've also been tirelessly working behind the scenes to change this limited restriction. I also understand this is confusing because it is a search syndication contract that is preventing us from doing a non-search thing. That's because our product is a bundle of multiple privacy protections, and this is a distribution requirement imposed on us as part of the search syndication agreement. Our syndication agreement also has broad confidentially provisions and the requirement documents themselves are explicitly marked confidential.
Taking a step back, I know our product is not perfect and will never be. We face many constraints: platform constraints, contractual constraints (like in this case), breakage constraints, and the evolving tracking arms race. Holistically though I believe it is the best thing out there for mainstream users who want simple privacy protection without breaking things, and that is our product vision.
Overall our app is multi-pronged privacy protection in one package (private search, web protection, HTTPS upgrading, email protection, app tracking protection for Android, and more to come), being careful (and putting in a lot of effort) to not break things while still offering protections -- an "easy button" for privacy. And we constantly work to improve its capabilities and will continue to do so, including in this case. For example, we've recently been adding bespoke third-party protections for Google and Facebook, like Google AMP/Topics/FLEDGE protection and Facebook embedded content protection.
Yes, it is. Your competitors in the privacy-centric browser space don’t have this restriction because they’re not search engines acquiring the majority of their data from an entity with a conflicting interest.
I’m inclined to blame Microsoft here; this is a nasty move on their part. However, your stance is problematic. This is a problem, and it’s a serious one. It undermines trust in a product that claims to be the bastion of privacy. And statements like this…
> Overall our app is multi-pronged privacy protection in one package (private search, web protection, HTTPS upgrading, email protection, app tracking protection for Android, and more to come), being careful (and putting in a lot of effort) to not break things while still offering protections -- an "easy button" for privacy.
…don’t help the matter. To me, that just sounds like marketing mumbo jumbo. Ultimately, if a privacy-centric browser is contractually obligated to load tracking scripts and is required to avoid disclosing that fact, I want absolutely nothing to do with either party.
In terms of our app and multi-pronged protection, it isn't mumbo jumbo. Our app is way more than just a browser (and increasingly so). For example, the app tracking protection mentioned for Android blocks trackers in all your other apps. The email tracking protection blocks trackers in your email (that you read in your regular email client/app).
I understand the concern here that we are working to address in a variety of ways, but to be clear no app will provide 100% protection for a variety of reasons, and the scripts in question here do currently have significant protection on them in our browser. From the comment "That is, the privacy thing most people talk about on the web (blocking 3rd party cookies) applies here to MSFT. We also have a lot of other web protections that also apply to MSFT-owned properties as well, e.g., GPC, first-party cookie expiration, fingerprinting protection, referrer header trimming, cookie consent handling, fire button data clearing, etc."
That's why the ad blockers that stop the scripts from loading to begin with will always due a much better job than the extra "mumbo jumbo" you're relying on. That stuff should be a fallback for when scripts slip through the filters, not the primary means of protection.
What's actually happening is you're forced to allow Microsoft scripts which do indeed do telemetry on users despite some restrictions you put on them, and they're still effective because fingerprinting works. That fact is embarrassing for a product you're trying to sell as promoting privacy so there's this mildly deceptive attempt to hide what's going on with lots of words and claims of protection instead of straightforward disclosure.
https://www.cisa.gov/uscert/bsi/articles/knowledge/principle...
Isn't this entire story about them disclosing this fact?
It seems to be, but they're claiming the details are confidential. It's rather confusing. I wonder whether Microsoft's intention was to prevent them from disclosing it altogether, or whether they just wanted to avoid the general details of the contract getting out (rather than this particular tidbit of info). I'm inclined to suspect it was the latter--just a general NDA. In any case, I don't like it.
What’s more helpful is to hear in which exact situations their blocking doesn’t work.
Just because other avenues exist doesn’t mean people walk them
> Ad clicks are managed by Microsoft’s ad network.
> Microsoft and DuckDuckGo have partnered [..] Microsoft Advertising will use your full IP address and user-agent string so that it can properly process the ad click and charge the advertiser
It seems DDG is not that privacy focused when it comes to ads.
[0] https://help.duckduckgo.com/duckduckgo-help-pages/company/ad...
"Microsoft Advertising does not associate your ad-click behavior with a user profile."
Does somebody else besides Microsoft Advertising do it? I'd guess so.
Is there another kind of association besides a "user profile" which has substantially similar concerns for an end user? I'd guess so.
This is all coming off as an attempt to cover up what's really going on with deception. That might not be the case, but if it were, this is exactly how I expect a "privacy focused" organization to communicate when they had been corrupted by a compromise to a third party.
It also ignores that governments like the NSA have tapped these very networks for data (this is what prompted Google's internal SSL drive). Even if we trust the legal entity, the fact is that the information itself is a target and so are those entities. It is always safer not to send the data, but in this case you're explicitly sacrificing that safety to benefit your ad partners.
What did you want—for the URL to go straight to the destination page with no redirect through an ad-network analytics provider, making your impression invisible to the network and thus unable to be costed against the advertiser? Why would any ad network even bother to participate in such a scheme? How would they make money? Prepayment for an arbitrary guess at predicted click-through count?
Still, millions of users do click those ads, because if nobody did, DDG would not exist. A less tech savvy user, who is probably DDGs main target, came on the promise of privacy and does click those ads and is also being tracked around the web by Microsoft if they use DDG browser (from what I understand).
This is less than ideal from the standpoint of "privacy simplified" promise, but really no other way around it when selling ads is your business model.
But because they are tech savvy, they the the ones friends and family ask what browser/search engine to use, so you end up with 20 more less tech savvy people on the platform, and they are probably the ones that end up clicking on ads (because, again, DDG is making a ton of money with that)
That brings us back to: What does Microsoft considers accounting purposes?
Fingerprinting the user/browser can be used for valid accounting purposes like identifying the user to prevent ad fraud.
From https://news.ycombinator.com/item?id=31492789:
> "Bing search contract prohibits DDG browser from blocking Microsoft tracking scripts by default."
I've taken a crack at a shorter version. Thanks!
You may be making it worse. Really need to dial down on click tracking (or, at least respect the dnt header).
Ex A: Searching for Cristiano Ronaldo (from Chrome Incognito but not Firefox, amusingly) returns this horrible href:
duckduckgo.com/l/?uddg=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FCristiano_Ronaldo&rut=4a9ada2347e29c8fce96a95bde34e6343c279202dbc22b4fe61524ab39bf8eff**
When you click on a link in our results page, your search terms are not sent to the site that you click on, which can be the case on other search engines due to something called HTTP "referers".
On modern browsers we accomplish this by adding a small piece of code to our page called Meta referrer. Some browsers (especially older ones) do not support this standard, however. For those browsers, and also in situations where meta referrer doesn't work, we send the request back to our servers to remove search terms. This redirect goes through r.duckduckgo.com.
You can disable this privacy feature. To do that, go to the settings page, select Privacy, and change the option Redirect to Off.
**
> ...generally in very old browsers that need to use our non-JavaScript site (http://duckduckgo.com/html).
I use duckduckgo.com/html & duckduckgo.com/lite on all my (up-to-date) browsers (Firefox Mobile for Android / Chrome for Debian as two examples); they are "not very old" at all, and I still get ddg-proxied hrefs.
A feature request (if I may): Old browser or not, if the dnt header is set, I'd ideally want ddg to not proxy/redirect anything at all on my behalf.
Go to https://duckduckgo.com/settings#privacy and disable the bottom setting, "Redirect (When Necessary)"
I'm not on an older browser.
I'm not using private tabs / incognito.
I haven't cleared cookies.
Really, Referer-related privacy problems should be fixed in the browser and any browser that still sends cross-origin Referer headers by default cannot claim to care about privacy - and that includes Firefox.
> Some browsers (especially older ones) do not support this standard, however. For those browsers, and also in situations where meta referrer doesn't work, we send the request back to our servers to remove search terms.
Disabling javascript is one of the first things to do to take back control of your privacy so you deciding to leak more data for those users who make that choice is not a good look.
You're not making any sense. Proxying all requests is the only way to shield you from being tracked by third parties. If DDG wants to track you they don't need some convoluted dance - you're already on their website.
I don't trust their website either but the user-agent that I use that has enough anti-tracking measures I trust (whether those might be defeated is an orthogonal topic). The redirects through their servers... I cannot control what runs on it, just as I cannot control what terms they sign up with Microsoft.
Huh? Ive just checked with multiple links, it’s quite fast. I don’t think that user experience that can be measured in fractions of second can be referred as “terribly slow”.
How’s the latency of ddg redirects depends on the region?
> given the number of times I use ddg in a day
What’s you estimation? A hundred? It’s something like a minute or two of accumulated time. It’s not even worth mentioning.
Also I doubt that the redirect delay should be taken in account at all. The workflow may vary (obviously) but I normally open a link in a separate tab. By the time you click on this tab all the redirect work has finished. What’s your workflow and how does the redirect delay impact it?
(you.com founder here - not getting access to kagi ;)
- You.com uses affiliate links.
- My ad blocker (uBlock origin) shows that you.com has tracking in the form of analytics (https://plausible.io/api/event).
- you.com makes a lot of requests to many 3rd party domains.
- Kagi lets you quickly create rules to customise results such as domain/url weighting, term filtering etc...
- I personally find the UI/design off-putting - it's very busy and the round style elements immediately remind me of the windows XP theme.
- The popup in the right corner nagging to "Make You.com your default" is very off-putting.
- you.com promotes the use of Google Chrome.
- you.com loads results slower.
- I'm not a fan of the square cards on you.com.
- Kagi has a clear and transparent business model, you.com seems pretty up in the air as to how it will be funded in the future, including statements such as "You.com currently has no ads." - currently is the key word there.
- It's not clear if you.com uses data from other search engines.
No offence intended with any of my remarks.
But this is exactly the problem. Sure, unlike Google DDG is not itself collecting data, and there appear to be limited tracking on MS properties, but unless I misunderstand the situation (a decent possibility) then the vast majority of the web, which are not MS sites, are still able to use MS scripts for tracking.
You are marketing a privacy-centric ecosystems of tools but your partner in one component (search) is preventing you from implementing that vision in non-search areas, so that should be clear. It should also be clear that it's still very much a search problem. The source of the limitation has search as a root cause, and a massive corporation with just as much interest in obtaining data on user browsing habits is still able to do so in some ways.
I admit this is still a better situation than Google, but you're providing an ecosystem of tools, they are inextricably linked with each other.
I don't have any proposed solution. I'm not sure there needs to be one aside from making boundaries clear. I still see significant value in your offerings. Partnering with a provider of quality search that solves some but not all privacy issues is still valuable. Each person chooses their own level of comfort & tradeoffs between product quality & privacy, and you offer what I consider to be a valuable middle ground in that range. But let's just be clear on what the middle ground is made of, though I otherwise do not judge harshly for an agreement like this.
Thank you for making great tools.
All this constraint is doing is limiting the increase in privacy you get from using the DDG mobile app on top of the privacy you get from using the DDG web search provider. At worst, DDG searches in the DDG app will be no less private than DDG searches done in any non-privacy-enforcing browser, e.g. Chrome. Which is to say—still pretty private.
Also, I presume that only a minority of DDG users are users of the DDG mobile browser app. (I didn’t even know it existed!)
A shorter answer would have more credence.
What do you think the title should be yegg?
Maybe something like:
Microsoft contractually prevents DuckDuckGo's browser from stopping Microsoft scripts from loading on 3rd party sites (FYI: not search related)
Thanks for making a definitive suggestion. I hate when someone knows something is wrong, but can't articulate what would be "right" (correct).
I agree with the first part of the title, but this part seems like you're going out of your way to defend yourself. The mention of "DuckDuckGo's browser" should already imply it's not search related.
I think for transparency sake, it could be helpful to list the Microsoft trackers that were essentially white listed and therefore allowed to load on a particular site, right under the list of trackers that were blocked.
This is what I've gathered from running uMatrix for years.
Additionally the way it is phrased implies Microsoft trackers get a free pass, when they are in fact heavily restricted, e.g., blocking 3rd party cookies, fingerprint protection, etc.
And the current title can further easily be misinterpreted to be about more than Microsoft scripts on 3rd party sites (e.g., other companies, which it is not).
As a long time DDG user, my stomach turned when I saw this. Following the link to Twitter, it required a lot of digging to find what was really happening.
For those of us using DDG search - this is a big nothing burger. For folks using DDG browser, this is misleading at best. The difference between the title and reality, from my understanding, isn’t nuance.
My reading of this title (and Twitter) made me believe DDG was sharing user data with MSFT across all of their properties (including search) by serving users MSFT trackers with DDGs content.
This looks like a textbook brand extension [1] issue.
Your brand is privacy. You built it on your search product. You're compromising those principles, perhaps reasonably so, in extending the search product's brand to a browser. This is coming back to bite the brand, search and all. (Per the Wikipedia article, it's highly recoverable.)
But you won’t, because you are explicitly not working to make these protections “as good as they can be”. You are working to capture the user’s entire session and monetize owning that. (I’d like to know the eventual purpose of owning that, and personally I’d like to see subscription somewhere, because any other monetization is likely to be user-hostile.)
FWIW, I’m super annoyed with you. I’ve gotten countless normals — and certain enterprises with 10k to 100k users — to switch defaults to you, and they now trust you (thanks to trusting me) on privacy in a simple binary way. Which you’re proving wrong.
Now I have to do diligence before recommending your brand, and that’s shitty.
This is a reasonable position. The shift in positioning that's driving the confusion is real, though.
DDG (search) has an almost absolutist stance on privacy. That was differentiated. The nuanced tradeoff you describe, between privacy and convenience, which I agree boosts the actual outcomes, is something else. It's more similar to Apple's philosophy. Which is fine. I use their products as well as yours. But it's different in a fundamental, and to many a meaningful, way. That's going to be difficult to brush away without making it look like there's something to hide. (None of this could be said to have been predictable ex ante.)
this change in emphasis has been palpable in the 4 Ps (marketing strategy) of duckduckgo over the past few years.
To answer your question though, comprehensive privacy protection prevents data profiles from getting created about you, which in turn prevents ad and other content targeting. This targeting, regardless of how it's done, enables general manipulation (e.g., exploiting personal characteristics for commercial or political gain), filter bubbles (e.g., creating echo chambers that can divide people), and discrimination (e.g., people not seeing job opportunities based on personal profiles).
More generally though, I view privacy as protecting you from coercion. Yes, it protects personal information, but that's not the real point. The real point is autonomy -- the freedom to make decisions without coercion. From this perspective in addition to helping reduce identity theft, commercial exploitation, ideological manipulation, discrimination, polarization, etc., it also helps reduce self-surveillance (i.e., chilling effects), and just general loss of freedom (e.g., mass surveillance).
You want me to pitch my mom on un-targeted advertising? How do you phrase it in practice? "On Google, you get evil ads relevant to you, such as restaurants near you. On Duck Duck Go your privacy is protected, so you get ads for restaurants in Omaha, Nebraska. Therefore you should switch to Duck Duck Go". Something like that?
This comment is based on the actual results I was served by DDG for "best burger".
I honestly do not understand the pitch, that's why I want to hear it from the horse's mouth. Scare words like "tracking" and "profile" and "targeting" are used by the privacy fear industry to disparage the practice of having implicit terms in your search query. These implicit terms greatly improve search quality, which is why the results on Google are so much better. Advertisements are their own separate search corpus where good ranking is desired and the implicit elements of the search vector are also helpful there. To me there can be no rational case made that omitting the implicit terms improves the quality of the result.
I'm not arguing that duckduckgo/bing are any better, just that these tracking convenience features have a dark side and many times work against your best interest.
1. Never heard any rant about it outside tech circles.
2. I've given DDG many chances when Google failed to return satisfactory results. In those many cases DDG results were just about the same or even less relevant. Google changing the query? Well DDG either also changes it or returns irrelevant results not containing the query anyway.
The single advantage of DDG I've noticed is that it doesn't CAPTCHA me on a VPN connection.
Lots of people (most people?) want to do the bare minimum with computers. Sacrificing convenience for privacy or whatnot isn’t something they would accept.
In any case, searching instead for "best burger near $CITY" doesn't seem terribly difficult (and that's in fact how I generally write my queries).
Please put your second paragraph up at the top of that page, maybe with some bullet points and icons and I'll send out the URL.
Your brand is privacy, and you have betrayed your philosophical principles.
Personally: you will never regain my trust. I'm sorry this happened.
https://pplware.sapo.pt/internet/duckduckgo-apanhado-a-dar-p...
Party #1: Me
Party #2: DDG
>currently prevents us from stopping Microsoft-owned scripts from loading
How is this not allowing 3rd party (Microsoft) tracking? Are they loading the scripts from DDG's servers?
Fool's gold. Privacy is never easy.
Private browsing is a small niche, and Brave does their best to drive competitors at every turn, and not by being obviously better at it. Kinda scummy, if I’m honest.
https://www.reddit.com/r/mildlyinteresting/comments/utgukp/i...
It's just as easy as selling bunker-beds and ammo for doomer-preppers (while stroking their ego).
But, shhh don't tell them and hurt their ego
At the very least I think it relegates people to buckets of either "People who believe tin foil hats give them privacy", and normal people.
Either way I don't think it's a productive dichotomy.
https://www.reddit.com/r/mildlyinteresting/comments/utgukp/i...
It should also be mentioned that using private information for unsolicited advertisements is something which is likely in conflict with GDPR.
This HN submission links to a tweet by a Brave employee. However, that tweet is just a screenshot of replies to the thread at https://twitter.com/thezedwards/status/1528808759027331072 written by a researcher who doesn't appear to be a Brave employee. I think it would be better if the link were directly to the tweet by Zach Edwards instead.
By posting a screenshot, the poster removed all context. Even being charitable, it's not a good luck for Brave.
I was really talking about this HN submission, not the tweet. That person's Twitter account only has 600 followers.
What you're actually upset about is that someone pointed out their hypocrisy?
I don't think it's great that Microsoft is exempt from some restrictions in the DDG browser, but this tweet is also referencing a post by a DDG employee freely disclosing the issue and stating that they're working to improve it. In my opinion, this sort of mudslinging makes the folks at Brave look petty while not really changing my opinion of DDG very much. I also think this is a case of letting the perfect be the enemy of the good.
Feel free to continue to tell me how I feel about things, though, internet stranger. You're clearly much more in tune with my opinions than I am.
Until DuckDuckGo separates itself from Microsoft and becomes truly independent, especially in its business model, you have to question why DDG even exists.
DDG was founded 14 years ago. I can understand initially bootstrapping on MS ads, but what's the excuse now? How about separating yourself from Microsoft first, before making a web browser that gives special exemptions to Microsoft?
As a consumer if you're happy with DDG's results this may not be relevant, but it doesn't seem like a great long-term strategy for DDG.
In my opinion you have:
- Google: best search results but you’re profiled to death - Bing: okay-ish search results and you’re profiled to death - DDG: okay-ish search results and you’re barely or not tracked at all
Easy choice for me.
(founder here)
DDG exists to make money for itself. It doesn't exist to protect your privacy.
From google to github to mozilla to everything, you would think the tech idealism would have died already. People working in tech, especially the elite, are some of the slimiest and greediest people on earth. Where money goes, so go the greedy slimeballs. It's pretty much a law of nature.
I don't think these are mutually exclusive. It all depends on how you make your money.
I would love to advertise on a search engine that's independent of Google and Microsoft. Unfortunately, DuckDuckGo is not it.
Its been useful for me for 14 years so idk about that.
[0] https://news.ycombinator.com/item?id=30703172
How is it unsurprising? Where on https://duckduckgo.com/ or https://duckduckgo.com/about do they mention Microsoft or Bing?
In fact they go out of their way to mention Google without ever mentioning Microsoft:
"Is DuckDuckGo owned by Google? No, we are not and have never been owned by Google. We have been an independent company since our founding in 2008 and, unlike some other search engines, we don’t rely on Google’s results for any of our search results."
It is unsurprising. They have openly admitted it here [0], no hiding that fact. From [0]:
We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).
What I'm claiming is that the general public, including people who use DDG as their default search engine, are generally unaware of this relationship. And also that DDG doesn't go out of its way to highlight this relationship, even though they do acknowledge it in relatively obscure places.
If you're moving to DDG, it's most likely for privacy-related reasons. If that is the case, would you not do the absolute minimum due diligence by reading some of their main explanatory pages (e.g. where sources are from, how ads work, etc.)?
I find it difficult to take someone seriously when they are complaining about a niche privacy-focused search engine, but don't seem to actually take their privacy seriously. The first step that should be done when using a new service/product, if you care about privacy, is to read the privacy policy and related documentation.
Because Microsoft's reputation is lower than dirt and that's probably a big part of the reason why so many people mock Bing and refuse to even try it.
Well... how about stopping all this "Bing on the background" thing and do like Brave search and Qwant (which i'm testing as to switch away from ddg for a few months now - because of you relying in Bing) and start believing a bit more on your own index???
Why not start being a "real" search engine???
I would say it's about time!!!
(If brave and qwant can do it, so can you - man... even Gigablast does it!!!)
On other search engines, they all rely somewhat on either Google's or Bing's web crawling: Qwant, Bing and Brave, Google (and Bing for images). This is easy to see as a webmaster since you don't see their crawlers much (if at all). Only Google and Bing are doing full scale web crawls. However, search is a lot more than traditional web links -- in fact it is about half now from instant answers that can come from dozens of sources and indexes (which the above comment gets into).
That's just corpo-speak. For the most part, Duckduckgo is Bing with some additional features. That's true to the extent that when Bing decided to censor the Tank Man image, it was removed from your results too. [1] Not that you guys refrain from censorship yourselves. [2]
The crawler (DuckDuckBot) doesn't have much of an impact on the search results, it's mainly used to provide instant answers. [3]
[1]: https://www.theregister.com/AMP/2021/06/04/search_engine_tia...
[2]: https://nitter.net/yegg/status/1501716484761997318
[3]: https://seirdy.one/2021/03/10/search-engines-with-own-indexe...
I'd like to correct some factually incorrect information regarding Brave Search.
Brave search crawls the web through the Web Discovery Project and has its own crawler, which fetches a bit more than 100M pages daily.
Brave search uses Bing API and Google fallback for about 8% of the results shown to the users, the remaining 92% are served from our own index, when we launched almost 1 year ago the number of results from 3rd parties was 13%.
There is no need to mention "multiple source" when a number can be given. The underlying theme here is not if DDG provides no value on top of Bing, it does, no one is questioning that. The question is whether DDG would be able to operate if Bing were to shut DDG down tomorrow.
If Bing and Google were to disappear tomorrow, for whatever reason, Brave search would continue to operate, that's the independence Brave search is building.
I was just hopping that, since it was based upon tailcat, it would create a bigger disruption by being opensource like Gigablast.
(However, i understand that, from a financial point of view, that's a risky move. But like DDG... i hope that maybe someday it will be)
Yegg said "they all rely _somewhat_ on either Google's or Bing's web crawling" and you confirmed it by saying "Brave search uses Bing API and Google fallback for about 8%". So... which part is factually incorrect?
Edit: Misread the second part, removed that portion of my statement.
No, that doesn't appear to be the question at all. The original post appears to be an attempt to smear DDG by posting misleading information that you know will confuse users into thinking that their search engine sends PII to Microsoft when you know it doesn't. The original tweet doesn't appear to mention Bing shutting down at all. Here's the entirety of the tweet:
"This is shocking. DuckDuckGo has a search deal with Microsoft which prevents them from blocking MS trackers. And they can't talk about it! This is why privacy products that are beholden to giant corporations can never deliver true privacy; the business model just doesn't work."
I see nothing in there questioning whether DuckDuckGo will still be around if Bing goes under. I also see nothing in yegg's response above that has anything to do with this irrelevant question you mention.
"it is misleading to say our results just come from Bing."
Discussing how many sources can one bring together it's a distraction to not discuss the degree of dependency between DDG and Bing. More-so when claiming that others suffer from the same, which is factually incorrect for Brave search.
But what i would REALLY like to see on DDG (beside it becoming fully OpenSource - someday) was having it taking the step of not relying on Bing.
That is a terrible shadow over you.
You should embrace the momentum you're having and step up and do your own thing. Your own index. Yes, you have to build/rely on others such as wikipedia... naturally
But please... NOT on big tech!!!
Or one day, someone else will get there and eat your lunch (Honestly, Qwant is a great alternative that apparently does not rely on big tech)
I maybe wrong, but i think this is what everyone one wishes from DDG!!! I believe everyone wants DDG free from Big Tech
(but everyone is free to correct me).
"Qwant uses different programming interfaces such as those of Microsoft Bing, Twitter, YouTube, or iTunes" -- https://about.qwant.com/en/legal/classement/
Is Gigablast, really, the only "alternative"?!?!?! (in the meanwhile, since there is no advantage, i'll be back to DDG....... for the time being...)
Still, my point remains: we could, at least, have a version of DDG that would only use it's own index!!!
Isn't that possible? (I would be using it!!!)
Independent take: https://seirdy.one/2021/03/10/search-engines-with-own-indexe...
> [your] results just come from Bing
You're the one being misleading here by suggesting that they did say that. Nobody is interpreting what they said as "All search results from DDG are just straight from bing". You're nitpicking words in almost every one of your responses.
> if you are not paying for the product then you are the product
I have switched to Kagi [0] a paid search engine (free in beta) as my default search engine and so far it has been working out great.
A better option would have been to let the community decide. You could have easily posted something to the effect of "One of the search engines wants us to sign an NDA and force us to allow more tracking than we are comfortable with"
Then let the community decide if we wanted a branded browser that is less secure or even if enough folks didn't care that you could still justify dev cycles on the browser.
No you took the shady approach and that is sad.
Microsoft aside, does the DuckDuckGo browser provide any privacy benefits over this setup? What keeps DDG users from switching to Firefox?
you can dive into google/ddg/etc's searches on the left, but its pretty decent!
Those are the saddest 5 words I have read all week.
Et tu, Brute?
"Partnered" is not a word one uses in connection with convicted criminal monopolists with a history of bribery, intimidation and fraud.
I choose my words carefully - Microsoft are gangsters who would sell their own grandmothers for beer money. For DuckDuckGo to be associated with them is a disgrace.
You can plausibly make the same argument about downranking sites like CNN or fox news.
The people who order counterfeit pills from "canadian pharmacy" sites probably don't consider those sites spam and could plausibly argue that they shouldn't be downranked.
Now they have deviated from that vision so I could just as well use Google instead.
Considering most propaganda we see are US/EU propaganda, wouldn't it be better to downrank US/EU 'news'? Why target one propaganda but not the others?
This approach tends to produce some false positives, as there are news sites that don't fact check stuff before repeating it, and they end up being indistinguishable from state-sponsored propaganda to outside observers. I'm not sure how much it matters in practice if sites that routinely publish incorrect stories accidentally get misclassified.
https://www.washingtonpost.com/business/economy/russian-prop...
Who is it that performs this service? And do all search engines consume the same classifications?
Considering how much obvious propaganda from Western sources I consume on a daily basis, I'm more than a little skeptical that they can be trusted to label what is and is not true from geopolitical rivals.
Search engines shouldn't be in the business of picking what content I see based on the owners of the search engines politics.
Most other search engines are just reverse proxies for Google or Bing, so they inherit those biases.
We actually do not intentionally censor any news results, meaning media outlets are not being removed or their stories displayed so far down in the results they are effectively removed. That is, unless legally prohibited, you should find all media outlets in our results, and they should generally show on top if you search for them by name or domain name. If you are seeing otherwise, please me know and we will investigate.
A search engine's primary job is to rank results, trying to put results that most quickly and accurately answer the query on top. We do this ranking in a strictly non-partisan manner. Ranking for news-related searches is particularly difficult because for most news stories there are often hundreds of media outlets covering the same story, many with similar relevancy in terms of keyword matching and popularity. As such, we look to another ranking factor to ensure just the top of the results aren't taken by obviously very low-quality news results so that users have more sources of relevant, high-quality news results to compare and choose between.
The non-partisan factor we've found to help accomplish this is a rare, but well-documented history of a site's complete lack of news reporting standards, such as routinely using spam or clickbait to artificially inflate traffic, consistently publishing stories without citing sources, censoring stories due to operating with very limited press freedom, or misleading readers about who owns, funds, and authors stories for the site. And since we do not censor sites, even state-sponsored media in countries with very limited press freedom, these sites will still show up in results, and even on top like when you search for them directly.
I'm curious of the implementation. Are these sites in a list and that alters their ranking in the results? Or some other approach?
Would also be very curious to hear an answer to this. Its been long rumored that Google has weights or boost/throttle values that are manually assigned to websites by humans which impact ranking. Would be great to understand if DDG is following in Google's footsteps.
If you are in the EU, though, the EU legally prohibited search engines from linking to RT & Sputnik.
They prohibited broadcasters [0]. The legality of extension to search is disputed [1]. The EU sent a request to Google [2]. Microsoft announced changes including "further de-ranking these sites’ search results on Bing" ahead of the EU sanctions and PR [3].
[0] https://www.consilium.europa.eu/en/press/press-releases/2022...
[1] https://twitter.com/tjmcintyre/status/1501596542624485382
[2] https://www.lumendatabase.org/blog_entries/notice-in-lumen-r...
[3] https://blogs.microsoft.com/on-the-issues/2022/02/28/ukraine...
That must be it then, I wasn't aware. Thanks for correcting me.
You weren't wrong. Try to get RT to show up without specifying "RT"/"Russia today"/etc. I can search RT article titles with quotes but the original RT link will be on the second page if at all.
Now repeat this experiment with the NYTimes and compare the outcome.
DDG may not delist RT but you would not be a fool to suspect DDG throttles the visibility.
You get lots of spammy stuff (top link is a site offering to sell a translation of RT), site-ranking sites (that rank RT), some rando lady quoting RT's meta description on Twitter (!!!), and even RT's pages on Twitter, Linkedin. And then it's finally there, just before getting into really high quality results like a page on rotten tomatoes ranking the 200 Best LGBTQ+ Movies of All Time.
"Curating" results is a great way to completely break your own search engine.
If I was designing a search engine, I wouldn't use keywords in meta tags for ranking. It's too vulnerable to keyword stuffing attacks.
Aren't all search engines in the business of picking what content you can see based on the politics of the owner?
There is obviously a massive problem with modern disinformation tactics, to the point that if you deny it, I presume you have an agenda to sell. There are most definitely terrible things that should not be legal to knowingly include in search results, ever (yes, actually, do think about the children, etc). So all reasonable people already accept that some things should be censored, as a moral and legal obligation. The correct solution is not as simple as unfettered libertarianism would have you believe.
Thus, agree or disagree with that action, I cannot reach the conclusion that they have somehow broken an inviolable code, especially since all algorithms include publishing choices anyways.
You lost me when you started telling us that you were going to start deciding what is unacceptable "russian propaganda" without any transparency into what that means or who will be deciding. Completely antithetical to your anti-bubble mission. It pains me to not be a cheerleader for you anymore, but that was such a betrayal that I can't get past it.
On the search filter bubble, that is very still much what we do. To be clear, unlike some other search engines, we don’t alter search results based on someone’s previous search history. In fact, since we don’t track our users we don’t have access to search histories at all. Those other search engines show you results based on a data profile about you and your online activity (including your search history), and so can be slanted towards what they think you will click on the most based on this profiling. This effect is commonly known as the search filter bubble, but using DuckDuckGo can help you escape it. This does not mean, however, our search results are generally “unfiltered” because, for every search you make online, a search engine’s job is to filter millions of possible results down to a ranked order of just a handful.
It's the difference between hiding results, and hiding that you are hiding results. (Yes, I know it was disclosed on twitter. It should be disclosed on each results page, like Google's "some results were omitted because DMCA etc" disclaimers. That is precedent for this sort of disclosure.)
The former is necessary. The latter is... smelly.
That's one reason I moved over from Google. I sure didn't feel lucky using THAT search engine.
What search engine doesn't try to detect and downrank sockpuppet spam? How's that working out for them?