Improving DuckDuckGo
duck.co
duck.co
If you'd like to chip, you can donate to the various privacy initiatives they support here:
https://www.crowdrise.com/duckduckgoprivacychallenge
(and here)
But, I very much want to reward the people who create the things I use.
So I'll never whitelist any site or service, but I'll absolutely donate or make sure they get paid in other ways (and usually willingly donate more than they'd get from showing me ads).
Economically, I'm not sure we're ever going to get away from ads. As I've said before in some of my comments, it's not just about paying for things you use. It's about paying more than the people who are willing to put up with ads, because otherwise companies will just do both: charge, and use ads. That's what we see in TV, in addition to many other places, and there's no reason to expect that not to continue unless there were widespread backlash, and I just don't see that happening. I don't like ads, but most of my friends don't seem to see things the way I do.
Why? Was there no human economic activity before they were ads?
If spaceships appeared in the sky, and shot lasers at all ads, and anyone making new ones, would we find no way to make and sell and buy things regardless? That seems less likely to me than said spaceships.
> I just don't see that happening.
True, and saying "it should be possible" is not the same as knowing how to get from here to there. But I wouldn't say "never ever" in this context.
A link posted on HN a while ago. Long story short it uses a raspberry pi as a DNS server which blocks all requests for ads.
Let’s be honest, the kids of hacker news commenters are probably going to be fine. It’s the way ads influence the unwashed masses that becomes problematic.
It's also just a thing we keep in mind and work around in life. We won't use any service that has ads, or any site that finds ways around the Pi Hole, or watch shows/movies with heavy product placement, etc etc. It's just part of mine and my wife's core beliefs (privacy in general) so we just guide our kids towards "good" content and taught them from the minute they could understand, that "ads are bad, we don't like ads".
don't allow commercial TV (they only watch Netflix or ABC Kids app
Even PBS is "commercial", given the growing practice of underwriter ads.
All that I had to do was assign a static IP address to the MAC address of the VM network interface, and configure my router's DHCP to hand out that static IP for DNS.
It's all done over a singular ethernet connection between my router and the server; the amount of traffic is very minimal.
I'm not sure if complete abstinence is good. I see my parents struggle with ads sometimes. They grew up as farmers in a rural enviroment with only a black and white TV (with 2 state channels). They seem unable to distinguish ads from proper content (unless it's obviously an ad), or they see ads as a kind of honest product documentation.
I think end users should have more fine grained control over what they share and see when getting served ads.
I feel some basic defaults are in order which can be sliders and checkboxes for more control, relinquish of control for more trusted sources.
I don't want to see ads about alcohol or gambling, or games that profit from it. I don't want to see ads that move if I am not hovering on them. I only want ads that are self hosted, not third party injected. (don't punt the malware liability to third party, serve it yourself and be more responsible)
things like this are important for security and privacy, and not an issue with ads, but types of ads. If certain publications run certain ads I may not purchase them, or lose faith in them, they can't punt the blame for yesterday's ads.
Some things are limited by laws for billboards and such, I don't agree with all those regualtions, but indeed this is something that is important - and with the web it's easy enough to fine grain limit certain things from appearing in front of children, or people struggling with divorce, or all kinds of things -
So we have the ability to make ads better, in some cases I would gladly share more info than google allows to be shared currently (I don't care if most web sites get the page and keywords I used to find them, it may help them help me.
I do care if third party beacons and trackers share that info with lots of other people I don't know. I should be able to control this on a site by site basis with sane defaults.
Personally I just want DDG to make a search API that is free and doesn't require API keys. That way I can setup search in mobile or desktop apps and have it automagically be searching through specific sites like Soundcloud and Youtube and such.
Like if they're reading this, DDG is very unique and if they were to focus upon reducing costs, making efficient software, and better algorithms, DDG could be the service we shove into apps.
It wouldn't be proof, but it would be a fairly strong commitment.
I didn't see the "!" documented anywhere but it seems to work at DDG. (We'll also add "\example" to do the same). https://duck.co/help/results/syntax
> We also of course have more traditional links in the search results, which we also source from a variety of partners, including Oath (formerly Yahoo) and Bing.
They do not have their own search index.
>In fact, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes).
Kinda silly to have a crawler that doesn't index anything.
In fact, DuckDuck Go gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from a variety of partners, including Oath (formerly Yahoo) and Bing.
We continue to partner with more and more sources to bring you the best Instant Answers from the best sources. Our focus is on synthesizing it all together into a superior search experience.
I suppose that's quite different than 'DDG is a frontend to Bing results'?
There is also searx: https://github.com/asciimoo/searx
It has a couple of instances, like: https://searx.me/
To clarify, we haven't closed anything that was previously open, and all open source Instant Answers will remain open.
For now, we've paused community contributions for new Instant Answers and non-essential changes, until we can find a better way to work with the community and their contributions.
For example, search DDG for "odroid". The official domain is hardkernel.com. The first result from hardkernel comes up at #6 for me. I'd love to have a way to suggest the official domain for a product like that so it shows up as #1.
I don't know if DDG is looking at their "!g" requests or not, but I'd be surprised if they weren't.
Specific instances vary.
I've a feeling that a lot of DDG users must be changing their search habits to suit DDG's engine, but I and plenty of others just can't.
Qwant did better, but still not close enough.
https://github.com/BurntSushi/ripgrep
https://github.com/ggreer/the_silver_searcherNot: -word
Site: site:example.com
Any of terms: (term1|term2|term three)
Recently DDG has been Googleifying by dropping one or more search words with an explanation "Not many results contain ____". So I have to click a link to search again for the words I entered. Quite annoying when it happens repeatedly each day.
Clicking this opens a form right on the results page which you can use to specify what the feedback is about (link results, image results, ads, etc.) and why you're happy/unhappy with what you're seeing.
This feedback is reviewed daily by our team.
Of course, you can't suggest the correct result for a specific search term, because search engines don't do a one-on-one matching of terms and results. But this will allow them to analyse at what types of queries their engine performs badly, and to improve it there.
(I don't know if they actually do this, but the theoretically can and should.)
Those who don't use launchers outnumber us greatly, of course. But by virtue of being able to specify an alternative search engine using the same efficient keyboard commands as everything else, bangs aren't much more than a pain in the butt.
Even putting launchers aside, not every DDG user is going to use bangs to get results from another site. I'm sure many, as I, have the muscle memory to bring up google.com in the flash of an eye, whereas awkwardly clicking in a search bar to type in a bang surely can't be used by even the majority of DDG users.
(Of course, I don't have data to back me up, but say DDG became as ubiquitous as Google, this would be unquestioned. All the users' habits in organisations where Qwant is set as the default could make an interesting data set — except, of course, it wouldn't be possible to track who used what)
tl;dr: Bangs are too inefficient and usage must be too inconsistent to use as the basis of improving the results.
Using DDG from the UK is currently a disappointing experience. You have the choice between US centric search result or "UK" filter that is too restrictive: for example "man cat" with the UK filter return mostly non unix stuff (unlike the default version)
!g is great, but when I use DDG on my iPhone, "!" is not an immediately accessible character.
1. They're hosted in the US. I wish the company would pick up and move to a country not within the 14 eyes and out of jurisdiction of the cyber super powers(US, China, and Russia). Maybe panama? At least move it virtually with a warrant canary in place.
2. The search results need better contrasting colors for glancing through. Google is noticeable better with it's bolder fonts and better typeface/colors. I feel this would be easy place to innovate in over google.
3. Better site specific searches. Right now you can search a specific site for results. For example right now you can search specific sites using `site:ycombinator.com rust programming langugage` and you will only get results from hackernews. It works great right now and I use daily in my workflows but it would be great to do this from the search bar for firefox or some better way.
Search engines have _alot_ of areas they can innovate in and I feel google with all its controversy(prism program, chinese state surveillance, etc) that many people would prefer to have their searches(and their life stories) in a company that values their users privacy to a fanatical extent.
Also lots of sites have !bangs available, like "!hackernews rust programming langugage" will go the the HN search directly.
Maybe that helps.
There are so many ways these search engines can still innovate.
I hadn't heard that term; in case anyone else hadn't, this page explains it pretty well:
They say they don’t Store IPs, but every request presumably has one. So it would be very easy for them to change this if they so wished, at which point their data collection wouldn’t be anonymous. As much as I like DuckDuckGo, I’d really like it if I could trust that they protect my privacy without having to trust them that they are doing so.
You could, however, use them via their Tor onion service: https://3g2upl4pq6kufc4m.onion/
It took me a while to find out somewhere where this is really run by them rather than a third-party proxy, but this seems to confirm it: https://www.reddit.com/r/duckduckgo/comments/1hdqs6/weve_upd...
Nope. This means that they tell us that they cannot tell what individual people are doing. But as they said right in the sentence before that, we are supposed to believe that they don't store IP addresses.
> And like with all other requests to DuckDuckGo, there is no personally identifiable information embedded in them, and we do not store IP addresses associated with them.
That is an outright lie even. HTTP Headers contain enough data already to make individuals distinguishable. Add some data science and you got some fancy profiles of your users.
I don't know what's going on with HN and DDG, but seriously is everyone just buying into this marketing bs? Why should I trust DDG with their claims about privacy instead of Google, Bing, or any other company?
I tried Startpage as a sort of in-between solution, but that site's so dang slow.
HN users use the downvote to show disagreement. Ideally it would only be used to bury comments that were off topic, spam, etc, but the only system I've seen with a correct implementation of this is https://lobste.rs.
Edit: This comment is a case in point since I've been downvoted less than 30 seconds after posting it. Ha. I challenge someone to prove me wrong.
But I agree, it's rare that auto-correcting helps me, more often than not it changes my search to something wrong.