DuckDuckGo now operates a Tor exit enclave | Hacker News Reader

DuckDuckGo now operates a Tor exit enclave

gabrielweinberg.com

174 pointsphsr90 comments

Congrats again, Gabriel! I don't use Tor myself (atm, at least), but I admire your efforts to help address this concern for people. DDG is impressing me more and more every day. It's been the default in my search bar for a couple weeks now, working great. Here's to the future of DDG!

That's great, too bad I can't run this for my own sites because a bunch of idiot network admins (like those running IRC servers) will ban all traffic from Tor exit nodes without taking into account exit policies.

You could dedicate an IP specifically for Tor?

Yes, but that requires paying extra for a dedicated IP just for that.

True. How come your irc client lives on the same IP address as your website(s) though?

Because I'm renting a single (Linode) VPS for development and hobby projects, not a bunch of machines.

It's not just IRC either, a bunch of other services do naïve checking of Tor exit nodes.

I know tor publishes lists of exit nodes, but does it need to publish ips of exit enclaves? It clearly does not publish lists of ips running hidden services, and an exit enclave seems closer to that than an exit node.

Well yeah, it needs to publish a list of those, how else would Tor clients know how to exit through them?

Same way clients are routed to hidden service nodes?

No, that's different. With a hidden service you're not exiting the Tor network, but when you're exiting you always know where you're exiting from.

That's why the Tor project itself publishes a list of Tor routers and exit nodes, if they didn't do it it would be trivial for someone else to compile and publish the list.

Relays (no-exit) are published and stupid ip blocklist admins happily add them to their "service".

About two years ago, spam and abuse were firmly among Justin.tv's biggest problems. It turned out the worst offenders were using tor for all network traffic. Unfortunately just configuring all of our servers to reject traffic from tor was by far the biggest bang-for-buck thing we did to kill the abuse.

If there's a better way, I'd love to know about it.

When Wikipedia had this problem they just blocked edit access from Tor exit nodes:

https://secure.wikimedia.org/wikipedia/meta/wiki/Editing_wit...

Seems like a more measured response than outright blocking of everything.

Well, firstly this isn't a big problem for HTTP based services like justin.tv, almost all Tor exit nodes allow HTTP exits, but it's a bigger issue for non-HTTP exits.

But the better way is to take into account exit policies. Only ban access from those IPs that are Tor exits and allow access through to justin.tv.

You can query the exit policies of a given exit with TorDNSEL: https://www.torproject.org/tordnsel/

I also whipped up a quick script that can be fed a list of Tor exit nodes on STDIN and will spew out a list of exits that aren't allowed to access a given IP:PORT: http://gist.github.com/523328

According to that 5/1338 Tor exits have policies that don't allow exits to justin.tv on port 80 (199.9.249.22:80), but e.g. 309/1338 ban exits to irc.freenode.org:6667.

That means that justin.tv is needlessly banning ~0.4% of Tor exit nodes, but someone using the same approach with an IRC server is needlessly banning ~23%. With SMTP (checked on a random GMail server) that percentage rises to 98.8%.

Excuse me if I'm being dense; obviously it is unnecessary, but what's the drawback of banning a Tor exit node that disallows access to your IP? It just seems far easier to ban them all.

Yes it's easier. It's also easy to just ban all of China. But the drawback is that you're needlessly banning people who aren't doing you harm, and discouraging others from running Tor exit nodes with limited exit policies.

E.g. I'd run a Tor exit node with policies that only allowed wikileaks.org and a few other selected sites, but I don't. Because I know it'll cause me more trouble than it's worth due to a bunch of overzealous administrators.

Who cares if you ban an IP that's already unable to access your site? There's no downside, and it sounds considerably simpler than querying exit nodes.

Because you affect the person running the exit node - not just the people exiting from their IP.

How? If the people couldn't make it to your IP anyway...

The assumption is that some (crazy) people are running tor exit nodes on IPs shared with other traffic. That is IMO a seriously bad practice; even putting stuff like that on the same /24 as critical servers is probably a bad idea.

The topic of this thread is that if you're running a Tor exit with a limited exit policy you'll get banned by lazy admins who presume that all Tor exit nodes have a promiscuous exit policy.

There's theoretically nothing crazy or bad about running a Tor exit node like that, e.g. one that only allows exits to a few sites that you yourself control.

But in practice it's bad for you because of people that think "let's just nuke it from orbit" is responsible behavior on the Internet, even going so far as to ban whole /24 networks.

Part of the goal of Tor is to have a bunch of normal people running Tor nodes on their own personal machines, VPSs, etc., in which case they're almost always shared with other traffic (the person's own traffic).

The downside is the person who's running the Tor exit node can't, say, upload videos to justin.tv.

Interesting stuff, thanks.

One correction though, justin.tv involves much more than HTTP. Our chat system is actually an irc network, and then of course we have the whole video system which is all non-HTTP traffic.

Right now we just block all tor traffic, for all ports. Sounds like perhaps we could do a bit better than that, and I'll let people know, but I can't say I think it will be a high priority for us unfortunately.

If you're blocking access to your site from specific IPs or ranges of IPs, then you fail at the internet.

If you have "IT People" who advocate such a policy, fire them on the spot, and then give them two hours to think about what they have said and plead for their job back.

I find it ironic that tor is routing around the internet damage from censorship, only to run into internet censorship from damaged people.

Do people actually use Tor, day to day?

Yes. Shitloads.

Shitloads of spammers.

https://www.torproject.org/faq-abuse.html.en#WhatAboutSpamme...

I think the state department does, or did at one point

can you quantify "shitloads" into an actual metric?

Looks to be about 15,000 regulars atm in Iran alone:

http://metrics.torproject.org/graphs.html#recurringusers

Plenty more graphs and stats on that page. Also read this for information about the type of people who use it:

https://www.torproject.org/torusers.html.en

FWIW Almost all other countries listed have no usage or very little.

All of the countries listed seem to have either thousands or hundreds of users. You think that's "no usage or very little" ? I guess that depends on your perspective.

Tor is never going to be used by tens of millions of people. It doesn't need to be used by tens of millions of people to serve its purpose and be successful.

I'm not sure why they only present that small subset of countries in their stats. I wish they'd provide more comprehensive stats.

I use it for my "distractive" browsing. I access reddit/slashdot and other alike sites only through it. On the one hand it slows down, making me more selective and aware of what I do (ie I do not "wake up" 10 wikipedia links later) and on the other I have total privacy.

Well, not total privacy. Your browser leaks uniquely-identifiable information, Javascript tracking bugs + cookies track you, and your hard drive is subject to seizure by customs or law enforcement.

Now, if you fake the UA with something very common, turn off most Javascript, and encrypt your hard drive in a deniable manner, then you are OK. (One of my side projects is to make the deniable encryption even easier for web browser history.)

I do nothing illegal so harddrive encryption or anything tin-foily like that is irrelevant. (I do it though as it does not hurt and is so easy, plus if a HDD fails I can discard it without having to bother.)

Cookies are per session. I use Firefox with Tor-Button.

In the worst case there is "someone" who does the things I do. But no-one does connect that to me. You could do complicate analysis and pinpoint my demographic through that but no-one does that.

I did for a while, but have gotten lazy. There is latency, though it is nothing like dialup was.

Why did I use Tor? Because a website whose FAQ said they don't log IPs (Slashdot) gave my IP to the Secret Service. The Service then came to my house with guns. Seriously.

(I showed them the comment, and they went away. The agents investigating did not actually read the comment before coming to my house to question me about it. WTF?)

Anyway, Tor is a necessity if you post content to the Internet. Although there is very little the legal system can do to you in the US for your protected speech, they can still fuck up and show up at your house to harass you. Use Tor, and they end up in Siberia instead.

Wow - what was the subject of the post they were investigating?

The parent post said, "what are some things you can't say on the Internet". I replied, "I am going to kill the president."

Guess I was right!

You may have just earned another visit.

This time I used Tor.

No kidding, using Tor with usernames that link to profiles is pointless

How much bandwidth will you allocate to tor?

I have it set as "BandwidthRate 1MB, BandwidthBurst 2MB" for now, though I'm going to monitor it and see what happens. Unfortunately, you can't prioritize exit enclave traffic atm.

Although this is great that duckduckgo is doing this it seems like it would only apply to a select group of people concerned about their privacy. It doesn't seem like the average person would be aware about what Tor is and would use it. I'm just wondering about the practicality of operating a Tor exit enclave.

this is really a great service. caution: I've read many horror stories about people running exit nodes, and being charged with things that probably they had nothing to do with , but are very serious charges. Thats why exit nodes take the most balls to run.

Only DDG traffic exits from the node. The rest is just encrypted traffic relaying.

Next submission: DuckDuckGo raided for child porn...

(unfortunately)

An 'exit node' would normally get you raided for child porn, subverting islam, leaking state secrets or whatever else, yes. Apparently an 'exit enclave' is different.

I really don't understand the strategy here. DDG Seems to be entirely targeted towards people who want to be ultra secretive about what they're searching for. People looking for kiddieporn? Terrorists? Who are the users here?

I'm not being negative here, I just have no idea what 'problem' is trying to be solved here.

It's not a strategy, just good policy.

Why?

What about when your service is used exclusively by law breakers? Will anonymity be a great policy then?

"What about when your restaurant is frequented entirely by murderers? Will steak knives be good utensils then?"

That's a very very poor analogy.

Advertising your service as an untraceable anonymous service, is attractive to certain groups of people. A large amount of those are going to be engaging in unlawful activities.

The analogy would be advertising your restaurant as one in which you are permitted to enter wearing full body disguises and helped with your exit through various escape routes. The restaurant owners will provide gloves for you, remove any traces of your DNA, etc.

Don't you think such a restaurant would then be used by murderers?

I would agree with your analogy in that an 'anonymous restaurant', where you are granted complete anonymity whilst inside, would also be a recipe for disaster.

You kinda forgot the 'anonymity' bit in your analogy though... Convenient ;)

There are plenty of good people out there who need anonymity. People whos lives depend on it. Just because you don't need anonymity doesn't mean you should deprive everyone else of it.

Talk about niche markets...

Not sure what your point is. How many users does a non profit service need before it's no longer niche and worth pursuing? Even if Tor only had 100 users who were safer for it, I'd say it was worth it.

There are many unjust laws, especially in countries other than the US.

Each of us has a right to privacy. This allows for nefarious activities but it also enables many forms of creativity.

Privacy = good

Anonymity = bad

Just my 2c.

how can you have privacy without anonymity from your information service providers?

If you want upvotes, please explain why you think this.

Who is "jrockway"? Why would being more anonymous or less anonymous make my comments more or less useful or interesting?

If anonymity is bad, why do you use a nickname here?

If anonymity isn't an issue, lets just get rid of the nicknames altogether and turn into 4chan.

Once you allow anonymity, you also allow impersonation, endless trolling, abuse, etc etc.

Anonymity is hardly necessary to hassle or troll (or even impersonate) people.

You're seriously suggesting HN users can't work out the difference between an 'o' and a '0'? I'd credit them with slightly more intelligence than that.

No, but they can't tell the difference in axоd. (The о there is the Cyrillic lookalike.)

It's a moot point. The nickname is different, and can be proven to be.

I think it's creepy that lots of random people can see what you're doing on the Internet, nefarious stuff or not. So I see enabling privacy wherever possible as a step in the right direction, regardless if it attracts users or not. In this case, I didn't think too hard about it. James proposed the idea a few days ago. I looked into it and set it up yesterday, and announced this morning.

As for the service being used exclusively by law breakers, I think that hypothetical is far-fetched. DDG has plenty of users now that don't care much about privacy, and I don't think most Tor users are law breakers. So I see this whole line of reasoning as a non-issue.

Well, good luck as always... Just remember what happened to Reddit.

If you are like me and use Google all the day (without giving much thought to the fact that you are logged in, e.g. via gmail), it can be pretty scary to take a look at https://www.google.com/history/

Total Google searches: 21717

None of these searches are in the categories you mention, but I bet you can still make up some interesting statistics about me.

... and why would making up some statistics about you be scary? They might be able to show you relevant advertising???? OH NOES!

Our research shows that all child molesters search for these 7 items. We randomly selected you for a quick search of your 20k search terms and found all 7 (separated by years but we decided that was irrelevant).

Since we have a "zero tolerance" policy when it comes to child molesters ("zero tolerance" meaning "guilty until proved innocent") we are accusing you of this crime (and telling your workplace and all of your acquaintances).

You may now use whatever meager life-savings you might have to defend yourself against these charges. This should not take long as you are also now friendless and out of work.

Have a nice day.

So why not just disable web history?

The users probably include democracy activists struggling against dictatorships. You know, people of higher moral caliber than your average child molester.

Exactly. See https://www.torproject.org/torusers.html.en for a list of the type of people who use Tor:

That is a list of people the tor project would like to use it. In my limited experience, that does not mesh in any way with those that actually do use it.

You don't even know what 1% of the people who use Tor, use it for.

You're right. I only know what the other 99% use it for.

You know, you're acting like a dick all over this thread.

In other words, people conducting illegal activity.

The problem that privacy is not a standard online.

and the solution is, 99.999% of people don't care.

Your viewpoint seems to be that those who do care must be have a malicious intent. I don't and I know of many others who don't. So you are ranting into nirvana, really.

My original point was that I don't understand the stratgey of pandering to a very small niche market, that is notoriously impossible to monetize (see Reddit).

Anyway, seems my views are falling on deaf ears.

People sometimes do things for reasons other than making money.

Except DDG has features other than a good privacy policy, so he's not just marketing to what you suppose to be a niche market - he's just adding them to his target market.

Really? Why are people so upset with Facebook oversharing, then?

Maybe people really do care about their privacy. Everyone has that stalker ex that they want to avoid.

It seems that many people who are "outraged" in reality stay on Facebook anyway because they really don't want to quit. Witness Facebook's ever-climbing popularity and the failure of all the "Quit Facebook Day" protests etc. to be a success. This reminds me of the threats many people made to move from the USA to Canada if Bush Jr. were re-elected...if you know people who said that, I'm guessing they stayed put (the people I know certainly did).

Just because something is illegal doesn't mean you want everyone to know about it. Do you want your ISP to read your email with your girlfriend? Do you want Google to know about your medical conditions?

Information is power; the less you give to other people, the more power you have over your own life. Why let your ISP and search engine learn more about you than they need to know?

DDG is targeting itself to tech-savvy people that don't feel like leaking their entire life to the Internet. The Tor relay helps this marketing goal.

>> "DDG is targeting itself to tech-savvy people that don't feel like leaking their entire life to the Internet."

The parallels with Reddit are certainly there I hope you can appreciate. I'm not saying it's completely the wrong strategy, but it's certainly not the one I'd choose.

Reply on news.ycombinator.com