Google disables SSL search at BT’s request | Hacker News Reader

Google disables SSL search at BT’s request

blog.al4.co.nz

307 pointsampersandy111 comments

At the moment, yes, no nosslsearch VIP will do this. However we're getting rid of it soon and replacing it with one that enables SafeSearch, but still over HTTPS: https://support.google.com/websearch/answer/186669?hl=en

However, if you want an encrypted search option, https://encrypted.google.com/ is always encrypted and isn't affected by these methods.

Some more information: http://googleonlinesecurity.blogspot.com/2014/10/an-update-t...

Ha, you're the guy behind Pond (hi!). As a security researcher, how does it feel to work for a company that (reportedly) (pro)actively collaborates with the NSA? Are you ever worried that the company might not be as ethical as it seems to the average Googler?

puts on tinfoil hat

edit: Thank you for the downvote[s]!

edit2: I just remembered a relevant example. Reading "How Google works", I clicked in many ways with their vision about smart creatives and how to run a company properly. However I then immediately realised that it's written by the same a-hole involved in the massive Google-Apple wage fixing scandal [0], and it made me question how much of what's in there is real.

0: http://www.cnet.com/news/judge-rejects-324-5m-wage-fixing-se...

You're welcome for the upvote.

I would not go that far, but it is funny how this was written after I suggested in HN comments that they do an apology letter to settle the scandal.

agl is one of a handful of people who are moving the needle on HTTPS usage, pretty much through sheer force of will. So of all the people you could pick on, you made a really lousy choice.

My questions were genuine, I am honestly interested in his or HN's opinion.

> if you want an encrypted search option, https://encrypted.google.com/ is always encrypted and isn't affected by these methods

As the article concluded, "DuckDuckGo it is" :-)

Append !g to your query and https://encrypted.google.com is where it'll send you.

You'll get a "feel" for which queries are more suitably answered by Google than DDG quickly enough--which is not all that often as you expect, not because DDG's own results are so incredible, but because as you get the hang of the other !bang operators, you'll find you search directly the very sites that you wished/expected your top Google hits to be in the first place (!w Wikipedia, !so Stackoverflow, !r Reddit, !snopes, !gi !yi !bi image search engines, !map Google maps, !yt Youtube, !wnl !wde !wxx Wikipedia country-code xx, to just name a few I use all the time).

This might just be a use of the magic "nosslsearch" DNS record: https://productforums.google.com/forum/#!topic/websearch/1l2....

It seems like that technique no longer works.

The end of the thread (from earlier this year) seems to indicate otherwise.

I tried it myself before making the comment and it did not work, but perhaps that was because of HTTPS Everywhere.

A Google employee addresses this: https://productforums.google.com/d/msg/websearch/1l2KMUfgyo4.... Apparently visiting `nosslsearch.google.com` directly does not have the intended effect, but setting up the CNAME record does.

That page describes exactly what is happening:

The network administrator can adjust the DNS configuration for www.google.com to point to our NoSSLSearch end point. For regular http traffic, the user will see no difference.

We will not serve SSL search results for requests that we receive on this VIP. If we receive a search request over port 443, the certificate handshake will complete successfully, but we will then redirect the user to a non-SSL search experience. The first time a user is redirected, they will be shown a notice that SSL has been disabled by the network administrator.

Google provides an option for network administrators to disable SSL searching on their own networks. It doesn't involve paying Google. You can see in this thread that filter manufacturers and their clients (schools, etc.) were the motivation for this feature and they'd simply block services that didn't allow filtering if the feature didn't exist.

Wow, that is very disturbing. Talk about intentionally shooting yourself in the foot, security/privacy wise.

"how to steal from my employer"

I really doubt that anyone in the western world would even attempt blocking Google - the blowback would be immediate and huge if regular people were banned from using Google on their computers.

I saw this behavior on a public wifi in london once. I think it also blocked "encrypted.google.com".

Apparently google has an option for network administrators to force a redirect to "nosslsearch.google.com". Oddly enough, the "learn more" page has removed the reference to this domain, but it's in the wayback machine: http://web.archive.org/web/20140827203531/https://support.go...

The Wayback page doesn't reference "forcesafesearch.google.com" as the current one does, so perhaps "nosslsearch.google.com" is deprecated as a means of blocking adult content.

Hm, reading the comments I get that it's always the same story: If you're not paying, you're the product.

BT is trying to make some cash selling data to third parties. Since google allows to specific network blocks (defined probably by IP address) to use non-SSL connections, BT installed public WiFi to offer internet access and gather data which could be sold to advertisers. Is it really a goldmine? I'm not sure, with Google and Facebok gathering much more personal data than BT ever will, I'm not sure if it's a Goldmine, depends on the quantity and accuracy I guess...

You already have to pay BT to use these hotspots.

I'm a BT customer and I immediately checked this out.

Using the latest Chrome/Firefox, searching for anything in the address bar is sent over https. Perhaps the author is being 'watched' as he is a surfing via BT's wifi pass?

Yes, this certainly only applies to BT public hotspots, not normal home internet.

Think this will just be on BT Openworld, which they resell from anyones BT router.

This is set up at the carrier's level whereby they disable HTTP for Google search. To re-enable it, you can define your search to: https://encrypted.google.com/#q=search

Are you saying adding "encrypted" to the URL fixes the problem? Because, the author does attempt to access google directly over HTTPS. And it is not set up at the carrier level; Google performed the redirection from HTTPS to HTTP.

No, it's set up at Google, whereby they disable the HTTP for Google search when asked to nicely (£££).

It costs nothing to make your DNS server point users to nosslsearch.google.com. I think you are letting your feelings toward Google cloud your perception of reality.

I work for a school district in the US and we have to have a gateway content filter to prevent students from accessing inappropriate web sites. We use an iboss content filter which can decrypt ssl and rencrypt on the fly. It can also force google safe search and such. I suspect this company uses something similar.

http://www.iboss.com/web_security_suite/wss_content_manageme...

>We use an iboss content filter which can decrypt ssl and rencrypt on the fly.

Is it safe to assume this is some sort of trusted MITM proxy?

I think this is really taking it too far. We use Lightspeed and they block SSL traffic during the handshake based on the domain its destined for. No need to decrypt anything.

It runs as a transparent proxy. We used to use Lightspeed but found it lacking in reporting. It can act as a MITM proxy if you turn the option on. We do however force safe search on search sites that the box works with.

THX for your efforts in bringing up the next generation of hackers and penetration testers!

(Perhaps you were being sarcastic) but these kids most probably won't be able to access a great many of the informative and educational websites that allow them to research and learn about how to properly secure computers and their online experience. Because (I've seen this) they are most likely dumped under the category of "hacking websites" ... even perfectly benign network tools.

I just noticed this for the first time in the AA lounge at Heathrow today. They use BT for their wifi and I got a notification that encrypted search had been disabled by my ISP. So it seems like it's all BT internet products that are doing it.

> "I got a notification that encrypted search had been disabled by my ISP"

How'd you set that up?

It popped up on the Google search results page as in the linked post

For anyone else wondering what the hell "BT" is, it's British Telecom, an ISP.

it hasn't been British Telecom since 1991.

these days they're into just about everything: landlines, tv, internet, mobiles, you name it.

they tend to have a notorious reputation for providing a bad service, with their internet service being the prime example... god help you if something goes wrong with your phoneline.

Would BT be the UK's version of Comcast?

More or less, except Britain is currently experiencing the aftershock of David "Think of the Children" Cameron and his band of privacy-hating merrymen. While Verizon was caught out recently tracking users and Comcast maybe does the same, at least they don't forcibly restrict you from using HTTPS.

I'd say a better analogy would be AT&T since it's the former monopoly

Yes although our government has forced them to open up the last mile to competition. So unlike with Comcast/AT&T, at least here you can switch to a more palatable ISP fairly easily & quickly. You're still somewhat at the mercy of BT's engineers though if you have a fault on your line.

Aren't all of those about telecommunications? The name seems apt.

They officially changed name in 1991 to be called just "BT".

I'm not sure they do have a reputation for providing a bad service.

BT Infinity blows the competition out of the water, has no bandwidth limits at the £26/mo option and is very reliable.

BT is the worst. Their Internet Hub or whatever they call their router does not allow changing DNS settings, because that would circumvent their crappy filtering. Would someone please stop thinking of the children!

In the UK there are lots of other ISPs to choose from though. This probably cuts down on more users being told by fraudsters to change their DNS settings than it makes complaints from users wishing to change them?

I hit this back in August when I moved and had to wait 2 weeks for my new line to be installed correctly[0].

I didn't care enough to find out why, I always use a VPN when using WiFi and to be fair to BT they recommend the use of a VPN when using the BT WiFi service.

[0] It took BT 2 weeks to install the line correctly after cocking it up twice! Third times a charm. Great going BT! /s

Why did BT recommend using a VPN?

They would recommend it because public wifi is unencrypted and open to sniffing.

Looking at that transcript, the redirection from https://www.google.co.uk/... to http://www.google.co.uk/... does appear to be served by Google. It's over HTTPS so it's signed by a key only Google has.

Or by a CA that is trusted by your browser....

That would be a huge scandal. When fraudulently issued certs are discovered that's news, and if you did get one issued you wouldn't use it to redirect random BT customers to http.

Unless you're in Chrome or Firefox in which Google's certificates are pinned

Grab a DO instance for $5/month, install openvpn on it, set it to serve over tcp & port 443 if you have to and then shove your DNS and everything else through that. Yes it's sad this is necessary but it's easy to do, costs virtually nothing and lets you sidestep most ISP filtering policies with the added bonus of protecting your traffic from whatever random wifi network you happen to be using to access the internet that day. If that's too much like hard work, there are also hundreds of 'VPN as a service' providers out there that will do it all for you for less than $7/month.

Either way, it seems more and more quaint to me that anyone connecting via a mainstream ISP assumes they'll get an unadulterated feed to the internet. If you're an adult and want to decide for yourself how you'll use the internet, get a VPN in place and relegate your ISP to being a dumb bit pipe.

The point is that Joe User won't be able, shouldn't have to do this.

Definitely true neither they nor we should have to do it.

Fortunately, there are still some good ISPs left in the UK. I'm with Andrews & Arnold and they're staunch supporters of an uncensored net (http://www.aa.net.uk/kb-broadband-realinternet.html). They're also just generally awesome - dual homed static IPv4 and IPv6 addressing, a geek answers the phone if you ever have reason to call, you can choose your backhaul transit provider and lots of other nice things. You can even opt for billing that follows the lunar cycle :)

Regardless of ISP though, I think even Joe User should figure out how to install & use a managed VPN service for when they're out and about using random wifi networks (e.g. from privateinternetaccess.com or similar). Of course, that assumes these VPN services are trustworthy which I'm sure a lot aren't...

Then automate it and charge $6 per month. :)

I was recently in the same situation as the author of this post. And as far as I can figure the reason HTTPS is disabled, is that the BT Wifi hotspots require you to login with username/password on a custom page before you can access the internet. Most people's default thing to do is google something, which then redirects them to the BT Wifi login page, but this only works if Google is being served up via HTTP, otherwise BT wouldn't be able to hijack the request and redirect you to the login page.

Hence it's probably not got much to do with privacy, and more to do with usability.

If +90% of users just got HTTPS/SSL security warnings from their browsers instead of a BT Wifi login page, they wouldn't be able to use BT Wifi unless they're of the minority who know and understand how HTTP/HTTPS connections work.

It's worth noting however that both recent Windows and Mac OSes at least detect captive portals automatically and show the login page themselves, making elaborate and insecure hacks like that unnecessary.

That is true, but everyone aren't running even recent-ish OSs. Also having relied on BT Wifi for about 2 weeks recently, I can definitely confirm that OSX's detection doesn't always work. About 80% of time, it's fine, the other 20% it's google for "asdf" and get redirected.

Also, BT Wifi tends to log you out every 20 minutes to 6 hours seemingly by random, forcing you login with your credentials again, and this need to re-login is something that OSX never detected.

Isn't that a security hazard? The mechanism of these captive portals is literally a MITM attack, and I don't see how to distinguish a benevolent from a malevolent use of it.

There is an official http status code, but obviously no one uses it yet.

> otherwise BT wouldn't be able to hijack the request and redirect you to the login page.

They do capture and redirect SSL traffic on first connection, resulting in a security warning on Firefox. So it's not a technical limitation.

Prior to login, all DNS requests for the new MAC are spoofed to direct to the login service regardless of protocol.

Note: BT = British Telecom, a UK internet provider

That adds sooo much context that I was missing. I was like Bit torrent is asking Google to do what?

This anti-feature is in place to support censorship by schools which wish to prevent students from Googling certain words. It's not surprising that it gets used for more nefarious things.

"this network has turned off SSL search"

Honestly, I assumed this meant that Google wasn't allowed to do it. And since they couldn't secure you, they wouldn't give you your account.

My school (when I still went to it) did this, presumably to allow filtering of search terms. It stopped you using Google over HTTPS to avoid filtering. The solution was simple: DuckDuckGo.

What Google seems to have done voluntarily will be forced on it tomorrow by repressive regimes all around the world.

Google seems to have failed us once more.

Repressive regimes like the UK and the US, et cetera.

I don't think it's fair to blame Google if they are complying with the law of the land and the wishes of society as expressed through the democratic process.

I think this is as bad as trying to comply with backseat driver directions, verbatim.

They are supposed to know how not to drive their product into the tree. Society doesn't. Their search is their product, and not "society's".

Their product does belong to "society" to the extent that it is bound by the laws of that society. And if we pass laws that require them to spy on their users for the government, we can hardly complain when they follow the law.

They still have the choice of telling us "no". That's because our laws prohibit forced labor. They even said "no" before, in China, for example.

But now it will be increasingly hard for them given what they sell for money.

I don't think that's reasonable to expect. You're asking an entire company of thousands of people - people who also participate in society and vote in elections - to teach society a lesson or something by refusing to do work you disagree with (and, for the record, I disagree with too). Probably a lot of people at Google are okay with spying - many of them helped elect Diane Feinstein, after all.

I have a good idea where this is all headed, and probably within my lifetime people like you and me are going to be able to deal out some pretty damn bitter "I told you so's". In the meantime I will try to keep that from happening by trying to educate people, for as long as doing so doesn't get me killed. But expecting a group of people, many of whom don't agree with me anyway, to practice some mild civil disobedience on my behalf, would accomplish very little other than to drive me mad.

The problem is not that they are pro- or against spying.

The problem is they undermine how secure internet and HTTPS works and how people perceive it.

They're heading us for the world where nobody will even be in control WRT how much info is collected and what it is about.

Yep my high school (in the US) has been doing this for years. They even blocked Duckduckgo this year. Google really needs to ditch this "feature".

Edit: Just saw Agl's response, I'm glad Google is changing this.

Dear Google: remember that "don't be evil" thing? This is evil.

They've "outgrown" the "don't be evil" mantra now, apparently... http://www.theguardian.com/technology/2014/nov/03/larry-page...

That's not what the article says, it says they've outgrown their mission statement, which was to “organise the world’s information and make it universally accessible and useful”. Which makes sense, since they've expanded beyond information services.

BT blackmailed Google into doing this. The alternative is no Google on BT at all.

I gotta be honest: I'd like to see what would happen if they tried this.

I assume you've got some evidence for your a claim that BT have broken the law like that?

Especially when changing the CNAME record is something that absolutely any DNS provider could be doing anyway...

Censoring wouldn't break the law, they're already doing that. And the blog post specifically mentions that it wasn't blocked using DNS but by a redirect from a Google server.

Blackmail might well be a crime..?

Oh come on. This is a specific BT service that allows your home router to be used by the public (for a payment to BT, presumably the home owner gets a cheaper service or some return on the deal?). Should BT really enable the searching of non-SafeSearch material via such connections? Should Google really prevent BT from implementing this system?

What is it about extreme internet content that you think is so important that BT should support it being downloaded via their customers home routers without those customers knowledge? Or is it that schools use such a system to block extreme content - presumably you think that the dreggs of the internet are appropriate for schools to allow students to access easily?

Google's not stopping you searching for whatever extreme content you like they're just limiting their enablement of such searches in circumstances where those in control of the internet connection choose for it to be limited.

1. It's in a separate channel from the home owner's data. They shouldn't care what's in it.

2. They're invading the privacy of everyone not downloading 'extreme content'.

3. Privacy invasion is not appropriate in anything that pretends to be generic internet access, extreme or not.

You get access to the wifi network on other routers. You don't get anything else in return - this is included in the cost of the bill. You can phone them and get them to disable it on your router.

Their routers seem to have no QOS - one computer doing an update will kill internet for others in the house, presumable this is the same if other users are on the 'BT Openworld' wifi it shares.

Ew. What happens if you use a normal https proxy server based e.g. in Germany?

I might get downvoted for this but I have two meta-questions:

I assume BT is a European (or British) ISP?

Is "seppuku" a common analogy people use? I just looked it up and was a bit surprised at the result.

British, and yes. The latter is also sometimes jokingly referred to as "sudoku" in communities like Twitch.

1. Yes.

2. "Suicide" is a common metaphor, and "seppuku" (a Japanese form of ritual suicide) is sometimes substituted as a more colorful synonym for this usage.

We changed the title because the original is baity. If anyone can suggest a better (i.e. more accurate and neutral) title, we'll change it again.

Sounds like you should run a VPN

We are Google's product, not Google's customer. Keep this in mind, and use Google sparingly.

For the privacy conscious, I don't really see a need to ever use Google (except perhaps for images). DuckDuckGo and Startpage are perfectly fine for your average search query; they aren't terrific for fresh news stories, in which case you can proceed directly to a news source or an aggregator.

This is probably too much trouble for your average user, but for me it is worth it. Once you gain a knack for it, it really doesn't expend that much time, either.

>I don't really see a need to ever use Google //

When you use DDG and Startpage you are using Google aren't you?

DDG uses Bing and some other search engines, but not Google (except when you explicitly redirect your query with !g).

I just discovered that DuckDuckGo has some stuff in place to make it easy to add DuckDuckGo as your primary search engine in Chrome. Took me about three seconds.

I am going to give it a try for a solid week at home and see if I can live with its results. I have no idea if they're as good, but I hope so.

DuckDuckGo's main problem is that they don't pick up new results as quickly as Google does. Otherwise, a query on DuckDuckGo looks very much like a query on Google (the top results are often identical.)

If you can't find something on DuckDuckGo, try using Startpage. Startpage uses Google as a backend, so it often has fresher results than DuckDuckGo.

I think the problem is even worse on Startpage, which does use Google search, but Google is probably intentionally limiting the API to very old data.

DDG's main issue is that their results are just not good enough. I really, really wanted to like them and use them, but in all but the simplest queries, I ended up back at Google.

I really want to ditch Google, but I don't see how. Except via Start Page.

That was my experience with DDG several years ago, but not recently. Sometimes I check results in both search engines; they're always very similar.

A tip about migrating to duckduckgo, if you break down with the results and need to go back to google, just add !g to the query and it'll route you forward to google with a redirect. Just be aware, even if you are using DuckDuckGo via the Chrome Omni bar for searching, those results still end up in your Search History on Google (see https://history.google.com/history/ )

As a result I'm on DDG + Firefox at this point

You know you can turn of Web History if you don't like it.

I've done that, but the fact that it was happening is non-obvious. Additionally, with it off, Google Now constantly pushes for it to be on.

Do they manage to collect all that if I never sign-in to Chrome and I never sign in to anything google-related unless it's in a privacy tab? I can't imagine how they would be able to do so given that I'm never signed in, but it wouldn't surprise me if they did somehow.

Sure they can track the searches done by your IP + other information from your browser.

I am almost never logged in to Amazon, but I've been noticing for awhile that they still recommend things to me, and they say up front "based on something or other to do with your Amazon activity." The only thing they don't do is call me by name when I'm not logged in. But they obviously know it's me.

You can also do !s to redirect to https://startpage.com. I believe they have the exact same search results as Google. (Without being Google)

How much do you pay for your search service?

Reply on news.ycombinator.com