Google shuttering domain fronting, Signal moving to souqcdn.com | Hacker News Reader

Google shuttering domain fronting, Signal moving to souqcdn.com

github.com

149 pointshapnin44 comments

In case it obvious to others: "Domain fronting is a technique that circumvents Internet censorship by hiding the true endpoint of a connection. Working in the application layer, domain fronting allows a user to connect to a blocked service over HTTPS, while appearing to communicate with an entirely different site. [...] This can be done if the blocked and the innocuous sites are both hosted by the same large provider, such as Google App Engine.' [1]

Amazon also recently acquired Souq so wondering if the 'same large provider' in this case is AWS?

[1] https://en.wikipedia.org/wiki/Domain_fronting

Do you mean, it is the same as a web proxy, that delivers content from another, sometimes hidden, location?

Yup, but proxying through a big company's popular domain/IPs (e.g. google.com) that would make a governments think twice before blocking all of Google just to stop your app.

What's really important about it is that it's using an edge cache ("edge caches" are a type of CDN, though there's also "push CDNs", which domain fronting doesn't do anything for).

A lot of companies, including Google and Amazon, run a bunch of reverse HTTP proxies (called "edge caches") all over the world as a way to reduce latency, since if the cache already contains a file, it doesn't have to go all the way to the backend server to get it. They don't just spread the backend servers themselves all over the world because they're more expensive, they're optimized for particular applications, and because database consistency gets harder and harder the more you spread out the replicas.

Since the edge caches are application-agnostic, the same one can also be reused for multiple apps. Your browser, then, can talk to the same edge cache all the time even when it's actually interacting with several different services. This is where domain fronting comes in.

As a quirk in the application protocol, your browser ends up sending out the domain name it wants to interact with three times:

1. During domain resolution, where it uses the DNS protocol to convert the relatively human-readable names into IP addresses. In a domain fronting session, this step has to use the "front domain".

2. During TLS negotiation, where it requests a signed certificate corresponding to a domain. In a domain fronting session, this step also has to use the "front domain".

3. During the HTTP request. HTTP doesn't just rely on the DNS one, because the same IP might have multiple domains, and it doesn't just rely on TLS, because HTTP is supposed to be able to work over plaintext. In most edge cache implementations, the HTTP domain is the only one that's actually used to decide which backend server to talk to, so a domain fronting implementation like Signal will use the "target domain" here, and since this is part of the HTTP session, MITM-based blocking systems can't see it.

Yeah, they're almost certainly using CloudFront.

    notriddle@DESKTOP-IIQA1VP:~$ host souqcdn.com
    souqcdn.com has address 46.137.109.63
    notriddle@DESKTOP-IIQA1VP:~$ host 46.137.109.63
    63.109.137.46.in-addr.arpa domain name pointer ec2-46-137-109-63.eu-west-1.compute.amazonaws.com.

46.137.0.0/17 which that address falls under is not part of the allocations for CloudFront infrastructure, but for EC2. I suspect their "CDN" is just EC2 instance(s), probably fronted by a load balancer.

no, cloudfront servers have rDNS records like

    server-52-85-146-234.iad12.r.cloudfront.net

Hey, everyone. We spent a decent amount of time at Signal trying to come up with alternatives when we first heard rumors that Google was disabling domain fronting on GAE.

We're using Souq because it is popular in the countries where we have Censorship Circumvention enabled (Egypt, Oman, Qatar, and UAE) but it would be nice to have other options on CloudFront as well. It's possible that we overlooked other highly ranked domains in these countries that use the CloudFront CDN.

If anyone has any suggestions, we would appreciate them.

Would adding federation to Signal help with users behind country-wide blocks? Seems like a distributed service would be harder to censor than a centralized one.

It's trivial to block several distributed hosts simultaneously. An aspiring censor would simply find the most common federated endpoints for a given service and block all of them. Only the users of that software would be affected. There wouldn't be any collateral damage.

If the censors somehow didn't hit every single worthwhile federated endpoint, users would still be left wondering why they couldn't communicate with most of their friends. Moving between federated hosts would also necessitate an entirely new identifier, so users would need to rebuild their social graph again.

In addition to being ineffective against censorship, there are several other properties and trade-offs that make federation a difficult proposition for an application like Signal: https://signal.org/blog/the-ecosystem-is-moving/

> users would still be left wondering why they couldn't communicate with most of their friends

That's not how federation works, at least in XMPP. You only need to connect to one server that's out of censorships' reach to be able to communicate with everyone.

Let's say I have an account on a federated server and a censor then blocks my ability to access that server from my home country.

While it's true that my friends on other servers might be able to send messages that will arrive on my chosen server, that distinction isn't very meaningful because I am unable to connect and retrieve those messages.

I wouldn't be communicating with my friends until I switched to a new server and rebuilt my social graph.

Rebuilding your social graph is easy - just import the roster and resend authorization requests as needed. The only inconvenience is a changed handle, so you have to point any potential new contacts to new JID.

Also, while it's not specified in XMPP (yet), it's easy to imagine a federated service that lets you connect to any server in the network that then behaves as a proxy to whatever server you have your account on.

Yes but it's not sexy. Quoting Moxie Marlinspike : "…it’s entirely possible to build a federated Signal Protocol-based messenger, but I no longer believe that it is possible to build a competitive federated messenger at all" https://signal.org/blog/the-ecosystem-is-moving/

Note that domain fronting is not only usefull to circumvent Internet censorship, it's also used by malware.

With domain fronting, you can exfiltrate data from a company by making the connection appear to go to a legitimate google service (ex: drive.google.com), whereas it actually is going to a server hosted on google cloud services and controlled by an attacker.

I’m pretty sure you can achieve pretty much the same thing by just uploading encrypted exfil’ed data to actual legit Google Drive using OAUTH to programmatically access a throwaway account (to avoid possible CAPTCHA requirements for non-programmatic access)

Gotta take the bad with the good. Some governments act like companies and treat tools like Signal as malware. Another discussion can be had for citizen freedom vs employee freedom, but public network restrictions benefit all kinds of censorship. It's also another discussion worth having on whether one's need to monitor all data in/out of their company is worth giving that power to other who use it for wrong (mitm devices and TLS termination w/ custom device certs notwithstanding).

Google or another more privacy-supporting company could block domain fronting for everyone _except_ Signal, Tor, and similar projects, with some sort of application process. Blocking everyone seems heavy handed but fronting itself is ultimately a sneaky way around censorship rather than an intended feature.

So the decision on what apps can be domain fronted because they need to get around censorship lies with Google or another big company, what could go wrong here?

I mean the entire trick to domain fronting is that some large company, whose site no country would dare censor, offers up their infrastructure as a front.

Who else do you think should decide who gets to host content through Google's servers?

> whose site no country would dare censor

Google is not accessible to about 1.4 billion people because the single government of China "dares" to censor Google. That's close to 20% of the world's population.

I don't think companies nor governments should get to decide this at all. Information wants to/should be free.

I cannot find a first-party source of why and when Google is shutting down domain fronting, but it makes sense from a cybersecurity perspective. Domain fronting is widely used by malware [1, 2] to evade network-based detection.

[1] https://www.fireeye.com/blog/threat-research/2017/03/apt29_d... [2] https://www.cyberark.com/threat-research-blog/red-team-insig...

Its also used to evade censorship...Lots of other ways to combat malware.

My bet is not because of malware but because of govt pressures.

Honestly, seems like a crummy move on Google's part. They're obviously not obligated to keep this running, but it's one of the best ways to evade censorship that doesn't get blocked. It's really a shame.

It's all fun and freedom fighting games until countries start blocking legitimate IPs and disrupting the interwebs: https://gizmodo.com/russia-blocks-millions-of-amazon-and-goo...

That’s the whole point. https://en.wikipedia.org/wiki/Collateral_freedom

That's the point, to keep governments from picking their preferred version of the internet. If this is the reason for Google disallowing the google.com front door to AppEngine or whatever, then Google is hypocritical because they support DNS-over-HTTPS for similar reasons. Guess if I started using a bunch of low-TTL TXT records to chat with they would stop that service too, heh.

That's because all major Western internet companies are just virtue signalers for the wine drinking, cheese munching western population that loves to pontificate on the topics of "freedom". Every single time a country with a reasonably large market told Google/Apple/Facebook/etc "Jump" these companies asked "How high would you like us to jump?" rather than respond with force -

Google can respond to Russia with a single "OK, lets see" and block all Russian IP addresses from accessing/using any and all Google resources. Such action would create the kind of pressure on the Russian government it would within days change its decision -- it is not that the top of the Russian society does not rely on services provided by Google in a day to day life. It is that top that matters. Leaning on them creates leverage.

Compromising one's principals allows competitors to exploit one's weak principals.

Can someone share a link to Google talking about shutting this down? I assume they are going to stop allowing appspot (i.e. AppEngine) host headers for google.com requests?

Afaik it hasn't been officially announced but they are shutting down support. :/

>Reached by The Verge, Google said the changes were the result of a long-planned network update. “Domain fronting has never been a supported feature at Google,” a company representative said, “but until recently it worked because of a quirk of our software stack. We’re constantly evolving our network, and as part of a planned software update, domain fronting no longer works. We don’t have any plans to offer it as a feature.”

https://www.theverge.com/2018/4/18/17253784/google-domain-fr...

For those wondering what "domain fronting" is,

https://en.wikipedia.org/wiki/Domain_fronting circumvents Internet censorship by hiding the true endpoint of a connection

How they do this is basically to send google.com in the TLS SNI then when they get to the HTTP level send

Host: signal.org

This will only work with compatible servers. In Signal's case they use AppEngine which is behind Google's network and the servers for google.com are able to connect to their app servers when given the appropriate Host header.

What are we looking at here?

Is this related to Russia blocking a ton of google IPs to enforce the telegram block, or is that a separate thing entirely?

> Is this related to Russia blocking a ton of google IPs to enforce the telegram block, or is that a separate thing entirely?

Completely separate.

Basics of domain fronting & how it relates to Signal: https://www.wired.com/2016/12/encryption-app-signal-fights-c...

In-depth explanation of Domain Fronting: https://www.bamsoftware.com/papers/fronting/

This will possibly affect the Tor project's Meek pluggable transport as well which is used for many bridges.

https://www.torproject.org/docs/pluggable-transports.html.en

> This will possibly affect the Tor project's Meek pluggable transport as well which is used for many bridges.

Not the case since meek-google was discontinued long ago, meek-amazon and meek-azure don't rely on Google as a front. It does affect however Moat and Snowflake, all of them are still only available in the alpha releases.

I was reading up on this topic a few days ago and found this to be an interesting introduction:

https://medium.com/@pmvk/domain-fronting-a-technique-used-to...

Is there any sort of product page listing for their CDN solution? I've heard of Souq as an online retailer, but didn't know you could have them act as a CDN.

Note: souq.com is based out of Dubai, as is Telegram.

That makes this an unusual choice, to say the least, given Dubai's record on human rights and free expression.

Possibly for tax and corporate reasons (wild guess) but Dubai has a special trade zone for Internet companies: https://en.wikipedia.org/wiki/Dubai_Internet_City

Reply on news.ycombinator.com