Identifying Airtel middleboxes that censor HTTPS traffic
iamkush.me
iamkush.me
It is regretful that organizations like NANOG, RIPE, ARIN, APNIC and others do not take a stronger stance (on a global geopolitical level) against censorship of the Internet, and attempts to create walled gardens and national firewalls.
The people who possess the equivalent of 'enable'/'configure' on the core routers of gigantic ASNs have real power to refuse to further harm the Internet.
We have seen a number of discussions on HN about ethics in software engineering. A quick search for posts with 'ethics' in the subject line turns up a number of things. In my opinion, ethics in the ISP, telecom and network engineering industry is equally important.
So now your company has got a content blocker installed. What exactly are your network engineers meant to do now? Demand personal refusal over any additions to the sites that these boxes will block? That seems highly unlikely to happen, and how could that even work in practice? Are all the engineers meant to vote on blocks, and only those unopposed sites get added to the list?
Can ethical network engineers usefully oppose content blocking?
a futile game of whack-a-mole that only serves to make politicians feel good, and so they can claim they're "doing something" about social threats.
malware domains can be adequately addressed at the application level through things such as: https://www.google.com/search?channel=fs&client=ubuntu&q=goo...
Authoritarian regime that forces all ISPs in a country to run networks funnelling all traffic through a government run central point where they do DPI and flow analysis on it (Chinese GFW for instance)? More of a real threat.
For instance there is one ASN in Iran that has transit connections to the outside world. All ISPs are forced to be downstream of it. https://bgp.he.net/AS12880
Agreed, the UK's DNS filtering is definitely a simple to defeat by anybody whack-a-mole (e.g. thepiratebay.org is blocked? Oh no! Let's just google for Pirate Bay and pick one of the many, many unblocked mirrors)
But the kind of DPI, forced blocking utilised by these middleboxes is certainly a step above that, to the point that most people will not be (say) using measures like a VPN to bypass the block.
0: eg, China, Iran, etc; if you don't think people getting caught is a problem then I question your basic human decency; if you don't think 5% of people getting caught is a problem then I question your sense of scale and/or ability to multiply numbers by other numbers.
There is no way - not even a theoretical way - to allow blocking of illegal content (for any definition of illegal) that won't allow for blocking of any other arbitrary content. Censorship is binary. You can accept either none of it, or all of it.
So if you say censorship is binary, it's already here, and has been here for ever. But I would guess that few believe that censorship is truly binary like you say.
You can't often choose your ISP so this makes it extra important for censorship of any kind of to be opt in rather than forced.
I think I just got everyone to take a step down the slippery slope.
Hi! Counterexample here!
Well, there is: take the person or body that ultimately determines whether content is illegal, and have them review each request and proposed response and decide whether to allow the content through to the requester.
For slightly better scalability, have that body review all content outside of any request-response cycle before it can be published and sign any approved content, then block any content they haven’t signed.
Somewhat more generally, as long as the specific blocking methodology is itself part of the definition of what content is legal, any blocking method can meet the standard of “allows blocking illegal content without allowing blocking of other content”, since any content blocked by the method is, ipso facto, illegal.
There are also civil and criminal liability concerns at the corporate level by assuming the responsibility for constructing and/or maintaining these filters.
But what you said reminded me of a conversation we had here a month ago. I think it may be that I reserve image upload functionality for users who have proven their humanness (and their humanity).
In my case image quality matters much more than quantity, so I can afford to make that choice.
(I have to deal with hijacks frequently and part of our investigation is beating on folks who have permissive filters and pressuring their peers to improve things)
But the objective of the project is only to identify governmental and ISP-wide censorships. And its capabilities for checking protocol-level censorship techniques are rather limited [0]. Perhaps we need a similar tool or fork for probing middleboxes and censorship in private networks.
How does that compare with something like RIPE ATLAS? Is there a list of such monitoring projects somewhere?
However, the reason I went for probing the entire path is because the TTL itself can be spoofed
IMO the only way to do that would be to either (i) block the IP (high collateral blocking) or (ii) block TLS 1.3 itself (GFC does this).
A major blocker in answering this is finding a potentially blocked website that also supports TLS 1.3
As a result IETF standards are only proposed and that is as you say "the end of the road".
Not really, after PROPOSED STANDARD there's INTERNET STANDARD, the STD series. For instance, IPv4/ICMPv4 (RFC 791/792) is STD 5, UDP (RFC 768) is STD 6, TCP (RFC 793) is STD 7, DNS is STD 13, and so on (STD 1 has the full list).
However, an IETF standard only reaches that level after it's been in use for a while; according to RFC 2026 (BCP 9), "A specification for which significant implementation and successful operational experience has been obtained may be elevated to the Internet Standard level. An Internet Standard (which may simply be referred to as a Standard) is characterized by a high degree of technical maturity and by a generally held belief that the specified protocol or service provides significant benefit to the Internet community."
TLS 1.3 works fine in China, but if you use TLS 1.3 with the earlier proposed encrypted SNI draft it is blocked. The Great Firewall can't tell which name you actually wanted, but it can tell you're encrypting the SNI and block that.
With the currently proposed Encrypted Client Hello with a GREASE-style dummy ECH on all connections (so the "real" Hello is sometimes in an encrypted block and sometimes that encrypted block was just noise), China would still be able to choose to block all ECH-enabled connections since their presence is detectable. This would break everything, but China can choose to do that. What happens next is a policy question.
If you want to sneak past nation state snooping you need something else, that's not what TLS is for. The TOR project does not directly offer this either, but they can help you find out how to connect to TOR in a sneaky way if that's necessary for you.
things generally along the same lines as obfsproxy, and traffic level steganography and obfuscation/mixing.
But Airtel really, really wants to run scripts and show ads on blocked pages.
Duckduckgo - https://i.postimg.cc/SqkRhpRC/Mozilla-Firefox-29-09-20-w-PA....
Pirate Bay - https://i.postimg.cc/qMmwMXVY/t-29-09-20-9-Dv.png
Reddit and Github have previously been temporarily banned in India due to similar "mistakes"
Whether that's a mistake or they've made an enemy of some sort is not clear. India is a democratic country but not an especially free one.
Netsweeper is a Canadian company in the business of content filtering: https://www.netsweeper.com/
They openly state that they supply ISPs with DPI hardware, I talked to them in person 15 years ago and they had no problem to admit that they also supply government institutions.
To be fair, there's probably less than ten manufacturers of their category of spectrum analysis gear (for commercial/non-military use) in the world, and their stuff is top quality.
I assume they can't do much about Toosheh since it's "read only", multiplexed with legitimate TV channels on the same transponder, and uplinked from the UAE.
Try setting up an independent two-way satellite based C or Ku band earth station in Ethiopia, offer service to your neighbours and armed men will come to dismantle it.
Commercial spectrum analysis tools are an essential and important things in the hands of network engineers, but also a tool to crack down on anything that transmits that an authoritarian regime doesn't like.
In the case of an FM radio, this means you can tell what station a given receiver is tuned to, if you make certain assumptions about common LO/IF frequencies. There was a company monetizing this called "Mobiltrak", but I haven't seen much about them lately.
In the satellite case, there's the IF of the LNB itself, which leaks fairly loudly out the feedhorn, but it only tells you which band they're tuned to. There's likely a second IF used in the IRD, which should be much fainter from outside, but if you could recover that, you could tell which transponder is being demodulated.
That still wouldn't tell you if the Toosheh packets are being saved, but if the program they're muxed with isn't particularly popular, it would be a strong hint.
https://i.postimg.cc/SqkRhpRC/Mozilla-Firefox-29-09-20-w-PA....
P.S. From the screenshot, it looks like you're trying to connect to [http]://duck.., hence shifting to https works.
This also hints towards a mixture of plain old http censorship and https censorship, which Airtel (in fact all ISPs) do randomly
When it was blocked a month ago, there was no notice. Now, there is a notice that it is a TRAI order. usually seen on sites that the govt themselves ask to block (piratebay, torrentz.eu, etc)
Until around 2009-10, when you are traveling out of state, you had to pay roaming charges. Worse used to be metro cities within their own states as they used to be different telecom circles. I used to pay roaming charges when going to Chennai from rest of Tamil Nadu. Even the operators were different sometimes. E.g. there was no Hutch (now Vodafone) originally in rest of Tamil Nadu and they operated only in Chennai. Similarly RPG (later Aircel which went bankrupt couple of years back) had 2 networks - RPG in Chennai and Aircel in rest of Tamil Nadu. It used to be a mess.
No operator had pan-India operation as every small operator had their own fiefdoms and the big operators like Airtel used to pay roaming charges to those operators for their subscribers to get signal.
This all slowly went away only early this decade.
tcpcrypt just does the encryption part. Once the connection is established, userspace on both sides can invoke an ioctl that provides a session nonce. If the nonce matches on both sides, the connection is not man-in-the-middled. It’s easy to confirm the nonce matches: Both sides sign it and send it to the other party.
This has two big advantages:
(1) operating systems can opportunistically encrypt traffic for unmodified legacy applications and network protocols, allowing the endpoints to detect mass surveillance without requiring any certificates.
(2) Like newer versions of SSL, it encrypts the information currently sent in cleartext in the SNI, preventing the type of censorship in the article.
A) some kind of secret that only the server knows that the client can verify in order to ensure it's not trading nonces with the MITM.
B) A way for the client to ensure that the nonce isn't being passed through a second tcpcrypt session between the MITM and the server with the connection being in cleartext between the 2 tcpcrypt streams.
Currently the best supported method of implementing both A and B is certificates, which means you may as well use TLS.
Even if you don't authenticate at all, it makes it much more expensive to intercept all these connections.
And TLS lacks a way to automatically apply it to all connections.
Also I don't understand what scenario you're outlining with B.
Client establishes a tcpcrypt session with what it thinks is Real Web Server but is actually Evil Middlebox replaying the request to the server and the response back to the client.
GFC does this, I really hope it doesn't happen here
In this case the report says the Great Firewall was determined to block the following specific combination:
* A ClientHello for TLS 1.3 that * Includes the 0xffce extension value (used for experimenting with an earlier SNI draft)
If you add a 0xffce extension full of random noise, the Great Firewall blocks it. If you use the same random noise but pick a different extension value (do not do this in production code - those aren't for your meddling!) the Great Firewall doesn't interfere at all.
We have yet to discover what happens if a big bang release of Encrypted Client Hello (the current iteration of the encrypted SNI work) just deluges the Great Firewall with ECH connections. But we do know TLS 1.3 has been used successfully for years from China.
You also mention this idea that it would "force hosts to drop down to 1.2 for connections".
It is hard to tell what you intended here, it would of course be possible to force the humans using a computer to downgrade, or to disable encryption, or to cease using a computer altogether, perhaps you could put a gun to their heads for example.
But TLS 1.3 has an anti-downgrade design. A [edited to add] modern TLS 1.3 capable web browser which connects to a TLS 1.3 capable web site but finds that the connection has been negotiated as TLS 1.2 instead will reject the connection as clearly under attack, you cannot reach that site until the problem is remedied. I think you would notice if all TLS 1.3 capable sites (about a third of popular sites) suddenly did not work from China, even the Chinese government might struggle to silence such confusion and dismay from their people.
Read https://gfw.report/blog/gfw_esni_blocking/en/ some time back, recalled it incorrectly
The one thing TLS1.3 with ESNI is not is hard to detect. It's a consistent traffic pattern if you throw a sufficient amount of CPU and RAM resources at doing DPI on each and every user's flows.
In an ordinary non censored ISP environment the ratio at which you export netflow data to a collector adjacent to the router is quite low. And not a great deal of CPU and RAM resources are put into doing detailed analysis of it, other than for basic things like figuring out who you should be peering with that you aren't peering already, and identifying percentages of traffic patterns (eg: at 10pm every night we see this much traffic from our on-net locally hosted netflix cache boxes going towards the residential GPON customers).
If you are a Chinese entity with access to the router-design people at Huawei and ZTE, and sufficient motivation to do so, there's no reason why you couldn't crank up the ratio greatly and (on a middle mile and per POP basis) export netflow by a dedicated 100Gbps link to a set of directly-adjacent high performance x86-64 servers, running custom flow analysis and DPI inspection software.
does that makes sense?
Tor would similarly work.
If a box was censoring after Airtel, we would have received a clean response (ICMP timeout) at hop k as well. Of course, the TTL itself can change during the run, but that wouldn't happen for so many cases :)
Or, you could decide to just go HTTP-only for your blog, and never bother doing any of the above, never worry about any automation failing for any reason, never worry about any expired or revoked certificates, never worry about the extra compatibility issues that TLS brings. There's no benefit for HTTPS for a personal blog. It's only there to restrict the access, increase attack surface, and cause compatibility issues.