How Syria Turned Off the Internet
blog.cloudflare.com
blog.cloudflare.com
Keep up the good work guys.
[1] http://blog.cloudflare.com/why-google-went-offline-today-and...
I wonder if it would be possible to hack into these "wall" routers from inside the country (or I suppose outside), but something tells me probably not (I doubt they would even be ping-able, but maybe through other attack vectors [like other internal computers]...). Seriously, I can't imagine how enraged I would be if someone decided to simply block the Internet.
It also seems like it would be a really cool place to work.
And, if you're interested, we definitely are hiring:
[1] http://www.renesys.com/blog/2012/11/syria-off-the-air.shtml
For those who are wondering your edge router (or border router) "advertises" that it can route to a particular subnet. That information propagates around and packets find there way there. So someone in Syria told all of their border routers to stop advertising routes to Syria's IP blocks. Now the fun thing you can do is since they aren't advertising those routes, if you are sitting in a data center somewhere and have peering access and a ASIN id you can advertise those routes and all of Syria's traffic will start heading your way :-) Of course if that monitoring tool is still running it will have all these lines suddenly running off the screen toward your data center.
The traffic that is actually in Syria can't get out. So its not like you could snoop on Syria or anything.
29/11/2012 10:20 to 10:30.
I'm not really routing savvy, but isn't it a little bit more like, the traffic could get out but no one could reply?
There's some guesses in here, but the title is rather misleading. No one still knows why these routes are no longer being advertised, only that they're not.
The blog post also makes mention of traffic being disrupted a couple of times on varying scales before this major outage occurred which kind of signals that they were testing bringing the whole Internet down beforehand.
Which I think is more likely, considering the relatively few points of failure.
http://thenextweb.com/me/2011/06/07/this-company-is-helping-.... http://www.thestar.com/news/canada/article/1218965--guelph-t....
They have previously been used by Egypt to close down their internet. I have no insight into how this particular incident was accomplished, but this sort of software is where I'd start looking.
The interview itself was interesting. The technical tests etc were relatively trivial, mostly stuff you'd expect about load balancing and responsiveness (as would be required for routing software). But more than half of the interview was them pitching the company to me -- they're quite aware of the reservations people might have about working for a company whose direction is to suppress free speech.
Their basic pitch was that they believed all 1st world governments would have internet filtering technologies in place in the near future -- that governments would legislate the use of it to stop the proliferation of things like child pornography, and eliminate the zombie-computer defence -- that if there were child porn on your computer, you went and got it.
We talked a little about the technical challenges of instantly shutting down the internet of an entire country, like Egypt, but they more or less blamed that on "improper configuration" of settings. At one point in the interview, the interviewer told me that you have to believe that the people that make the weapons aren't the ones firing the guns, and that dangerous tools can be used for good, but that yes, there were nights in which he cried himself to sleep.
Super interesting interview.
That leaves a few options of how to shut off the internet of a country, and they give a pretty plausible one, and who would have the authority/power to do it. And they back it all up with evidence.
Unfortunately, non of these hosting companies want to give an alternative to HOW to bring the network back up...
Anonymous seems to be the only group oriented at actually helping the citizens of the nation of Syria regain communication via alternative methods such as TCP/IP over HAM radio, and satellite links, personal wireless mesh networks using WiFi on mobile devices.
Everyone can bitch about HOW to take DOWN a nations internet but, it takes real humanitarians & 1337geeks to consider & implement HOW to bring a nations communications infrastructure back UP.
So, what are you waiting for...
HELP.
It is the same source in which it stated that the data center that handled the BGP routes was being held by 6 remaining techs whose current location is now unknown.
I wish I could find the link as it proves much of what the OP posted. Not to mention I would love to see someone repair one, let alone four cables in fifteen minutes. Heck, I bet it may take more than fifteen minutes to cut one.
Maybe you're right, what would a company who was voted Most Innovative Network & Internet Technology Company of 2011 & again in 2012 by the Wall Street Journal know about networking and how Internet traffic is routed, right? The very fact they even embedded a video showing the traffic dropping off in the blog post I think proves they know a thing or two about network traffic, they provide content delivery and domain name server services to a lot of happy customers after all.
This was a good read and informative. One question, though.
> When the outage happened, the BGP routes to Syrian IP space were all simultaneously withdrawn from all of Syria's upstream providers.
Does withdrawn mean not advertised or was a message sent out saying these routes are no longer available?
CloudFlare, if you have access to their data, what's interesting about them?
What gives Syria the authority to do this? What gives anyone authority to do this? What prevents malicious routing?. Could they route all traffic to 8.8.8.8 and overwhelm another network? If it's ICANN, can they come in and revoke control and give it to a third party intermediary?
It has been rumored a small staff of 6 stayed trying to keep the routes up. Their current status is not known. I understand in those cases no intermediary would help. But from a pragmatic standpoint, I'm curious.
While I understand this goes against the "rules" but if I have a DNS server and the roots drop a zone, and I don't agree, I can add it back in. As a local user I could add to etc/hosts like in the old days.
The above assumes I had a large user base like openDNS or google pDNS to be effective. Can the same be done with BGP? Can major broadband providers decide to ignore the dropped routes and send traffic along?
I understand Syria would just toggle off some other "switch" and terminate core routers but it would at least send a tiny message of sorts.
How are they stopping satellite access?
How are they stopping cellular based access?
Is there any form of TCP over Ham radio? TCP over laser? USB over carrier pigeon (seriously)? What are the bare bones options here for getting data in and out and where is that closest point of access?
No one has put in long range wifi links of the 20 mile line of site type, or is that still too short a distance to get a few users online?
What about dialup?
If CloudFront can see this much traffic, they must be doing pretty well. What is the point of Facebook, reddit, and many others using CDN's and Amazon and such when they could probably half their hardware and push the rest to ClouFront. Or is CloudFront really only best for static sites that are hit hard and need lots if bandwidth. Dynamic sites would still bottleneck at the database/drive/physical/etc layer?
Thanks. Sorry if these are rudimentary questions. I haven't even met that many people who have had BGP access. I'm going to go look up what the format files look like now, just out of curiosity.
> Can major broadband providers decide to ignore the
> dropped routes and send traffic along?
A - B - C - Syria
This is my understanding of how it works.
> How are they stopping satellite access?
> How are they stopping cellular based access?
> Is there any form of TCP over Ham radio?
> TCP over laser?
> USB over carrier pigeon (seriously)?
> What are the bare bones options here for getting data in and
> out and where is that closest point of access?
> What is Syria's cellular data infrastructure like?
Also, Syria enforces a web filter(similar though not as sophisticated as china's) which also affects browsing over the cellular data network. So even cellular providers eventually go through some choke point which is state-owned so that the filter can be applied.
In other words, if the state wants to cut off internet, cell isn't going to save you.
> USB is a client-server protocol. It probably wouldn't do too well over a carrier pigeon.
HERE is a link to an article featuring full-duplex voice over cheap laser/LED optical link over 95 miles.
http://hackaday.com/2012/09/22/communicating-with-a-beam-of-...
If anyone can get that to the "right-side" then, by all means this may very well be an option for establishing high-speed TCP/IP as well.
Yes, there is AX-25 packet radio and there is TCP over that.
And yes, if the phone system is still operational, the easiest method is to use an international dialup service.
Further corroborating that theory, when we first got there, we were told by Syrian friends never to discuss anything that might be possibly misconstrued as political over the phone and especially international calls. I was there when the trouble started(I eventually left around June of 2011, some 3 months later) and during that time I'd have friends/family call to check up on me and I could never give any status updates about what was happening just that my surrounding area was calm and that I was fine.
TL;DR; I would guess that international dialup, if you even managed to keep it up for any length of time given the difficulty of calling international would quickly flag you as "suspicious" as these things are heavily monitored.
OK, I'm curious! How do they know? How many cables connect, say, Brazil to "the Internet?"
Is this publicly available somewhere?
http://nicolasrapp.com/?p=1180
Good find!
and if you've got an hour (or two) to waste http://www.wired.com/wired/archive/4.12/ffglass.html
This seems to be like some sort of incompetence, more like they tried to set up some sort of spying choke point and it massively failed. ask nokia-siemens, they helped iran set up their chokepoint, most likely some infosec whitehats with zero ethics are currently flying out there to assist in the holocaust, er I mean rebellion put down, by working with Assad to get the tubes back up.
It only hurts Assad to keep the tubes out, his loyal base needs to buy their louis vuitton bags off alibaba to keep their minds off the constant public shooting of protesters and shelling of entire cities full of "terrorists"
Also, no, having the internet cut off absolutely does not hurt Assad. Syria is very sectarian in nature. In other words your loyal base absolutely does not correlate with what services you provide, only what religion/sect/tribe people are.
The rebellion didn't happen because people suddenly started hating Bashar el Assad, it happened because those same people have hated this regime for at least the past 40 years. In 1982, Hafez, the guy who established the regime put down an earlier rebellion that had gone on for something like 3 years by completely destroying entire sections of cities(and pretty much all of Hama) and killing, by conservative estimates, at least 10,000 people. This was during a time when the USSR still existed and where such things within its sphere of influence were par for the course, which is why no international response happened at the time.
The rebellion happened now because of a perceived weakness in the regime's freedom to act(no USSR anymore, much more US influence in the area, weak(er) dictator) and because of perceived international support for similar things happening in Arab nations. That's it. So if the internet in any way helps the FSA get international support by exposing atrocities committed against civilians by the government then you can bet it's in the government's best interest to shut it down.
They've also been targeting wireless uplinks across borders and adhoc cell towers. Syria levelled an entire apartment building full of kids in Turkey to destroy a suspected adhoc cell tower they claimed was being used by "terrorists". This is serious business, no piratebay UAV drone solution is going to work. Mesh networking is also certain death, any building with a mesh router on it is going to have all its inhabitants dragged out of their apartments and shot in the street so they've been using good old walkie talkies and bouncing signals all over the place to confuse the syrian army, and in most cases simply stealing Syrian army comsec devices and pretending to be them talking in slight code with each other to avoid suspicion. Al Jazeera ran a story with some smugglers who used gps disabled cellphones but changed the IMEI every couple of hours to avoid being found
I haven't kept up on Syria or Libya (I wasted 2003-2010 on this stuff in Iraq/Afgh/etc., and am trying to do a "normal" tech startup now), but while I think Syria (and Libya) had better European gear than Iraq or the Taliban, it isn't on par with the US, UK/FR/DE, RU, CN, etc. It's basically "good commercial equipment designed for law enforcement", which is very heavily cellphone focused.
The #1 vulnerability with satellite systems remains "operator assistance to the adversary", or "network configured in a way which relays location data of connected terminals to everyone in the footprint", both of which can be addressed if you control the network.
How could the information on that page be spread within Syria? SMS, MMS, Phone, Fax, Mail?
Guess they still give access to those who are loyal...
Apparently, bombing people turns out to be bad PR when they tweet about it.
Did those routes stay up? If not, what are the repercussions of that going to be come open time? Or any foreign trade, anything that relies on network time, etc. the list is pretty large for services that need Internet at least infrequently in order to keep basic services up.
I can't imagine how many deaths there would be in the USA if this happened. All those televisions stop working, that's gotta be a few million heart attacks right there.
So no, no markets in Syria.
Does the government have to pay for those lines? Or is that handled by some other entity?
Err... well, that's great. I'm glad that this information free blog post pimping your services ended with your sincere token of solidarity with the Syrian people.
For an article with real information from people who actually understand what a network is, go here: http://www.renesys.com/blog/2012/11/syria-off-the-air.shtml
(Yes, I'm a little annoyed. But that's just because a lot of companies are submitting their "informative" blog posts on HN while really it's just more pimping of said service.)
As far as I can tell, both blogs are run by companies in the network infrastructure space. The chief difference appears to be that CloudFlare bothered to express any solidarity at all.
.. but its true his link has less pimping and is more of a tl;dr; than a "oh we rock because we can read a bgp table" ;-)