Internet traffic for Google, Facebook, Apple was briefly rerouted to Russia

I just have to ask, not knowing how to setup BGP but having a lot of experience with PF and ipfw; is it not possible to create a sort of default deny filter so you only exclude the routes you have?

The way your post read it sounded like you had to create a filter for all routes in the world but I assume that's not the case in the practical sense.

I understand ISP's are to blame here, but how is it not about Russia, the article specifically states that only certain entities were targeted in a suspicious manner. Seems like nefarious purposes at work here no?

It's unfair to say it's a "feature". It was an oversight back in 1989 based on nobody expecting we'd end up where we are today.

Filtering is easy if your just doing it for downstream customers.

But how do Tier 1's filter from each other? Once you go up the tree a few times it gets really hard to build any sort of valid filter list of what you should be getting from provider X or provider Y.

BGPSEC and RPKI are unfortunately not really workable yet, in terms of full-path validation. And obviously the source data to do it is a long way off.

I work for a medium sized ISP, we have about 500 peers over exchanges like LONAP and Linx. You'd be surprised how many of our peers won't use MD5 passwords or even filter properly.

Yes, and, to be fair, it's been a known problem for years. There are some potential solutions in the works but you can't just pick an arbitrary day and flip a switch and turn it on.

Most of these incidents are accidental and could easily be prevented. The network operator community can't even get everyone to implement BCP38, though, so something like a full Internet-wide RPKI is a pipe dream. Even basic filtering would prevent much of this but that's apparently asking too much as well.

The amount of filtering, if any, varies tremendously from network to network. Some won't accept routes without IRR entries, LOAs, and a bunch of paperwork. Others (such as AS31133, apparently) will accept any prefix you want and as many of 'em as you want.

I'll give (some of) the Tier 1's a bit of a break, as filtering on every single BGP session just isn't feasible (although some of them wouldn't bother filtering even if they could!). Pretty much everyone else should be filtering.

But they aren't. And they won't be any time soon. And so we'll continue to have incidents like this for the foreseeable future.

Yes, but to be honest, it's not just BGP. One of the things that BGPMon won't tell you about since they're in the business of monitoring BGP is that the same attack is often performed by changing forwarding tables i.e. in network's data plane.

An attacker compromises a router (there are known vulnerabilities and back-doors) or gains leverage over a technician authorized to configure routers in some network. The attacker then manually changes the router's local routing table to forward packets to specific destinations through a different path. In such cases, the BGP path and announcements look absolutely fine and you'd need to look at the actual path at the data plane level to detect that something is wrong.

This is often done in big exchange points where many networks interconnect. These are places in which routers from major western countries can sit basically side-by-side with routers from Russia and China.

This isn't a paranoid fantasy by the way, I work for a company that monitors these kinds of attacks for a living.

If you believe that Russia is less worrying than US, then... err... I don't believe you exist actually.

This is much better nowadays but bgp is still problematic.

The internet works by organisations "announcing" the IP ranges they're allocated to other networks (ISPs, other large companies etc.) Networks that receive an announcement will route traffic for those IP ranges to the network that announced them.

The protocol by which this is done is called BGP. The original version of this was created in 1989 and back then everyone involved trusted each other. So there is no security built in to prevent a rogue company announcing addresses they don't own, and therefore pulling in traffic people send to those addresses.

Organisations can and do put filters in place to try and stop this, but because it's not a core baked in feature of BGP not everyone does, and issues can happen.

Sometimes these things are just mistakes engineers make, re-announcing routes they've learnt elsewhere. Sometimes they are deliberate hi-jacks.

Basically, a router somewhere in Russia claimed to be the owner of some ip addresses belonging to Google, Facebook, etc. Other neighbouring routers began forwarding packets from actual users to this router. The packets contain the HTTPS requests people were making to these sites.

Internet traffic goes through whatever path will get it to it's destination fastest. Big Routers advertise to each other how long it will take to get somewhere through them. There have been times when either due to bugs, or due to a desire to slurp traffic, some machines have falsely advertised shorter routes causing traffic to pas through routes traffic wouldn't normally pass through. This time it was somewhere in Russia. This can be mitigated like most malicious acts / network errors if you write clever enough rules, but is hard because you have to identify the bad actors, or paths that lead to them.

Sure, it's a reason to blackhole the isp, but this is the Internet we're talking about. So say its upstream providers will backhole it, say even the tier 1 providers do the same.

That won't stop it from connecting to smaller networks and announcing the same fake routes there. And even if the small networks blackhole it, the attackers can just register a new AS.

Also, there are cases in which attackers are performed from large networks, with millions of users. Are you going to blackhole such a network? They'll apologize and claim that it was a configuration error.

Note that these types of attacks are generally not performed by amateurs. They're performed by states, intelligence agencies, criminal organizations and the like.

> Is it fair to say BGP in general relies on “good will” trust model?

Yes.

> I wonder if there exists an authoritative mechanism similar to browser punishing rouge or incompetent CA

Not really. Although if you get a reputation for being notoriously problematic, the "big boys" - the large networks who are the responsible adults in this game might deny you service or stop accepting your announcements.

> I supposed the integrity and the authentication is achieved by implementing and deploying RPKI

RPKI doesn't really solve the issue. It validates only the initial announcer, so BGP hijacks can still occur. It would however help reduce hijacks due to mistakes in configuration.

It's better than it used to be. Back in the mid-90s I was a newly-minted network engineer working in my first datacenter and we were all very excited about this new "BGP" technology...

Fifteen minutes later, half of Fairfax County is stuck in a routing loop, and we're getting calls from NOCs as far away as London asking us what the f* we're doing to their traffic.

That was the wild west: back then, a typo could (and sometimes did) actually take down The Internet. There's more filtering and checking and second-guessing today. But, ultimately, it's based on a web of trust model, you're right, with all the weaknesses that come with that.

In BGP you have sets of IP destinations, which you summarise as a prefix. Announcing or advertising means exactly what it sounds like, you announce to your peers that you have paths to these networks.

A prefix is an IP subnet, basically.

Most here are familiar with Google's public DNS resolver, 8.8.8.8, so let's use that as an example.

Google has been assigned the autonomous system number (ASN) 15169. The 8.0.0.0/8 IP address space was allocated to Level3 and they have reallocated a small part of it, 8.8.8.0/24, to Google.

Google, when speaking to their BGP peers, "announces" routes, including an announcement for the 8.8.8.0/24 prefix.

An announcement is basically Google's router telling, for example, Level3's router, "Hey, I know how to get to 8.8.8.0/24. If you have traffic going there, you can send it to me."

Level3 then passes that along to other peers, like me. Level3 says, "If you have traffic for 8.8.8.0/24, you can send it to me and I'll send it on towards its origin". The announcements continue to be passed along to other peers.

A withdrawal is the opposite of an announcement: "Hey, I don't have a route to 8.8.8.0/24 anymore" or, in many cases, "the route to 8.8.8.0/24 has changed, here's the new one".

Here's a (slimmed down) example from one of my routers:

  # sh ip bgp 8.8.8.0/24 | b 3356
    3356 15169, (received & used)
      4.69.248.15 from 4.69.248.15 (4.69.180.167)
        Origin IGP, metric 0, localpref 100, valid, external, best
        Community: 3356:3 3356:86 3356:575 3356:666 3356:2042

This shows the "AS path" (the path through the various ASNs back to the origin), "3356 15169". AS3356 is Level3. We can see that AS15169 (Google) originated the prefix (8.8.8.0/24), and announced it to their BGP peer, Level3. Level3 then passed that announcement along to my router.

Here's the same thing for 66.232.224.0/24, mentioned in the article:

  # sh ip bgp 66.232.224.0/24
  BGP routing table entry for 66.232.224.0/24, version 1125190207
  Paths: (1 available, best #1, table Default-IP-Routing-Table)
    Advertised to update-groups:
          4
    3356 209 40839, (received & used)
      4.69.248.15 from 4.69.248.15 (4.69.180.167)
        Origin IGP, metric 0, localpref 100, valid, external, best
        Community: 209:209 209:13070 3356:3 3356:22 3356:86 3356:575 3356:666 3356:2042

The AS path here is "3356 209 40839" which shows that traffic from me to 66.232.224.0/24 will first go to Level3 (AS3356), then to Qwest/CenturyLink (AS209), and finally to AS40839 (Kohl's Department Stores).

With regard to the hijack described in the article, this means that another organization announced the prefixes into BGP, effectively saying, "So, all of that traffic you have that's going to Apple/Facebook/Google/etc., just go ahead and start sending that to AS39523 now". Instead of ending up at Apple/Facebook/Google, it'll instead be redirected to this unknown organization in Russia.

(N.B.: Before anyone chimes in to "correct" something, this is a bit simplified. There's much more to BGP than this and there are a number of different factors which determine which route is chosen. A longer prefix -- mentioned in the article -- is one way to "defeat" a better route.)

Sometimes it's slightly simpler to show rather than tell.

The CIDR Report is a publicly accessible compendium of BGP routing announcements. Mind that this isn't the entire Internet, or even entirely current, though it's a very large and generally current version of it. One of the complications is that you're only seeing the routes the CIDR Report itself is aware of (there may be others, though generally those will be small/obscure).

But if you want to know what "the Internet" is, this is pretty much it: a bunch of rules for routing data.

http://www.cidr-report.org/as2.0/

As for BGP, Border Gateway Protocol:

https://en.wikipedia.org/wiki/Border_Gateway_Protocol

The routes advertised by the AS (autonomous system) in question, AS39523:

http://www.cidr-report.org/cgi-bin/as-report?as=AS39523&view...

(There are a maximum of 2^16 == 65,536 autonomous systems possible under BGP, at least until those counters are extended. What happens under IPv6 may be a bit of a mess....)

Edit: There's 4 billion and change as ASNs are now 32 bit, see: https://news.ycombinator.com/item?id=15915417

It's not just the public, it's professionals too.

There are probably more brain surgeons on earth than people who understand how BGP works.

Basic security. Like how everyone failed at keeping Equifax in check.

All AS's can be hijacked. Your suggestion is meaningless.

> I think governments should regularly perform live tests to check whether their countries' AS's can be hijacked.

That's a waste of money and effort. They should just write "yes" on a piece of paper, there's really no reason to check.

It was assigned by RIPE to "Vasilyev Ivan Ivanovich, 8 Pionerskaya st., office 10, Nekrasovka, Khabarovsk region Russia".

ALFA TELECOM s.r.o., in the Czech Republic seem to have acted as sponsoring LIR for the allocation.

RIPE rules require that the sponsoring LIR submit documents to them showing the legal standing of the organisation that will use the resources. So in theory that should have happened here.

Putin's approval rating is in the 80s.

That should be enough to tell you what they think of you without looking at polls.

(IIRC they were trying to shut down the one independent poll firm recently)

Assuming that state level actors don’t have ways to get around https? I’d expect a nation like Russia could acquire or procure new key for the targeted domains or perform some unknown 0 day attack on the targeted companies to acquire the original key.

Even with encrypted traffic, an attacker can learn things, especially if you get DNS and SNI data

I suspect (but cannot prove) that this might have been a leak from Russia's internal internet into the wider global net. Since Russia isn't well known for it's privacy respecting nature, it might have been a traffic scanner to see if people are being good citizens. However, that is just speculation and I hope it's wrong.

China did the same but for bigger scale in the past. The reason to do this is to store all traffic for future analysis. Now it is protected by https but there is no protocol or cryptography which couldn't be exploited in the future.

https://arstechnica.com/information-technology/2010/11/how-c...

Performing a DNS hijack has more uses than just reading the traffic. It gives you information on the amount of data that is sent towards a given prefix (among other uses, this can help for creating DDoS attacks later). It also makes those prefixes unavailable for as long as the attack persists, which is already a form of denial of service.

And a new ICO is born

It's called UUCP.

Transactions take 3-5 days to process, are sometimes lost, but hey, it's the next great thing, so worth every expensive penny.