Something is amiss with the Interwebs: BGP is a flapping
isc.sans.edu
isc.sans.edu
And that's just the top 30 networks — if every network cleaned up their announcements, it would eliminate ~232,000 routes (~45% of the table).
Adding to the deaggregation problem is the inability to easily filter out route announcements based on RIR minimum allocations without having to add tons of exceptions for CDNs that operate as islands of connectivity and carve out IP space for each island from a single address space allocation. (There's no covering route for the islands of connectivity since these CDNs have no "backbone" connecting the islands, so if you filter out those smaller announcements, you lose connectivity to those islands.)
There are many people who think this problem will just magically go away as IPv6 adoption increases, but all increased IPv6 adoption will do is make limited CAM space even more limited as network engineers have to balance dividing precious CAM space between a ballooning-quickly IPv4 route table and a ballooning-slightly-less-quickly IPv6 route table.
(To be clear: I think ubiquitous, functioning, end-to-end native IPv6 connectivity needs to happen sooner than later, but it's not a magic bullet for the Internet's technical problems.)
The maximum IPv4 de-aggregation possible is 2^24 - 2 ^ 21. The maximum IPv6 de-aggregation with what's currently being handed out (mostly between /32 and /48's) is way, way, way more.
So IPv6 has more addresses, yes. Just so long as you don't actually use them. The problem of course is that the memory requirements for IPv4 assignments were going up linearly. If you bought gear with a good amount of memory, you could therefore expect it to last a few years. Clearly the network vendors that designed IPv6 saw this as a problem ... how can we make it explode ? Well IPv6 was the answer. Problem is that they went completely batshit insane overboard.
"If" IPv6 deaggregates we'll need routers with about 2^(48-24) TIMES more memory. Storing a deaggregated IPv6 routing table (which has to be in memory) requires 524288 terabytes of memory.
The rest of IPv6 isn't much better. It's some academics wish-list, with total disregard for real-world concerns. It is not possible to use 1/10th of the IPv6 features. Multicast ? Won't work (on any public or even just somewhat large network). Encryption ? Won't work (too many devices don't support it). Anycast ? Won't work. Site-local anycast ? Won't work. Larger address space ? Won't work (in half the world). NAT-avoidance ? Won't work (didn't get past security engineers). Autoconfiguration ? Actually kinda handy in some scenarios, but again, won't work on most networks, where you fall back to the IPv4 mechanism. Faster routing due to "smarter" headers ? Doesn't work according to my load stats. Better Qos ? Doesn't work in either of the major routing gear vendors. Mobility ? Let's not go there. Instead of implementing sane "live" re-addressing they went with ... Aargh. Let's not go there. Automatic network renumbering ... riiiight. Heh. I wonder if this was put in as a joke.
There are known solutions to all these problems (well, except the multicast and anycast ones), but of course, the IPv6 designers knew better.
Welcome to the "solution". On the other hand, solving the solution pays rather well.
But it is still a lot of network activity and IPV6 is just around the corner so probably not worth building today.
1) you'd need at least 1 byte to identify a next-hop, right. So we're talking 8 bytes * 2^32 = 32 gigabyte, not 16 2) Luckily IPv6 fixed that. For IPv6 you'd need millions of terabytes to do this trick. 3) At, say, 64 10Gbit ports and 130ms lookups you'd need a mere 215 of these memories to actually be able to forward line rate. 4) Network devices are currently moving up to 40G and 100G ports in the top models. 8*100G forwarding cards exist today.
I used a similar 'the address is an operand' trick to create a very fast 8x8 multiply on an 8 bit Z80, arg 0 was the upper 8 bits arg 1 was the lower 8 bits which was applied to A0-15 of a 64K x 16 EPROM, the contents of the eprom at any given location was the product of arg0 and arg1. That allowed me to save enough time to respond to packets coming in at 38400 baud over an FM side band at the time.
If you had 100M of L2 cache, or 100M of SRAM, you could fit the third level in what amounts to L2 cache. You'd still have (at most) 1 memory lookup for any route (1 lookup in SRAM, 1 in RAM, worst case), but you'd need something like 256M of ram instead of 16 gigs. Vast majority of packets (going to < /24 routes) would be routed in < 10 ns, and the 130 ms becomes a 140ms worst-case. With average < 10ns routing suddenly having one of these processors for, let's say, 160 Gbit worth of ports becomes possible. And I'm sure your customers would immediately ask you to put 320G of ports on that processor.
(Principle of a trie is that IP address is a.b.c.d. So you build an array so that route_table[a][b][c][d] yields the next hop. route_table[a][b][c] happens entirely in cache + SRAM, since it only requires 100M)
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-...
Takeaways: a) 512K routes isn't necessarily a hardware limitation, it's the default TCAM allocation for IPv4 and B) most people most of the time don't need their routers to take a full BGP feeds worth of routes - and I hope those that do aren't running 6500's in Q3 2014 ;)
http://www.cisco.com/web/about/doing_business/memory.html
Reminds me of when I worked at a large public university... the networking group planned on losing/replacing dozens of switches and power supplies during the summer when the power plant underwent maintenance. Losing power and reloading surfaces all sorts of defective components. And, when you have thousands of switches...
Btw, thanks for the link! I suspect more than a few 6500 linecards that I've seen die in years past were due to something similar, as opposed to what we'd previously assumed to be the root cause - either damage in transit or "cosmic rays" ;)
It's very odd that "grey market" equipment is considered the standard way to refer to genuine Cisco equipment when sold by a company other than a Cisco partner. I understand keeping out counterfeits, but given the first-sale doctrine, how is a resold piece of equipment anything other than completely legitimate?
If you think about it, and get past the "it's just industry standard" mentality, it's generally insane the way that Cisco uses these pseudo-monopoly tactics. In the old days, say with maintenance on IBM Selectric typewriters, such schemes were called "bundling" and "tying," and the DOJ would pursue the companies for anti-trust violations.
Now, the DOJ arrests people based upon nebulous complaints from Cisco's general counsel. See e.g. http://abovethelaw.com/2011/07/sue-a-giant-corporation-get-r... wherein a British citizen was arrested in Canada for starting a company that competed with Cisco maintenance.
The Canadian court quashed the request for extradition after the DOJ's request trapped him in a foreign country for years. He remains under indictment here, despite the Canadian judge stating that the DOJ's case was a fairly transparent copy of Cisco's civil suit. The ruling was incendiary, stating that The extradition process to bring the applicant before United States Courts… involved innuendo, half truths and complete falsehoods.
The judge concluded:
The only reasonable inference I can draw from the facts is that the criminal process was used to pressure (unsuccessfully) the applicant into abandoning his antitrust suit against Cisco…. Any well-informed person acquainted with the truth would conclude that the collective result of the mistreatment of Mr. Adekeye offended fundamental notions of justice.
Now, there are court-recognized exceptions to this rule in various circuit courts (most significantly by the 9th Circuit in Vernor v. Autodesk, SCOTUS cert denied) for digital goods under the so-called "shrink-warp licensing" exception. [1]
So given that the highest court has allowed the ruling to stand in a large circuit, yet has consistently expanded the first-sale doctrine for four decades [2], the precedent is unclear.
Cisco does rely on Vernor and its progeny as it's justification for blacklisting resold items. But that argument is unlikely to hold water outside of the software-laden 9th Circuit.
The question, stated in layman's terms, is, "To which is a Cisco router or switch more similar, a copy of Windows or an iPod?" This is an open question from a legal standpoint, and thus we must unfortunately resort to that most unreliable of legal tools: reason.
Both options are almost equally unpalatable to the Supreme Court for policy reasons. Expand the shrink-wrap exception to cover Cisco and you risk encompassing all products that contain firmware, from TVs to microwaves, and thus strangling the half-trillion dollar secondary sale market for electronic goods. Keep the exception narrow and you choke off the single largest source of revenue (all told, including gains from new purchases, maintenance contracts, and paid software updates close to $20 Billion, or 40% of annual revenue) to the government's largest IT hardware provider (over 85% of DoD in particular), leaving the government stranded with vast networks of legacy Cisco hardware with little to no new development from the company.
The existing legal precedent creates a tough needle for the Court to thread. As a result, we have the current state of limbo, and thus discussion of "grey markets," which are neither clearly legal nor clearly illegal.
[0] https://en.wikipedia.org/wiki/First-sale_doctrine [1] https://en.wikipedia.org/wiki/Vernor_v._Autodesk,_Inc. [2] Most recently in Kirtsaeng v. John Wiley & Sons, Inc., No. 11-697 (U.S. Mar. 19, 2013)
>leaving the government stranded with vast networks of legacy Cisco hardware with little to no new development from the company.
Do we have reason to believe this is true? If resale was allowed, do we believe network hardware companies would just shrivel up? How do we square this with the facts of many platforms being intentionally crippled for marketing reasons? Or does that not happen on top-end platforms?
Network hardware companies in general wouldn't suffer, but Cisco's increasing reliance upon support, service, and firmware for revenue means that unrestricted resale would bite into Cisco's revenue hard, perhaps 25-50%.
Right now Cisco hardware powers the most critical government information systems, including military and financial communications (over 85% of the gear is Cisco.) Since many of these networks use Cisco proprietary functions, have scores of personnel trained on Cisco gear, and relationships with networks of Cisco suppliers, transition to another hardware manufacturer would be both expensive and difficult, at a time when the government has little funding for new initiatives.
Since the government needs Cisco to be a smoothly functioning company to meet critical information needs, it makes sense that they maintain tight relations.
Hmm, I just realized, if enough peers with 6500s/7200s have flapping BGP sessions with their upstreams, that would be a problem even for upstreams with beefier routers and more available TCAM - too many flapping peers == too many dampened routes == lots of "fun" resetting BGP sessions and coordinating with peers. Yipes.
http://markmail.org/message/n32fmeb2dmtnbsff
I find the economics of the routing table to be fascinating. When someone announces a route, it makes use of a constrained (and often expensive, TCAM-based) resource on routers all over the world. More discussion:
edit: I'll take it by the downvotes without responses that's a "no"?
BTW, whatever you do don't overuse the 'flag' feature on stories as it seems when this is revoked it is permanent! I lost mine when trying to encourage news other than the passing of Steve Jobs.
However, applying it successfully to IPv4 is severy compromised because, due to rampant address space fragmentation during the last years, increasingly longer prefixes (i.e., smaller address blocks) have had to be assigned. IPv6's much larger address space makes it unlikely that the routing table will ever become that fragmented in the foreseeable future.
http://www.iana.org/assignments/ipv6-unicast-address-assignm...
8 years of stability is hardly a "poorly planned mess".
To the point that the linux documentation project still recommends the following as an ipv6 default route example: /sbin/ip -6 route add 2000::/3 via 2001:0db8:0:f101::1
Yea.. it's /workable/, but a lot of decisions with respect to IPv6 feel like wasted opportunities that have only slowed adoption and promoted general confusion.
Some gotchas emerge when you start thinking about actually implementing this. How can you aggregate a city under a single prefix if it has multiple ISPs?
OH WAIT, WE DIDNT FKIN THINK OF THIS IN OUR RUSH TO PUSH A BROKEN INCOMPLETE SOLUTION.