BGP Implemented in Go | Hacker News Reader

BGP Implemented in Go

github.com

136 pointseatonphil66 comments

I've been working on a BGP implementation in Rust over the last few years: https://github.com/net-control-plane/bgp/

It's still in a proof of concept stage and nowhere as complete as GoBGP though.

Thank you for having a front page that actually says what BGP is.

I'm not sure if this will help explain the culture or not, but this is from the NTT (the Japanese telecommunications company), so probably they thought that everyone knew what BGP is (since that you probably won't encounter this otherwise).

Also, they have a Rust version: https://github.com/osrg/rustybgp

Searching around, I found someone that benchmarked gobgp, rustybgp, and some others, and gobgp didn't seem to do all that well compared to other implementations.

https://elegantnetwork.github.io/posts/comparing-open-source...

rustybgp seems to do pretty well (see #5 post of blog series) and is created by the same author(s) of gobgp. Additionally, it looks like the configs are transferable between gobgp and rustybgp.

Reading through the documentation it looks like this emulates the legacy IOS model of announcing all BGP routes to neighbors by default unless there is a policy configured.

This outdated model has caused countless route leaks over the years. Any new or updated BGP implementations should require an explicit allow policy for any route advertisement to take place (at least to eBGP neighbors). IOS-XR and JunOS require explicit policies.

This is covered in RFC8212: https://datatracker.ietf.org/doc/html/rfc8212

This is great, I have been learning "bird" and finding information on the web about using bird is very challenging since adding bird to any search doesn't necessary make the results better. If this project can maintain and keep their documentation clear, I would totally consider switching to this instead.

Yeah... Bird configurations are a little rough to learn. The examples they have on the website are pretty lacking. Their mailing list is usually helpful though.

Quagga is usually easier to find examples for since it's modeled after Cisco switch syntax for configuration (same basic syntax of arista switches).

Both of these really suck to learn as a programmer since you're usually coming into these without much of a network engineering background. I remember struggling a lot with both of these config syntaxes until a network engineer took me under his wing and taught me basics, principles, the engineering challenges of packet switching at scale, etc.

As a BGP alternative (in Go) has anyone checked out SCION?

https://github.com/scionproto/scion

I gave up when I saw new packet headers which means you might not be able to use existing ASICs.

SCION looks very promising, and by now it has also been deployed in Switzerland's financial network, so it seems ready for production:

https://www.six-group.com/en/newsroom/media-releases/2021/20...

(Actually, the original creator of SCION teaches my CompNet course, which is pretty cool. His rants on BGP are quite legendary.)

>"His rants on BGP are quite legendary"

I'm assuming the context of these is lectures? Are they online anywhere? If not could you share what specifically about the protocol they feel so strongly about?

Indeed, he talks about BGP and SCION in the lectures. Sadly I think the lectures aren't available to non-students. What is available though are explanatory videos by him (Adrian Perrig) on the SCION website, where he talks about the issues BGP has: https://scion-architecture.net/pages/videos/

The tl;dw is that today's internet has comparatively poor availability, packet paths can be hijacked and there's little transparency in routing. SCION sets out to solve these problems, and has a few nice things in store, such as guaranteed bandwith between two hosts making DDoS attacks much less effective (COLIBRI QoS), or an incredibly high throughput packet filter (LightningFilter).

Thanks for the link, this is great. Cheers.

Maybe a dumb question nowadays because I stopped using Go around the 1.6 days, but is there still a stop-the-world garbage collector, or any other potential blockers for high performance/real time?

A previous company I worked with chose to build in C++ instead of Go because of that, as the performance requirements were very, very high. I would imagine a serious BGP usage would have very high requirements?

It depends on the context. Not every bgp speaker needs the full internet table and frequent route changes. If you use the network to balance traffic across servers, you just need a bgp speaker on a server to inject a few routes for the ip addresses it will handle. In that case a good api outweighs performance.

BGP doesn't have very high performance requirements. It does not handle any internet traffic. All it does is configure the kernel routing tables. Even if there's some crazy 1 second GC pause it will not greatly affect the performance of the router.

Sending or receiving an internet full table (and potentially applying filters/aggregations) is still quite performance intensive, since it needs to happen quickly.

A one-second GC pause won't hurt you but managing the necessary data structures in memory can be challenging.

Anecdotally, gobgp is quite a bit slower than hand-optimized C implementations like frr (not that it matters for most gobgp use cases).

What's challenging about it? Go gives you control over the layout of your objects in memory.

> A one-second GC pause won't hurt you but managing the necessary data structures in memory can be challenging.

How so? It's a very common pattern in garbage collected languages to preallocate an array or structures and read/write into it, then "release" the array on finishing the function.

> Sending or receiving an internet full table (and potentially applying filters/aggregations) is still quite performance intensive

Nope. I am running 15 sessions with full transit tables, multiple exchanges, and private peers on a four core ARM CPU with 4 GB of RAM.

How does it take to receive a full table?

How long? If I down all the peers and bring them up it will take 5-10 seconds to rebuild. But if you do that you're scheduling an outage anyway.

I don't think BGP requires real-time. Increasing throughput via multithreading is probably much more impactful (for large deployments) and otherwise it doesn't matter.

A BGP route server might be the platonic ideal of a Go problem: it's demanding enough not to be tenable in Python, but not real-time or kernel resident, and mostly data structure --- and just a couple data structures --- intensive.

> not to be tenable in Python

https://github.com/Exa-Networks/exabgp is written in python

Sorry, I wasn't being precise. Lots of people use BGP for easy lifts, like internal policy routing inside data centers (I find this mystifying but whatever) or simply to advertise Anycast prefixes to upstreams. I was thing more about doing default-free global Internet routing with multiple peers.

I've used exabgp running on a small VM to inject 300,000 /32 routes to multiple peers. Not quite the size of the full v4 routing table, but not too far off either.

Were you taking a feed in? Or just publishing your own addresses?

just injecting routes. There are a bunch of looking glass implementations that use exabgp though.

Go's garbage collector has super short stop the world now, about 1ns IIRC. Most of the work is done concurrently from the main process threads.

> 1ns

Microsecond, perhaps. Likely millisecond. Nanosecond is faster than a single CPU instruction.

I may have missed it, but I read the read me, looked at the main site, looked at the beginning of getting started… nowhere says what BGP stands for / is.

I get it’s written in go, but I have no idea what BGP is. Big green pages, binary grepping pipes, boisterous gambling pachyderms?

Jokes aside, y’all might want to give a brief about what this is for us uninitiated.

In this context, BGP stands for Border Gateway Protocol.

I opened a PR to fix that. https://github.com/osrg/gobgp/pull/2535

I am finding for a long time that what BGP is… The docs even linked Golang without linking BGP. I thought it was something about board games

That’s because the people who are interested already know, and the people who don’t know won’t be interested.

If you DDG, the top ten hits explain it without even clicking. Try that.

Border Gateway Protocol how the internet shares its routing table.

Border Gateway Protocol (https://en.wikipedia.org/wiki/Border_Gateway_Protocol), basically the GPS of the internet (not technically the same, but it's close enough).

Cloudflare has already a good explanation on how it works, so I'll just point everyone out to this article: https://www.cloudflare.com/learning/security/glossary/what-i...

Also, about that undefined BGP. I'm not sure if this will help explain why BGP wasn't explained, but this is from NTT OSRG (Open-source Research Group), so probably they thought that everyone knew what BGP is (since that you probably won't encounter this otherwise).

Also, they have a Rust version: https://github.com/osrg/rustybgp

BGP is pretty foundational to the internet. At some point, you have to assume hour audience knows the basics. Not everything is an intro.

Also, if you don’t know what BGP is, you aren’t going to be interested in a project implementing BGP.

It’s still wise to define acronyms before you use them to enable future Googling (links are better!). Three extra words is a small price to pay to make documentation more friendly to newcomers.

Again, at one point you have to assume either basic knowledge or at least basic knowledge about how to search for things, otherwise everything would be very verbose.

Take one of your repositories as an example: https://github.com/timraymond/pricegrabber

> A gem for interacting with the Pricegrabber API

Should you have to add "Application Programming Interface (API)" the first time you use "API" anywhere? Realistically no, because the audience can be assumed to understand what it means. If they don't, it's only a search away.

Same goes for networking and DNS/BGP/CNAME and more. If you're using a library/server for BGP, you either know what those terms are, or know how to find out more.

But not all resources are meant for beginning who don't know the basic terms in a field, and that's OK, because otherwise everything would be extremely verbose when it doesn't have to.

Of course, you can do whatever you like, but so far everyone who has thrown up such protest over this has done so by literally writing out definitions to show how hard it would be :-/

Does spelling out Border Gateway Protocol even help here?

HTML does have the <ABBR> tag. I use it all the time on my blog any time I use an acronym [1]. At least Firefox and Safari will display the title attribute as a tool tip, so hovering over the following HTML: "<abbr title="Border Gateway Protocol">BGP</abbr>" on those browsers will show "Border Gateway Protocol". I'm sad that it's not used more often.

[1] An example: http://boston.conman.org/2022/02/16.1

I just looked up BGP on DDG. The first ten hits explained it without even clicking on them.

If something isn’t clear on HN, try searching for it. If you don’t find it immediately, then you can ask without losing face.

I’ve been on the internet for quite a long time, but I’ve never heard of it before. So maybe it’s basic knowledge for a given discipline, but maybe not for others?

It’s also, I assume by the authors, submitted to hackernews which is a pretty diverse place. “Basics” is relative.

I am not saying there should be a dissertation about how the thing works, but it’s generally bad form to use abbreviations without using the long form at least once - for people who would indeed like to search for more information.

If you don’t know what BGP is you probably don’t need to know about it . It’s only really needed for configuring large networks.

> large networks

Like the Internet... way to understate it.

BGP can be used for small things too.

Many enterprises use it for their connectivity. If you are multihomed it's basically essential.

Even if you are not multihomed BGP is the most robust way to handle failover between two uplinks to the same ISP. If you are not using full routes and just taking default from each ISP the requirements/load on your router(s) are almost nothing.

Another popular use case is inside private networks, especially over GRE tunnel meshes where the filtering features of BGP allow better/easier control over topology than OSPF.

I've even seen individual servers use BGP for anycasting (usually DNS servers) or http load balancing (with DSR and L4 hashing on the upstream router).

Sorry when I say large I include corporate networks where you want the sort of redundancy that IBGP can give you. I guess Large/medium would be a fair statement.

How can someone know if they're interested or not in an implementation of BGP if they don't know what it is?

I have been a professional software developer for 22 years with a 4 year CS degree from a good school, and I only ever heard of BGP for the first time last year. Most developers never need to know about BGP. People learn new stuff every day.

The point stands. Define your acronyms unless they're really really basic, like HTTP or JSON.

HTTP and JSON may count as "really really basic" to you and BGP may not, but I worked on BGP years before I ever touched HTTP or JSON so the opposite would be true for me. People from different backgrounds are familiar with different acronyms and as a result different acronyms are assumed to be known in different contexts. Don't assume your experience to be universal.

It is good form in technical writing (or any writing, really), to define your acronyms the first time they are used. For example, this would have been helpful:

"GoBGP is an open source Border Gateway Protocol (BGP) implementation..."

Writers should never assume that knowledge of an acronym is universal.

Would you post something like "go-json is an go implementation of JavaScript Object Notation"? Or "Nginx is an Hypertext Transfer Protocol Server"? Please.... It is not that hard to search for "BGP" on the web.

Maybe so, but I wrote about mixing acronyms towards the end of this post: <http://boston.conman.org/2003/11/19.2>, where IRA can stand for one of three different things. My solution: use use <abbr> (<acronym> seems to be deprecated these days).

> Most developers never need to know about BGP.

That is my point. If I have a repo for a BGP implementation, the only people who are going to use my code are going to know what BGP is. No one is going to stumble upon the repo, read about what BGP is, and then decide to use the code. Really, they wouldn’t even be ABLE to use the code… they aren’t going to have access to a router that supports BGP… if you have access to a router with BGP, you already know what BGP is.

I don’t want to sound rude, but I’m shocked that you have a 4 year CS degree and never heard of BGP. Does your degree predate widespread use of the internet? I went to a fairly average state school and both networking courses I took (one required, one elective) covered BGP (at least at a high level).

Well if they started their 22 year career right after they graduated, then they were in school from 1996-2000… BGP was very new, and likely not taught.

Nope, I’d rather HN was not cluttered up with introductory material.

Use DDG if the HN thread doesn’t immediately give you a hint.

It's that thing facebook messed up the other day and nobody could connect to it.

BGP is the de facto routing protocol of the Internet. The basic problem that it solves is computing and distributing routes between independently operated networks. It is actually a pretty ramshackle protocol (among other things it usually uses TCP, which can create some interesting problems [1]) but it is the only widely deployed protocol that can handle the scale of the Internet.

In theory BGP can be used within a single network, though it is not well-suited to such an application except in very large networks. Facebook is known to use BGP internally [2]. Usually an "interior gateway protocol" like OSPF or IS-IS is better for internal routes, with BGP then distributing those routes to the external networks from the gateway routers (i.e. those routers that interconnect the networks). BGP can also be used for other, more niche applications within a single network, like setting up VPNs (VPLS, VPNv4, etc.).

[1] https://blog.benjojo.co.uk/post/bgp-stuck-routes-tcp-zero-wi...

[2] https://research.facebook.com/publications/running-bgp-in-da...

Reply on news.ycombinator.com