BFD (Bidirectional Forwarding Detection) in OpenBSD [pdf]

I was more surprised that this got to number 2 on the front page, it seemed that it was something people might have actually been waiting for, was wondering why that would be the case.

My day job is networking for a UK ISP/DC, and the only thing I could really think of people wanting this was on devices like route reflectors within your network, or an peering exchange.

However I suppose people are just using OpenBSD to do more than I thought.

Agreed, but generally you'd expect to see OpenBSD sit at the edge of a network (Server, VPN, Firewall), where as BFD typically is more of a Core protocol.

Very interesting, for a network engineer that really does sound like heaven, full layer 3 all the way down to the hosts!

I recently did some Cisco ACI/Nexus work with virtualisation, however the client was only really interested about having massive layer 2 bridge domains spanning across two DC sites to make their App design/failover easier.

What types of problems are you solving in this space? Internal infrastructure, service provider? Do you not need any sort of layer 2 adjacency between your servers? It sounds very nice indeed to be able to resolve your location/identity split at the VM infrastructure layer. I had a very hard time getting customers away from wanting L2 adjacency between entire datacenters in my network consulting career for things like DR in their enterprises...

I'll echo that - although I won't call it new. Using routing protocols at server level, binding a service IP address to the loopback interface and announcing it through an IGP has long been a tool in the infrastructure arsenal. It is especially useful in multi-tenanted services and/or logically partitioned infrastructure, although I first came across it around fifteen years ago whilst building new-wave IP carriers in Europe.

This pattern avoids some of the use cases for overlay networks (especially tunnels, but anything that reduces MTU can still cause problems in this day and age), reduces layer 2 domain scale, aids in service mobility (especially great for anything in a logical partition whether it's a JVM, a jail/zone or a VM), adds multipath resilience for clustered services, gives infrastructure-level visibility on group membership, gives a cheap failover mechanism for intermediate devices like reverse proxies or load balancers, and is simple to extend e.g. more service addresses for new tenants. And yeah, the network guys love it.

If you are very clever it can also extend recursively into the virtual domain. A VM binding a service address to its loopback address, announcing that via an IGP over a virtual interface to the host, which is basically now just a router. This reduces L2 domain size issues for VM farms (albeit by moving them to L3), and solves IP configuration issues e.g. during VM migration to another host or during DR processes. However, the usual Knuth caveats about being very clever apply to this situation. If you do this with, say, a JVM running with docker inside a Xen domU inside a physical host then you get what's coming to you. Or, you could aggregate the announcements at the host if you didn't need the migration part.

The downside of all this is that ops people without ISP or MAN/WAN experience generally need this pattern explained a few times before they get it (some never do - try not to hire those ones though), and practically no *nix distribution or management/config tools support this out-of-the-box, so there's a build & maintenance tradeoff. You can also build things that make some applications get really, really confused. Inadvertently creating an anycasted service might be fun, might also be a recipe for inconsistent application data.