Hacker News Reader

Comment by justinsaccount | Hacker News Reader

Google: https://cloudplatform.googleblog.com/2016/03/Google-shares-s...

The design of all 3 is very similar.

I haven't looked at what Google has released, but there are big differences between GLB and Katran. (Not affiliated with any of those companies)

In terms of technology, Katran uses XDP and IPIP tunnels, both upstream in the Linux kernel. GLB uses DPDK, which allows processing raw packets in user space, and Geberic UDP encapsulation + a custom iptables module. Neither DPDK nor the module are upstream.

There are architectural differences as well. Katran is much closer to a classic load balancer, and uses connection tracking at the load balancer to know where to send packets for established flows. GLB has no per-flow state at the load balancer, which gives it the very nice property that load balancers can be added and removed from an ECMP group without disturbing existing connections. There is an academic paper about a system called Beamer, which most likely influenced GLB (or maybe the other way round?). It's a good read, and relatively short.

Finally, Katran is really a C++ library you could build a load balancer on, while GLB comes with batteries included.

I think GLB looks nice, hats off to GitHub for open sourcing it.

> uses connection tracking at the load balancer to know where to send packets for established flows

Kind of, from katran post:

> Each L4LB also stores the backend choice for each 5-tuple as a lookup table to avoid duplicate computation of the hash on future packets. This state is a pure optimization and is not necessary for correctness

> ..

> Katran uses an extended version of the Maglev hash to select the backend server

If the connection tracking state is really not required I don't understand how Katran works. How does it deal with the set of live backends changing? Using Maglev hashing gives you "minimal disruption", not no disruption.

I haven't looked at the code, but this is what they say:

> Katran uses an extended version of the Maglev hash to select the backend server. A few features of the extended hash are resilience to backend server failures, more uniform distribution of load, and the ability to set unequal weights for different backend servers.

GLB also does the same thing with caching:

> The hashing to choose the primary/secondary server is done once, up front, and is stored in a lookup table, and so doesn’t need to be recalculated on a per-flow or per-packet basis.

GLB has that primary/secondary thing which seems to be how it better handles backends coming and going.

Ok, I've looked at the code, and based on that I think what I said in my original post is correct.

GLB is more complex, but will maintain connections in more circumstances.

connection tracking could be turned off. but yeah, but default it uses it (for cases where you need to drain a host w/o distruption of existing connections). it's easy to make katran behave the same way as glb (by disabling connection tracking + writing host module which is doing more or less the same as what glb's agent is doing)

It is unfortunate that Fastly did not open source faild. That's the most elegant solution to draining that I have ever seen.

there is no big difference compare to what glb is doing. they are encoding next server in mac. glb is doign the same w/ metadata inside gue

Fastly's MAC-based solution to this was actually one of the existing implementations we read about back when designing the original implementation of GLB in 2015/16, along with Facebook's IPVS-based solution. We loved the ideas behind Fastly's model, but didn't want to mess with Layer 2 to do it. GLB Director took some inspiration from both designs in the creation of L4 second chance and the L4/L7 split design.

Thanks for the details! I'd love to hear more about any data you have on the efficiency of a "second chance" design vs expanding to three or more failover servers. Very curious if a single alternate is enough to cover majority of incidents (xxx out of 1000 events?) Or how frequently you see failures that fall outside the two chance design decisions.

Keep up the great work!

katran has example of how to use this library. which is (example) more like ipvsadm + kernel.

The Github folks could have used XDP similar as with Katran, I think their solution is very elegant in that the same node would still be able to process other workloads/jobs which is one of the reasons why Facebook decided against DPDK:

https://atscaleconference.com/videos/networking-scale-2018-l...

Reply on news.ycombinator.com